I'm attempting to write some C bindings for the Go language, and have run into a sort of sticky situation when setting up the Cgo compilation in Windows. I have code that looks like the following:
// #cgo windows CFLAGS: -I C:/dev/extlibs/include/
// #cgo windows LDFLAGS: -lMyLib -L C:/dev/extlibs/lib/
// #include <mylib/mylib.h>
import "C"
which is allows me to avoid installing Dlls, libs, and header files directly into my C:\Windows directory, but doesn't allow for much flexibility when other developers are working with a different file system setup (they all need the libs to be in C:/dev/extlibs/...).
Is there a way I could referent an environment variable from within the code? Perhaps something along the lines of:
// #cgo windows CFLAGS: -I $EXTLIBS$/include/
Or is there another way that people solve this issue that I have missed? I've spent some time googling around on this subject and haven't seen much that has been useful, so any information and/or resources could be a real help!
According to the docs for CGO:
When building, the CGO_CFLAGS, CGO_CPPFLAGS, CGO_CXXFLAGS and
CGO_LDFLAGS environment variables are added to the flags derived from
these directives. Package-specific flags should be set using the
directives, not the environment variables, so that builds work in
unmodified environments.
Using this knowledge, I have had success building a third-party package that wraps a C library so long as it provides it as a system package. The example I linked to:
package sdl
// #cgo LDFLAGS: -lSDL2
// #include <SDL2/SDL.h>
import "C"
Even though it specifies a system package for SDL2, and I have SDL2 installed in some non-system directory, I am able to still build this package using some of the environment variables I mentioned, such as in the following:
export SDL_PATH=/home/mark/where/I/installed/sdl
CGO_CFLAGS="-I$SDL_PATH/include" CGO_LDFLAGS="-L$SDL_PATH/lib" go build hello.go
LD_LIBRARY_PATH="$SDL_PATH/lib" ./hello
Of course, this is Linux, but you can probably use the same ideas in Windows.
You could try using environment variables, the Gentoo Linux Wiki page on Safe C Flags has an example in the following format
CXXFLAGS="${CFLAGS}"
So you may be able to do something like
// #cgo windows CFLAGS: -I "${EXTLIBS}"/include/
but my syntax may be off, and that may be Makefile specific.
You could also try setting a CPATH environment variable which:
specifies a list of directories to be searched as if specified with -I, but after any paths given with -I options on the command line. This environment variable is used regardless of which language is being preprocessed.
The equivalent for -L is, I think, LIBRARY_PATH (Described at the CPATH link).
According to http://golang.org/cmd/cgo/ one sort of recommended way to get around this in a platform independant way is to use pkg-config.
// #cgo pkg-config: mylib otherlib
It's available for windows (http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/) and there's some more information on installing it at this question (How to install pkg config in windows?)
Other than that, put all the dependencies into a sub-directory of the go-code, use relative paths in your CFLAGS and LDFLAGS, and share the entire bundle with other developers.
Related
I wrote a demo using libpq to connect to a PostgreSQL database.
I tried to connect the C file to PostgreSQL by including
#include <libpq-fe.h>
after I added the paths into system variables I:\Program Files\PostgreSQL\12\lib as well as to I:\Program Files\PostgreSQL\12\include and compiled with this command:
gcc -Wall -Wextra -m64 -I "I:\Program Files\PostgreSQL\12\include" -L "I:\Program Files\PostgreSQL\12\lib" testpsql.c -lpq -o testpsql
It first raised three errors, like
libssl-1_1-x64.dll is missing
libintl-8.dll was missing
libcrypto-1_1-x64.dll was missing
After I downloaded these three files and put them into I:\Program Files\PostgreSQL\12\lib, and compiled it again, it shows the error
The application was unable to start correctly (0xc0150002)
when I type testpsql. But if I type ./testpsql on git bash, it works. Anyone can please tell me why?
The code that I used was the first example from here.
Environment: PostgreSQL 12, Windows 10, MinGW64
“Download the DLL files” sounds dangerous. From where?
I would get rid of these files again. Since you probably don't reference these libraries from your code, it must be the dependencies of libpq.dll and are probably found in I:\Program Files\PostgreSQL\12\bin (if you used the EDB installer).
The problem is probably that you the PATH environment variable is different in git bash and in cmd.exe, and in the latter case not all required shared libraries can be found on the PATH. The solution is to change the PATH so that it includes all DLL files the executable requires, not to start copying around files.
It is probably enough to include I:\Program Files\PostgreSQL\12\bin in the PATH. To resolve missing dependencies, use a tool like dependency walker or this replacement.
There are various Go libraries that rely on the cuda.h file and the cuda library (specifically ML libraries). Every time I try to install one of these libraries on Windows, I get an error saying
fatal error: cuda.h: No such file or directory
//#include <cuda.h>
I am aware of what I need to do (link the Cuda library/header files to the go library that I am trying to install), however, I am not sure how to go about doing this especially on windows. I am using GCC and not MSVC for various reasons, but even when I've tried using MSVC, I've had the same issues.
Is there some way that I can link the cuda compiler/header files directly to my Go env or do I need to manually point the go/cgo compiler to the directory holding the Cuda headers and how do I go about doing this?
I've tried asking a few of the developers who make these libraries for help but most of them are linux users so they don't really know. An exhaustive google search has really lead me nowhere so I'm asking here.
I was able to find the answer.
The windows cuda installer installs things by default into a weird path:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\include
This path caused a lot of pain for the compiler as a result of the spaces in the folder names. After reinstalling Cuda into C:\CUDA\v8.0 and then appending my CFLAGS and LDFLAGS appropriately in my CGO file I was able to get things to run correctly.
For reference here are the CFLAGS and LDFLAGS that I used to get this to work:
//#cgo windows LDFLAGS:-LC:/cuda/v8.0/lib/x64
//#cgo windows CFLAGS: -IC:/cuda/v8.0/include
import "C"
This was with the github.com/chewxy/cu go library. I also appended the new flags into that library in the cgoflags.go file because the maintainer did not have version 8 or version 9 in there already. I mentioned this to him and he might update it later but for now that's what you have to do.
I hope this is a basic question. I am trying to build a Go package which includes functions from a library written in C. The structure is basically as follows:
package too
/*
#cgo LDFLAGS: -L/usr/local/lib include -lbar
#include mybar.h
*/
import "C"
func MyGoWrapper () {
C.orig_func()
}
Running go build foo.go fails with an "undefined reference" for orig_func. Note that the header is mybar.h; I created a prototype for orig_func that was not included in the original library. Do I need to recompile the library first, including this header file, before it will link with the Go build? Or am I misunderstanding something else entirely?
When linking against an external library, you do need to separately compile it for your target architecture. cgo can't replace the configure/make (or whatever) to compile the library; it only knows how to build a few .c files in your package directory, and a library's build process might be more complex.
I'm less sure of how to accomplish the larger task of linking in an external library when cross-compiling (and I'm not sure what you've already done). The (closed) Go bug on cross-compilation with cgo looks useful here. You may want to build the Go toolchain with some environment variables set that are described in godoc cmd/cgo:
To enable cgo during cross compiling builds, set the CGO_ENABLED
environment variable to 1 when building the Go tools with make.bash.
Also, set CC_FOR_TARGET to the C cross compiler for the target. CC will
be used for compiling for the host.
After the Go tools are built, when running the go command, CC_FOR_TARGET
is ignored. The value of CC_FOR_TARGET when running make.bash is the
default compiler. However, you can set the environment variable CC, not
CC_FOR_TARGET, to control the compiler when running the go tool.
CXX_FOR_TARGET works in a similar way for C++ code.
The bug also mentions someone who uses -ldflags="-extld=$(CC)" (where $(CC) is the name of the cross-compiler they want to use).
In your example code there's an explicit -L/usr/local/lib and I don't think that'll work: I think when you build libraries for the target, you're going to want to put them in a directory distinct from the lib for your host arch. For example, this ARM cross-compilation HOWTO uses a /usr/local/arm-linux prefix or install_root in some places.
My issue started identical to this one: Python executable not finding libpython shared library
I updated .bashrc with export LD_LIBRARY_PATH=$HOME/local/lib/python/2.7.6/lib and things were fine. Python works, and I installed pip. But now, I'm running into something similar when installing cython with pip. I get this error message when I execute pip install cython:
gcc -pthread -shared build/temp.linux-x86_64-2.7/tmp/pip_build/cython/Cython/Plex/Scanners.o -L. -lpython2.7 -o build/lib.linux-x86_64-2.7/Cython/Plex/Scanners.so
/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
I cannot add $HOME/local/lib/python/2.7.6/lib to /etc/ld.so.conf and run ldconfig as I do not have root. I was under the impression that setting the LD_LIBRARY_PATH was the way around this, but this appears to not be true for compilation. Is there a way to get the compiler to see this local library without running root commands?
Update:
The LD_LIBRARY_PATH is only used by the dynamic loader at runtime, not at build time, so that is not the issue. The issue is that you forgot to put the -L/path/to/pylib before the -l. I've never had to use LIBRARY_PATH because a build requires path extension that is specific to a given build, so you never set LIBRARY_PATH you just use -L. You would only set if if you are going to regularly do builds that use a specific library, and even then I find it better to use -L because sooner or later this will cause linker to find the wrong lib and by then you will have forgotten that it's because LIBRARY_PATH is set permanently.
There are many ways to set -L values in a build: if you run the compiler from command line you don't need that env var, you just specify as many -L as required as part of the command; if you use a makefile, you edit whatever make variable you are using, such as CFLAGS or other, different platforms have different conventions. So whereas setting -L directly will always work, setting CFLAGS will only work if that is the variable used by the makefile.
Now this is a python installation so where to set this may not be obvious, but I am sure there is another way than setting LIBRARY_PATH. In principle any python package you install, if it involves compilation of C++ modules, could require edit of the setup.py to set library paths. For example
Extension(...,
library_dirs=['/usr/X11R6/lib'],
...)
Since you mention nympy, another place to set this might be in site.cfg (see Supplying NumPy site.cfg arguments to pip).
Old (wrong) answer:
Set your LD_LIBRARY_PATH in your bash console. If this doesn't work then it's because you have the wrong path: check by echoing the environment var.
Once you get that to work, edit your .bashrc or .profile then exit your shell and restart it. Echo the env var to verify that contains the part you added.
Also, ensure that you are appending to the path rather overwriting it:
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/...
export LD_LIBRARY_PATH
Because python lib might depend on .so in other folders, if the linker can't find them it may appear as though it is the python lib that was not found. This is not explained on the page you linked to in your question.
OK after some more digging I found this: LD_LIBRARY_PATH vs LIBRARY_PATH
Setting LIBRARY_PATH to the same path as LD_LIBRARY_PATH made the compiler aware of the python lib. cython/numpy/scipy all built and installed no problem afterwords.
I have a c program which needs a library named libnuma to be installed. But I dont have root access in the parallel machine in which I need to run this program. So I downloaded the source code of libnuma and compiled it. I have a libnuma.a file which i assume is the library. I need to link this library with the c program that I have. This program uses autotools for generating the configuration files and the makefile. I am new to autotools. Please tell me what I have to do to link this library without being root.
Ajay.
It should be sufficient to set CPPFLAGS and LDFLAGS. First, try:
$ ./configure LDFLAGS=-L/path/to/lib CPPFLAGS=-I/path/to/include
(where libnuma.a is /path/to/lib/libnuma.a and numa.h is /path/to/include/numa.h.
That is, specify the directories.) If that does not work, check config.log to see what went wrong. If the configure script for the program you are using was built with an old version of autoconf, you may need to do:
$ LDFLAGS=-L/path/to/lib CPPFLAGS=-I/path/to/include ./configure
instead. (The second invocation will only work in Bourne shells. With csh/tcsh, you will need to set the environment variables some other way, for example with env.) You also have the option of making those settings in the environment of your shell (eg, in a .bashrc) or in a config.site file.