I am building a shared library (we'll call it "foo") that makes use of another library (we'll call it "bar"). "bar" makes use of some functions from OpenSSL.
Here's where the problem surfaces.
"bar" was compiled as a static library and it would appear that OpenSSL was too. So when I link the library ("foo"), I include the:
object files for "foo"
static library libbar.a
OpenSSL static libraries libcrypto.a and libssl.a
The build command looks something like this:
g++ -Wl,-soname,libfoo.so -shared file1.o file2.o libbar.a \
libcrypto.a libssl.a -o libfoo.so
However, I get a ton of errors:
ld: ./obj/libbar.a(file1.c.o): in function initialize_openssl:
ssl.c:117: error: undefined reference to 'SSL_library_init'
Running the following command:
nm libssl.a | grep SSL_library_init
Produces the following output:
00000000 T SSL_library_init
So obviously there is nothing wrong with the OpenSSL libraries. What could have possibly caused something like this? Here are the three commands used to build OpenSSL:
export cross=arm-linux-androideabi-
./Configure android --prefix=~/openssl-arm
make CC="${cross}gcc" AR="${cross}ar r" RANLIB="${cross}ranlib"
The compilation process completed without any errors, so I'm utterly baffled.
Why am I getting linker errors that refer to a bunch of OpenSSL symbols that clearly exist?
The problem was caused by the order of the libraries in the link command. Switching the order of libcrypto.a and libssl.a resolved all of the symbols.
GCC uses LD by default, and its a single pass linker. When you have two libraries, like libssl and libcrypto linked in a particular order, it means libssl depends on symbols from libcrypto. Therefore, libssl must precede libcrypto (or libcrypto must follow libssl). It should be no surprise libssl relies upon libcrypto since libcrypto provides the crypto used by libssl.
Related
I've read this question: ld cannot find shared library even with -L specified, but I'm asking a follow-up: why does GCC do this?
This is something I ran into while building a binary with links to two in-house libraries.
gcc cannot find the symbols from one of the libraries with -l, but uses the other one just fine!
Originally, the command from my Makefile was gcc baz.o qux.o -lfoo -lbar, with the linker unable to find the symbols from libfoo, while finding the symbols from -lbar. The libraries are the exact same type of file in the same locations: headers in /usr/local/include and libraries in /usr/local/lib. In fact, libfoo depends on libbar.
Corrected, the command is now gcc baz.o qux.o -lbar /usr/local/lib/libfoo.so. I have determined this is not an ordering issue.
Why does gcc need the shared object instead of an -l? Is there a better way to do this, other than using the absolute path? This solution seems kludgy to me.
The exact code and output I'm using are as follows:
Output from using -lsandbox
The content of libsandbox.h
The Makefile I'm referencing, annotated.
Thanks!
Currently I am building a foo.h and foo.c with:
$ clang -I . -dynamiclib \
-undefined dynamic_lookup \
-o foo.dylib foo.c
I am able to use this in other C libraries like this:
clang -I . -dynamiclib \
-undefined dynamic_lookup \
-o bar.dylib bar.c foo.dylib
I would like to use this library in an assembly project.
$ nasm -f macho64 test.asm \
&& ld -e start -macosx_version_min 10.13.0 -static -o test test.o foo.dylib
$ ./test
ld: warning: foo.dylib, ignoring unexpected dylib file
Wondering how I link together the C -> asm system to get the C functions working in asm. Then I would like to go further and use that compiled asm to use in either a C or asm project, so wondering how to do that.
When using the assembly in C, I would like for you to basically get functions and import #include "myassembly.h" or something like that, so it feels like a real library. Then you have a function like myfunc which is defined in assembly, but you can use it in c as myfunc(1, 2, 3); sort of thing.
If I change it from static to dynamic linking with the -lSystem flag (and removing -static), I get this:
dyld: Library not loaded: foo.dylib
Referenced from: ./test
Reason: image not found
make: *** [...] Abort trap: 6
You're specifying -static which means:
-static Produces a mach-o file that does not use the dyld. Only used
building the kernel.
dyld is the dynamic loader. If you're not using the dynamic loader, you can't use dynamic libraries.
Update for edited question:
When a dylib is created, it gets an "install name". When an executable is linked to that dylib, the executable stores the install name of the dylib in its reference to it. (Note, it does not store the link-time path of the dylib file it linked against.) When the executable is loaded, the dynamic loader looks for the dylib using the install name it recorded, by default.
You can specify the install name using the -install_name <name> option to the linker. It could be the absolute path to where you expect the library to be installed (e.g. /usr/local/lib/foo.dylib), if you expect it to be installed in a fixed location. Often, though, that's not useful. You want a more flexible means for the dynamic loader to find the dylib.
The dynamic loader understands certain special path prefixes on install names to support such flexibility. See the dyld(1) man page. For example, if you specify an install name of #executable_path/foo.dylib then, at load time, the loader will look next to the executable for the library.
You can see the install name of a dylib by using otool -D foo.dylib. Your dylib may not have an install name, in which case its effective install name is just its file name with no path.
If the loader doesn't find the library by using its install name, it has a search strategy. By default, it looks in ~/lib:/usr/local/lib:/lib:/usr/lib. You can use some environment variables to alter the search strategy. For example, you can set DYLD_FALLBACK_LIBRARY_PATH to a colon-delimited list of directories to search, instead. These environment variables are also listed in the dyld(1) man page.
I am trying to create a dynamic library which is meant to be linked and loaded into a host environment at runtime (e.g. similar to how class loading works in Java). As such, I want the dynamic library to be left with a few "dangling" references, which I expect it to pick up from its host environment when it is loaded into that environment.
My problem is that I cannot figure out how to create the dynamic library without explicitly linking it to existing symbols. I am hoping to produce a dynamic library that does not depend on a specific host executable (or host library), rather one that is able to be loaded (e.g. by dlopen) in any host as long as the host makes a couple symbols available for use.
Right now, any linking command I've tried results in a complaint of missing symbols. I'd like it to allow symbols to be missing (ideally, just particularly specified symbols).
For example, here's a transcript with the error on OS X:
$ cat frotz.c
void blort(void);
void run(void) {
blort();
}
$ cc -c -o frotz.o frotz.c
$ cc -dynamiclib -o libfrotz.dylib frotz.o
Undefined symbols for architecture x86_64:
"_blort", referenced from:
_run in frotz.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
If I do the same thing using a GNU toolchain (on Linux), it helpfully tells me:
$ gcc -shared -o libfrotz.so frotz.o
/usr/bin/ld: frotz.o: relocation R_X86_64_PC32 against undefined symbol `blort'
can not be used when making a shared object; recompile with -fPIC
and indeed, adding -fPIC to the C compile command seems to fix the problem in that environment. However, it doesn't seem to have any effect in OS X.
All the other dynamic-linking questions I could find on SO seem to be about the more usual arrangement of libraries, where a library is being built to be linked into an executable before that executable runs, rather than the other way around. The closest related question I found was this:
Can an executable be linked to a dynamic library after its built?
which unfortunately has very little info, none of it relevant to the question I'm asking here.
UPDATE: I distilled the info from the answer along with everything else I'd figured
out, and put together this example:
https://github.com/danfuzz/dl-example
As far as my knowledge goes, you want to use weak linkage:
// mark function as weakly-linked
extern void foo() __attribute__((weak));
// inform the linker about that too
clang -dynamiclib -o bar.dylib bar.o -flat_namespace -undefined dynamic_lookup
If a weak function can be resolved at runtime, it will then be resolved. If it can't, it will be NULL, instead of generating a runtime (or, obviously, link-time) error.
My build environment is CentOS 5. I have a third party library called libcunit. I installed it with autotools and it generates both libcunit.a and libcunit.so. I have my own application that links with a bunch of shared libraries. libcunit.a is in the current directory and libcunit.so and other shared libraries are in /usr/local/lib/. When I compile like:
gcc -o test test.c -L. libcunit.a -L/usr/local/lib -labc -lyz
I get a linkage error:
libcunit.a(Util.o): In function `CU_trim_left':
Util.c:(.text+0x346): undefined reference to `__ctype_b'
libcunit.a(Util.o): In function `CU_trim_right':
Util.c:(.text+0x3fd): undefined reference to `__ctype_b'
But when I compile with .so like:
gcc -o test test.c -L/usr/local/lib -lcunit -labc -lyz
it compiles fine and runs fine too.
Why is it giving error when linked statically with libcunit.a?
Why is it giving error when linked statically with libcunit.a
The problem is that your libcunit.a was built on an ancient Linux system, and depends on symbols which have been removed from libc (these symbols were used in glibc-2.2, and were removed from glibc-2.3 over 10 years ago). More exactly, these symbols have been hidden. They are made available for dynamic linking to old binaries (such as libcunit.so) but no new code can statically link to them (you can't create a new executable or shared library that references them).
You can observe this like so:
readelf -Ws /lib/x86_64-linux-gnu/libc.so.6 | egrep '\W__ctype_b\W'
769: 00000000003b9130 8 OBJECT GLOBAL DEFAULT 31 __ctype_b#GLIBC_2.2.5
readelf -Ws /usr/lib/x86_64-linux-gnu/libc.a | egrep '\W__ctype_b\W'
# no output
Didn't notice that the libcunit.a is actually found in your case and the problem with linakge is rather in the CUnit library itself. Employed Russian is absolutely right, and he's not talking about precompiled binary here. We understand that you've built it yourself. However, CUinit itself seems to be relying on the symbol from glibc which is not available for static linking anymore. As a result you have only 2 options:
File a report to the developers of CUnit about this and ask them to fix it;
Use dynamic linking.
Nevertheless, my recommendation about your style of static linkage still applies. -L. is in general bad practice. CUnit is a 3rd party library and should not be placed into the directory containing source files of your project. It should be rather installed in the same way as the dynamic version, i.e. like you have libcunit.so in /usr/local/lib. If you'd supply prefix to Autotools on the configure stage of CUnit, then Autotools would install everything properly. So if you want to link statically with CUnit, consider doing it in the following form:
gcc -o test test.c -L/usr/local/lib -Wl,-Bstatic -lcunit -Wl,-Bdynamic -labc -lyz
I'm attempting to do a release of some software and am currently working through a script for the build process. I'm stuck on something I never thought I would be, statically linking LAPACK on x86_64 linux. During configuration AC_SEARCH_LIB([main],[lapack]) works, but compilation of the lapack units do not work, for example undefiend reference to 'dsyev_' --no lapack/blas routine goes unnoticed.
I've confirmed I have the libraries installed and even compiled them myself with the appropriate options to make them static with the same results.
Here is an example I had used in my first experience with LAPACK a few years ago that works dynamically, but not statically: http://pastebin.com/cMm3wcwF
The two methods I'm using to compile are the following,
gcc -llapack -o eigen eigen.c
gcc -static -llapack -o eigen eigen.c
Your linking order is wrong. Link libraries after the code that requires them, not before. Like this:
gcc -o eigen eigen.c -llapack
gcc -static -o eigen eigen.c -llapack
That should resolve the linkage problems.
To answer the subsequent question why this works, the GNU ld documentation say this:
It makes a difference where in the command you write this option; the
linker searches and processes libraries and object files in the order
they are specified. Thus, foo.o -lz bar.o' searches libraryz' after
file foo.o but before bar.o. If bar.o refers to functions in `z',
those functions may not be loaded.
........
Normally the files found this way are library files—archive files
whose members are object files. The linker handles an archive file by
scanning through it for members which define symbols that have so far
been referenced but not defined. But if the file that is found is an
ordinary object file, it is linked in the usual fashion.
ie. the linker is going to make one pass through a file looking for unresolved symbols, and it follows files in the order you provide them (ie. "left to right"). If you have not yet specified a dependency when a file is read, the linker will not be able to satisfy the dependency. Every object in the link list is parsed only once.
Note also that GNU ld can do reordering in cases where circular dependencies are detected when linking shared libraries or object files. But static libraries are only parsed for unknown symbols once.