Using linker command line options passed via 'node-gyp' I specify that the library path and library name that I want the program to link with. But the resulting executable does not reference the file I specified, it references a different name in /usr/lib.
I'm using the libraries section in binding.gyp to reference a local lib directory.
'libraries': [
'-lao-oboe',
'-L<(module_root_dir)/lib/',
'-Wl,-rpath-link,<(module_root_dir)/lib/',
'-Wl,-rpath,<(module_root_dir)/lib/'
],
node-gyp seems to be passing the options correctly because the linker returns /usr/bin/ld: cannot find -la-oboe if I change the -L path to one that doesn't contain libao-oboe.so. The linker also returns an error if I change the name of the requested library to be different than the one in lib.
The problem is that the local library will not be loaded at runtime. ldd shows that the node-gyp output file is not referencing the file that was specified - it is referencing a library with a different name altogether - /usr/lib/liboboe-1.0.so.1. See the second line of ldd output:
linux-vdso.so.1 => (0x00007ffee20f5000)
liboboe-1.0.so.1 => /usr/lib/liboboe-1.0.so.1 (0x00007fa476377000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa475ff5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa475c2b000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa475a27000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa47580a000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa475501000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa476922000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa4752eb000)
The local library directory contains:
lrwxrwxrwx 1 bruce bruce 15 Sep 8 02:50 libao-oboe.so -> libao-oboe.so.1`
-rw-r--r-- 2 bruce bruce 1640848 Aug 31 15:01 libao-oboe.so.1
It is the case that the local library file, libao-oboe.so.1 is the same as the system library file being referenced in the executable (as shown by ldd): /usr/lib/liboboe-1.0.so.1.
Does the linker somehow notice that the local file is the same (via hashing or some signature) and substituting the library file from the standard location?
Why does node-gyp's output file reference a library file that was never requested as part of the build process?
According to wikipedia - soname the SONAME field in the .so file is the "base compatible version". It's clear from the behavior I found in the problem above that ld inserts the SONAME into the file being linked to the shared library - not the filename specified in the ld command.
Renaming a .so file doesn't change the SONAME so an executable linked to a renamed file will try to load a library named by the SONAME field, not the filename.
My solution was to not rename the file.
Related
I'm using gcc to build a portable shared object. I'm applying the technique outlined in this answer to ensure that my binary will work on systems with older versions of glibc.
For the symbol pthread_getattr_np, I use
extern "C" {
__asm__(".symver pthread_getattr_np,pthread_getattr_np#GLIBC_2.2.5");
int __wrap_pthread_getattr_np(pthread_t thread, pthread_attr_t* attr) {
return pthread_getattr_np(thread, attr);
}
}
along with the option -Wl,--wrap=pthread_getattr_np to link to a specific portable version. However, pthread_getattr_np recently migrated from libpthread to libc for newer version of glibc. Even though the versioned symbol pthread_getattr_np#GLIBC_2.2.5 is the same, it's defined in a different library.
When I try to load my binary on an older system, I get the error
error: unable to load shared object
symbol pthread_getattr_np version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference
If I run
objdump -T /lib/x86_64-linux-gnu/libpthread.so.0 | grep pthread_getattr_np
0000000000009420 g DF .text 000000000000035e GLIBC_2.2.5 pthread_getattr_np
I see that the symbol is defined in libpthread.so; and if I run ldd on my binary, I see that libpthread is a dependency
linux-vdso.so.1 (0x00007ffec17fc000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa3725cd000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa3723c9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa37202b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa371c3a000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa3735dd000)
but the symbol won't resolve correctly.
Is there a way to set up the linking for my shared library so that pthread_getattr_np will resolve to either libpthread or libc depending on where it's defined?
First I describe what I have done.I am using cygwin on windows 7 64 bit.
I first just created simple program in C say famous hello program.Then I compiled using 'gcc' directly to generate 'hello.exe' .Then I checked the dynamically linked libraries using 'ldd' command and checked their paths.
$ ldd hello.exe
ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77a70000)
kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll (0x77850000)
KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll (0x7fefdb40000)
cygwin1.dll => /cygdrive/c/Users/vishu/cygwin1.dll (0x180040000)
Then I created object file 'hello.o' instead of generating executable and then used linker to link object files and the shared libraries to create 'hello1.exe'.But when I tried to run it it showed no output .What I am missing?
Note : When I checked using 'file' command for both the files then output are
$ file hello.exe
hello.exe: PE32+ executable (console) x86-64, for MS Windows
$ file hello1.exe
hello1.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
Edit for comments : I do not think that I have written wrong commands as the shared libraries for two executables is the same.I think the type of two executables is different but I do not know what 'stripped to external PDB' means.But as you are asking I will post them below.I copied all dll to same directory as object file hello.o (It is same object file for hello.exe and hello1.exe).
$ ld -o hello1.exe hello.o ntdll.dll kernel32.dll KERNELBASE.dll cygwin1.dll
Sorry if this is too silly question.
This is a more general question. I know windows DLL's can have a resource file set up with the dll version information, but I'm wondering how to do the same for linux shared objects.
The problem I'm encountering is actually when running just about anything at the terminal, I get a message about libz.so.1 version information not being available. This is due to an application being present with its own version of libz that I've compiled. The library is actually libz.so.1.2.3 and the same version exists in /lib. The files are actually the SAME version of the library, but one of them (which I compiled) says it's missing version information.
So, that leads me to wonder how to actually include the version information in the binary rather than just in the file name. It would be ideal if there's a solution like
./configure .... some_version_option=1.2.3
If I use the working version of the library:
ldd /usr/bin/git
linux-vdso.so.1 => (0x00007fffdfbff000)
libz.so.1 => /lib64/libz.so.1 (0x00007f3797fa7000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c56000000)
libc.so.6 => /lib64/libc.so.6 (0x0000003c55c00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003c55400000)
If I use the version I compiled:
ldd /usr/bin/git
/usr/bin/git: libz.so.1: no version information available (required by /usr/bin/git)
linux-vdso.so.1 => (0x00007fff872b1000)
libz.so.1 (0x00007f83c9270000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c56000000)
libc.so.6 => /lib64/libc.so.6 (0x0000003c55c00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003c55400000)
You need to pass the library's symbol version map file with -Wl,--version-script <file.map>. The map file should be included in the library source.
How does ldd knows it's depending on libc.so.6 ,not libc.so.5 or libc.so.7?
libc.so.6 => /lib64/libc.so.6 (0x00000034f4000000)
/lib64/ld-linux-x86-64.so.2 (0x00000034f3c00000)
It is recorded inside application binary itself (specified at compile time, more exactly at link step, done with ld):
$ readelf -d /bin/echo
Dynamic section at offset 0x5f1c contains 21 entries:
Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libc.so.6]
...
(there are some additional columns for how elf does store information in dynamic section. but you can see that libc.so.6 is hardcoded with .6 suffix because of SONAME)
or even without any knowledge of ELF file format:
$ strings /bin/echo |grep libc.so
libc.so.6
To find, how does linker find a library (it is done at final step of compilation), use gcc option -Wl,--verbose (this asks gcc to pass option --verbose to ld):
$ gcc a.c -Wl,--verbose
...
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.4.4/libc.so failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.4.4/libc.a failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.4.4/libc.so failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.4.4/libc.a failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.4.4/../../../libc.so succeeded
opened script file /usr/lib/gcc/i686-pc-linux-gnu/4.4.4/../../../libc.so
opened script file /usr/lib/gcc/i686-pc-linux-gnu/4.4.4/../../../libc.so
attempt to open /lib/libc.so.6 succeeded
/lib/libc.so.6
Linker doesn't know anything about .digit suffix, it just iterate over all library search directories trying to open libLIBNAME.so and libLIBNAME.a, where LIBNAME is a string after -l option. ( -lc option is added by default).
First success is /usr/lib/libc.so which itself is not a library, but a linker script (text file). Here is content from typical libc.so script:
$ cat /usr/lib/libc.so
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-i386)
GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /lib/ld-linux.so.2 ) )
So, script /usr/lib/libc.so is found earlier than actual library, and this script says, what file will be linked, libc.so.6 in this case.
In more common case, lib___.so is symlink to some version like lib___.so.3.4.5 and there is SONAME field filled in lib___.so.3.4.5 which says to ld link not to lib___.so but to lib___.so.3.4 which is another symlink to lib___.so.3.4.5. The .3.4 name will be recorded in NEEDED field of binary.
http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#dynamic_section
Has the meaning of each dynamic tags. The 1 indicates it is a DT_NEEDED tag meaning in this case the
typedef struct {
Elf32_Sword d_tag;
union {
Elf32_Word d_val;
Elf32_Addr d_ptr;
} d_un;
} Elf32_Dyn;
structure has d_val union valid and look up at an offset specified by thi union member in DT_STRTAB table to find the name of library that this binary/SO depends on.
I'm trying to compile samtools on a Solaris server where I do not have root. Samtools depends on zlib. The system zlib on this machine is not compiled with large file support, so compiling samtools against this version has the expected effect: samtools only handle small files. I need it to be able to handle large files. Luckily, there is a version of zlib compiled by the admin in /usr/local/apps/zlib-1.2.5/ with large file support. I can compile against this by adding -R /usr/local/apps/zlib-1.2.5/lib to CFLAGS, but this seems not to work. The symptoms are as follows:
When I try to run samtools, it crashes with this error:
ld.so.1: samtools: fatal: relocation error: file samtools: symbol gzopen64: referenced symbol not found
If I add /usr/local/apps/zlib-1.2.5/ to LD_LIBRARY_PATH, then samtools works fine.
Analyzing samtools with ldd and readelf yields the following:
$ ldd -r samtools
libnsl.so.1 => /usr/lib/libnsl.so.1
libsocket.so.1 => /usr/lib/libsocket.so.1
libresolv.so.2 => /usr/lib/libresolv.so.2
libm.so.2 => /usr/lib/libm.so.2
libcurses.so.1 => /usr/lib/libcurses.so.1
libz.so => /usr/lib/libz.so
libc.so.1 => /usr/lib/libc.so.1
libmp.so.2 => /usr/lib/libmp.so.2
libmd.so.1 => /usr/lib/libmd.so.1
libscf.so.1 => /usr/lib/libscf.so.1
libdoor.so.1 => /usr/lib/libdoor.so.1
libuutil.so.1 => /usr/lib/libuutil.so.1
libgen.so.1 => /usr/lib/libgen.so.1
symbol not found: gzopen64 (samtools)
$ ldd -s samtools
...(snip)...
find object=libz.so; required by samtools
search path=/usr/lib:/usr/openwin/lib:/usr/dt/lib:/usr/local/lib (LD_LIBRARY_PATH)
trying path=/usr/lib/libz.so
libz.so => /usr/lib/libz.so
...(snip)...
$ readelf -d samtools | grep RPATH
0x0000000f (RPATH) Library rpath: [/usr/local/apps/zlib-1.2.5/lib:/usr/local/apps/gcc-4.5.1/lib]
So /usr/local/apps/zlib-1.2.5/lib is clearly in the binary's RPATH, which I understand is supposed to be searched at runtime for shared libraries. However, ldd -s shows that this directory is never searched. Adding this path to LD_LIBRARY_PATH and re-running the ldd commands has the expected effect: the directory is searched and the correct version of libz is found.
So how can I force samtools to search in /usr/local/apps/zlib-1.2.5/lib at runtime without using LD_LIBRARY_PATH?
Edit: The documentation here would seem to indicate that the -R option is the correct thing to do. But it doesn't work.
I'm by no means a Solaris expert, but this line:
find object=libz.so; required by samtools
search path=/usr/lib:/usr/openwin/lib:/usr/dt/lib:/usr/local/lib (LD_LIBRARY_PATH)
seems to indicate to me that LD_LIBRARY_PATH is already set, and the /usr/lib path in it is taking precedence over any runtime linker paths. Can you unset LD_LIBRARY_PATH if it is in fact present and see if that resolves it?