I am having difficulty importing a python extension module that depends on a separate shared library that I am building via a separate build system. When I attempt to import the module, I get an undefined symbol error:
import ext_mod
ImportError: /ext_mod_path/ext_mod.so: undefined symbol: __mangled_func_name
This makes a certain amount of sense, since it's missing from the extension module shared library, as it's contained in the separate shared library.
$ nm /ext_mod_path/ext_mod.so | grep __mangled_func_name
U __mangled_func_name
But the extension module library can successfully find the shared common library:
$ ldd /ext_mod_path/ext_mod.so
libmodcommon.so => /user_lib_path/libmodcommon.so (0x...)
And the symbol is defined there
$ nm /user_lib_path/libmodcommon.so | grep __mangled_func_name
000000000002e750 T __mangled_func_name
Why can't the symbol be seen in ext_mod.so?
For reference, the linker command for the extension module is:
g++ -o ext_mod.so -shared -Wl,-rpath=/user_lib_path ext_mod.os -L/user_lib_path -lmodcommon
Related
I am compiling files a.c & b.c with flag -fstack-protector-strong which results to introduce new symbol __stack_chk_fail in object files a.o, b.o
nm a.o | grep __stack_chk_fail
U __stack_chk_fail
nm b.o | grep __stack_chk_fail
U __stack_chk_fail
I am creating one static library libstat.a using above object files :
ar rc libstat.a a.o b.o
Finally, I am trying to create dynamic library libtest.so using above static library. The above symbol __stack_chk_fail is defined in library libssp.so which is in gcc tool chain path /home/test_usr/gcc-10.3.0/aarch64-unknown-linux-gnu/lib64
objdump -T /home/test_usr/gcc-10.3.0/aarch64-unknown-linux-gnu/lib64/libssp.so.0.0.0 | grep __stack_chk_fail
0000000000000ff0 g DF .text 0000000000000020 LIBSSP_1.0 __stack_chk_fail
Build command:-
bin/gcc -include ... -L /home/test_usr/gcc-10.3.0/aarch64-unknown-linux-gnu/lib64/ -lssp -o libtest.so libstat.a
i was expecting to have the symbol defined in my final library libtest.so, But no luck.
nm libtest.so | grep __stack_chk_fail
U __stack_chk_fail#LIBSSP_1.0
Am i missing some thing here? Why this symbols is not getting defined though i try to link with -lssp which has this symbol definition. Please help!
No, this is not how shared libraries work. If the symbol is defined in some .so it does not "migrate" to every library or executable that gets linked with it. You are thinking of static libraries, not dynamic libraries.
A static library, a .a file, consists of multiple object modules with exported symbols. Any symbol that's referenced by whatever you link with the shared library results in its module being copied out of a static library and into an executable or another library that gets linked with it.
A shared library (or a dynamic library as you call them), a .so file, does not work that way.
The library gets loaded at run time, and at runtime all unresolved symbols in your executable or library get resolved with the symbols exported from the shared library, so nothing needs to get copied out of the shared library.
A static library does not get loaded at runtime, and does not need to be installed, because everything that's needed gets extracted from the static library, by your linker, and included in the executable or library that gets linekd with the static library.
The whole reason to use shared libraries, .sos, is to not bloat everything that gets linked with the library with the code from the linked library. The code, and all exported symbols, remain in the .so file, they do not get "defined" or added into whatever gets linked with it.
On our project at work, where I use a build system managed by another team, I am running into an issue where I am able to compile and link the binary, but then when executing it, it complains about being unable to load a shared object file, it should never have been linked against:
/home/me/build/bin/executable: error while loading shared libraries: libicui18n.so.52.1: cannot open shared object file: No such file or directory
The curious part is, that the only instance of the file name libicui18n.so.52.1 I can even find on my machine are /opt/onlyoffice/desktopeditors/libicui18n.so.52.1 and /home/me/.local/opt/foxitsoftware/foxitreader/lib/libicui18n.so.52.1, which I definitely don't link against, at least according to linker command executed by the build system:
It does not occur as any of the explicit /absolute/path/to/libsomething.so.42.3 files.
Nothing related is given as -L option.
Nothing related is given as /absolute/path/to/libsomething.a file.
Nothing related is in $LD_LIBRARY_PATH or $LIBRARY_PATH.
The Intel Fortran compiler has no reason to include those in the default paths.
So now I am wondering, how else the libicui18n.so.52.1 may be linked against, and how I may go about debugging such an issue.
how else the libicui18n.so.52.1 may be linked against
ELF libraries can specify "what name should be used at runtime" independently of the name of the library itself. Example:
gcc -fPIC -shared -o libt.so t.c -Wl,--soname=libfoo.so.1.2.3
gcc main.c -L. -lt
./a.out
./a.out: error while loading shared libraries: libfoo.so.1.2.3: cannot open shared object file: No such file or directory
Here, the library is built in a way which requires libfoo.so.1.2.3 at runtime, although no such library exists. It is up to the packaging system to provide such library (or a symlink).
You can examine a library to see what name it expects to be used at runtime like so:
readelf -d libt.so | grep SONAME
0x000000000000000e (SONAME) Library soname: [libfoo.so.1.2.3]
You can examine the list of libraries a.out requires at runtime like so:
readelf -d ./a.out | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libfoo.so.1.2.3]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
how I may go about debugging such an issue.
As above. Your next question is probably "how can I fix this issue?".
Since libicui18n.so.52.1 is installed, but not installed into a directory which the dynamic linker will search by default, all you have to do is tell your binary to add the non-standard /opt/onlyoffice/desktopeditors directory the list of directories to search.
You do that by adding -Wl,-rpath=/opt/onlyoffice/desktopeditors to your executable's link line.
You can also add /opt/onlyoffice/desktopeditors to LD_LIBRARY_PATH, but using -Wl,-rpath is preferred.
I am building a shared library (we'll call it "foo") that makes use of another library (we'll call it "bar"). "bar" makes use of some functions from OpenSSL.
Here's where the problem surfaces.
"bar" was compiled as a static library and it would appear that OpenSSL was too. So when I link the library ("foo"), I include the:
object files for "foo"
static library libbar.a
OpenSSL static libraries libcrypto.a and libssl.a
The build command looks something like this:
g++ -Wl,-soname,libfoo.so -shared file1.o file2.o libbar.a \
libcrypto.a libssl.a -o libfoo.so
However, I get a ton of errors:
ld: ./obj/libbar.a(file1.c.o): in function initialize_openssl:
ssl.c:117: error: undefined reference to 'SSL_library_init'
Running the following command:
nm libssl.a | grep SSL_library_init
Produces the following output:
00000000 T SSL_library_init
So obviously there is nothing wrong with the OpenSSL libraries. What could have possibly caused something like this? Here are the three commands used to build OpenSSL:
export cross=arm-linux-androideabi-
./Configure android --prefix=~/openssl-arm
make CC="${cross}gcc" AR="${cross}ar r" RANLIB="${cross}ranlib"
The compilation process completed without any errors, so I'm utterly baffled.
Why am I getting linker errors that refer to a bunch of OpenSSL symbols that clearly exist?
The problem was caused by the order of the libraries in the link command. Switching the order of libcrypto.a and libssl.a resolved all of the symbols.
GCC uses LD by default, and its a single pass linker. When you have two libraries, like libssl and libcrypto linked in a particular order, it means libssl depends on symbols from libcrypto. Therefore, libssl must precede libcrypto (or libcrypto must follow libssl). It should be no surprise libssl relies upon libcrypto since libcrypto provides the crypto used by libssl.
I'm trying to use methods contained in a shared library (libscplugin.so).
I have satisfied all of the libraries requirements:
libc.so with a symlink to libc.so.6
libz.so with a symlink to libz.so.1.2.8
libstdc++.so with a symlink to libstdc++.so.6.0.20
Upon compilation I get the following error message:
$ gcc test.c -o test -L/usr/lib/arm-linux-gnueabihf/ -lscplugin
/usr/bin/ld: test: hidden symbol `stat' in /usr/lib/arm-linux-gnueabihf/libc_nonshared.a(stat.oS) is referenced by DSO
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
The only reference I could find to libc_nonshared.a is in /usr/lib/arm-linux-gnueabihf/libc.so:
$ cat libc.so
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/arm-linux-gnueabihf/libc.so.6 /usr/lib/arm-linux-gnueabihf/libc_nonshared.a AS_NEEDED ( /lib/arm-linux-gnueabihf/ld-linux-armhf.so.3 ) )
In test.c I'm attempting to use two functions of the shared library (totally unrelated to stat). What can I do to get this to compile ?
Your problem is here:
I have satisfied all of the libraries requirements:
libc.so with a symlink to libc.so.6
That is not a valid setup for glibc. libc.so is supposed to be a text file (linker script) as you saw at the end of your answer. You're doing something weird and unnecessary if you're making symlinks to shared libraries like this yourself. Use the libc.so that's provided (and you need to do this when building your other shared libraries -- your problem right now is that they were mis-linked) and everything will work fine.
In Linux, why are stubs required for standard libraries?
stubs are required to ensure proper linking of an executable across various linux releases without building the object files.
For example:
Let a be the executable we are building:
gcc -o a test.o test1.o test2.o -lz
In the above case executable a has a dependency on the libz.so (-lz is to link with libz.so). The linker resolves libz.so using the LD_LIBRARY_PATH.
Now let us see the problem:
In RHEL4(Linux Zseries):
objdump -T /usr/lib64/libz.so.1 | grep stack_chk
In RHEL5(Linux ZSeries);
objdump -T /usr/lib64/libz.so.1 | grep stack_chk
0000000000000000 DF UND 0000000000000031 GLIBC_2.4 __stack_chk_fail
In RHEL5, we see an undefined symbol in the libz.so.
Unless we pass libc to the linker after lz in the above command, this cannot be resolved.
Stubs:
If we generate the stub for libz.so, packing all the symbols of libz.so in to a stub library and link with the stub library instead of the real library, we don't see any errors:
Modified link line:
gcc -o a test.o test1.o test2.o -L/home/lib/stubs/ -lz
In the /home/lib/stubs directory, we have a stub library for libz.so by name libzstub.so.
Linker gives higher priority to the path given in the link command than LD_LIBRARY_PATH.
Now even if we link in the RHEL5 release, the linker resolves the symbols for the libz.so from the /home/lib/stubs directory.
Here the configuration details of the boxes i have used.
Loader takes care of loading the coresponding function at runtime.
RHEL5:
libcmpiutil-0.4-2.el5
glibc-utils-2.5-42
libc-client-2004g-2.2.1
libcap-1.10-26
libcap-1.10-26
libchewing-devel-0.3.0-8.el5
libchewing-0.3.0-8.el5
libcxgb3-1.2.3-1.el5
libcap-devel-1.10-26
glibc-common-2.5-42
libcxgb3-static-1.2.3-1.el5
libcroco-devel-0.6.1-2.1
compat-glibc-headers-2.3.4-2.26
libcroco-0.6.1-2.1
compat-libcom_err-1.0-7
libcmpiutil-devel-0.4-2.el5
compat-glibc-2.3.4-2.26
glibc-headers-2.5-42
glibc-devel-2.5-42
libcap-devel-1.10-26
libc-client-2004g-2.2.1
libcmpiutil-0.4-2.el5
libcroco-0.6.1-2.1
libc-client-devel-2004g-2.2.1
glibc-2.5-42
libchewing-devel-0.3.0-8.el5
libcroco-devel-0.6.1-2.1
compat-libcom_err-1.0-7
libc-client-devel-2004g-2.2.1
libchewing-0.3.0-8.el5
libcxgb3-1.2.3-1.el5
libcmpiutil-devel-0.4-2.el5
glibc-2.5-42
glibc-devel-2.5-42
compat-glibc-2.3.4-2.26
RHEL4:
rpm -qa | grep libc
glibc-2.3.4-2.41
libcxgb3-1.1.4-1.el4
libc-client-2002e-14
libcroco-0.6.0-4
libcap-devel-1.10-20
glibc-kernheaders-2.4-9.1.103.EL
compat-libcom_err-1.0-5
glibc-devel-2.3.4-2.41
compat-glibc-2.3.2-95.30
compat-libcom_err-1.0-5
glibc-common-2.3.4-2.41
libcroco-devel-0.6.0-4
libcxgb3-1.1.4-1.el4
libc-client-2002e-14
glibc-utils-2.3.4-2.41
libcap-1.10-20
glibc-headers-2.3.4-2.41
glibc-profile-2.3.4-2.41
libcxgb3-static-1.1.4-1.el4
glibc-devel-2.3.4-2.41
compat-glibc-2.3.2-95.30