I'm trying to compile a program, I need to set a prefix path on shared library path, I try using -Wl,-rpath -Wl,-dynamic-linker and what I got from ldd was:
linux-vdso.so.1 => (0x00007fff75336000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f19d55b8000)
/pathtolib/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x000055d3d67e1000)
but what I expecting to get was:
linux-vdso.so.1 => (0x00007fff75336000)
libc.so.6 => /pathtolib/lib/x86_64-linux-gnu/libc.so.6 (0x00007f19d55b8000)
/pathtolib/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x000055d3d67e1000)
I really need to setup this and force program to use non-default path (/pathtolib) for all shared libraries.
this is the command I used to compile:
gcc list/list.c -o bin/list -Wl,-rpath,/pathtolib -Wl,--dynamic-linker,/pathtolib/lib64/ld-linux-x86-64.so.2
what did I do wrong?!! :|
The -rpath option does not specify a prefix for individual directories in the search path; rather, it specifies a colon-delimited list of paths. The dynamic linker searches these directories first, but it continues on to the rest of the search path if it needs to do. Thus, if you want the dynamic linker to resolve libc.so.6 to /pathtolib/lib/x86_64-linux-gnu/libc.so.6, then you must specify not /pathtolib but /pathtolib/lib/x86_64-linux-gnu in your rpath.
I am not aware of a link option or ELF header field that would instruct the dynamic linker to do what you seem to be asking: to modify the standard library search path by prefixing each element. If you want to do something like that, then you probably need to provide your own dynamic linker.
Alternatively, perhaps you want to construct and use a chroot environment instead of of messing with rpaths. That requires considerably more work to set up, but it would have the effect of allowing you to use a whole different set of libraries.
Related
I'm trying to gather all the dependencies needed by some .so file. I use the Recursive ldd script, but that doesn't really matter for the manner of sake.
I want to put all the .so files in one directory, say it's in
/home/user/project/lib
I'm having a weird experience: Say there's a file libmat.so which I want to gather all its dependencies. So I ran
/home/user/project/lib$ ldd libmat.so
...
libmwboost_system.so.1.65.1 => /home/user/project/lib/libmwboost_system.so.1.65.1
libmwboost_filesystem.so.1.65.1 => /home/user/project/lib/libmwboost_filesystem.so.1.65.1
...
So we see that ldd recognized the libmwboost_system.so.1.65.1 file in the current directory.
Turns out that libmwboost_filesystem.so.1.65.1.so also depends on libmwboost_system.so.1.65.1,
But when I run:
/home/user/project/lib$ ldd libmwboost_filesystem.so.1.65.1
...
libmwboost_system.so.1.65.1 => not found
...
How come ldd can find it when I run in on libmat.so and can't when I run it on libmwboost_filesystem.so.1.65.1 ?
I would be glad if someone could provide an explanation in the context of the linking process. As far as I know, when you link a file against a library, you use the following flags:
~$ gcc my_program.c -Lpath/to/solib/for/static/linker -lnameoflib -wl,-rpath=path/to/solib/for/dynamic/linker
This -Wl,-rpath flag embeds in the executable the path of the library that the dynamic linker will search for at run time. In the case of a shared library that depends on other libraries - does it work the same?
Ok so that's how it works:
When linking a binary - whether it's an executable or another shared library - against a shared library, the static linker embeds in the binary the names of the libraries we linked against. Those libraries will be loaded by the dynmaic linker - ld.so - once the program is run by the user.
Now the question is where the dynamic linker will search for these libraries at runtime. Briefly, according to the man page of ld.so, when the dynamic linker first inspects the binary to resolve its dependencies, it goes through the strings of the dependencies. If a string contains a slash "/" then the string is interpreted as a path. This can happen if the library name was specified with a slash at link time, like this:
~$ gcc prog.c ../path/to/library.so
In this case, the string embeded in the executable will be: ../path/to/library.so and the dynamic linker will search it relatively to the location of the binary.
If not, it will search for the library in a list of locations (the whole list is specified in the man page). The first location is the directories specified in the DT_RUNPATH section attribute of the binary. This can be set at link time using the -Wl,-rpath flag:
~$ gcc prog.c -Lpath/of/lib/ -lmylib -Wl,-rpath=path/of/lib
In this case, the path/of/lib will be searched for the library by the dynamic library.
You can inspect this attribute using readelf:
~$ readelf -d binary | grep RUNPATH
In my case, the libmat.so library contained a RUNPATH attribute set to $ORIGIN, meaning that libraries will be searched in the same location of the binary library. Whereas, the libmwboost_filesystem.so.1.65.1 didn't have this attribute set, that is why ldd didn't find the library.
ldd is just using ld.so to try and load the libraries, and shows where it found them, according to the search path specified in the ld.so man page.
I am trying to use the Intel MKL libraries for the first time. I am using CMake to build a simple project in which MKL is used. I work in the KDevelop 4.6 environment.
The project is built and installed without errors. Linking the libraries is thus succesful. While executing within KDevelop, I get the following error:
Error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory
The CMakeLists file looks essentially as follows:
project(testmkl)
cmake_minimum_required(VERSION 2.6)
enable_language(Fortran)
set(CMAKE_C_FLAGS "-std=c99 -Wall -lpthread")
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
# MKL
find_package(MKL REQUIRED)
include_directories(${MKL_INCLUDE_DIR})
find_package(M REQUIRED)
include_directories(${M_INCLUDES})
add_executable(testmkl ./main.c)
target_link_libraries(testmkl ${M_LIBRARIES} ${MKL_BLAS} ${MKL_LAPACK} ${MKL_INTEL} ${MKL_SEQUENTIAL} ${MKL_CORE})
install(TARGETS testmkl DESTINATION .)
libmkl_intel_lp64.so is found in the first folder of the LP_LIBRARY_PATH environment variable, so I wouldn't expect any error during the execution. Actually, when running the program from the command window, everything seems to work fine.
The ldd output for the executable is:
>> ldd ./testmkl
linux-vdso.so.1 => (0x00007fff951fe000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003061a00000)
libm.so.6 => /lib64/libm.so.6 (0x0000003061200000)
libmkl_intel_lp64.so => /opt/intel/composer_xe_2013_sp1.0.080/mkl/lib/intel64/libmkl_intel_lp64.so (0x00007f6f65ef6000)
libmkl_sequential.so => /opt/intel/composer_xe_2013_sp1.0.080/mkl/lib/intel64/libmkl_sequential.so (0x00007f6f65846000)
libmkl_core.so => /opt/intel/composer_xe_2013_sp1.0.080/mkl/lib/intel64/libmkl_core.so (0x00007f6f64317000)
libc.so.6 => /lib64/libc.so.6 (0x0000003060e00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003060600000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003061600000)
The linked libraries are thus correctly found. Why won't the program work in the KDevelop environment?
Any help is welcome, thanks!
First one short question: are you excuting your program from a terminal or trying to execute from KDE gui (e.g. by double-click)?
On many linux OS, now LD_LIBRARY_PATH is deprecated. LD finds the dependencies thanks to what is listed in /etc/ld.so.conf and /etc/ld.so.conf.d . You may have to add a script somewhere there to include your libraries. In fact you just have to add the path to your libraries in those scripts.
example from /etc/ld.so.conf/libc.conf:
# libc default configuration
/usr/local/lib
This may work for you.
edit:
you shoud also run ldconfig to update the LD database and/or use ldconfig -v, which does the same but with extended output
edit2: ldconfig may require root privileges to be understood by the GUI
I had the same problem and now it solved through configuration of environment variable in kdevelop.
Go to project->open configuration
Select make tab
Click setting icon in Active environment profile bar and add your environment variable:
LD_LIBRARY_PATH = /your/shared/library/path
I have a shared library's libmyworld.so in /opt/my_prog/lib and also in /home/user1/lib
Irrespective of the order I specified in LD_LIBRARY_PATH (LD_LIBRARY_PATH=/home/user1/lib;/opt/myprog/lib); my binary SHOULD always look for libmyworld.so FIRST in /opt/my_prog/lib;
Can this be done using GCC during compilation time? without modifying my_prog binary. Thanks in advance.
The search order for dynamic libraries in Linux (from ld.so man page) is the following
Using the DT_RPATH dynamic section attribute of the binary
if present and DT_RUNPATH attribute does not exist. Use of
DT_RPATH is deprecated.
Using the environment variable LD_LIBRARY_PATH. Except if
the executable is a setuid/setgid binary, in which case it
is ignored.
Using the DT_RUNPATH dynamic section attribute of the binary
if present.
From the cache file /etc/ld.so.cache which contains a
compiled list of candidate libraries previously found in the
augmented library path. If, however, the binary was linked
with -z nodeflib linker option, libraries in the default
library paths are skipped.
In the default path /lib, and then /usr/lib. If the binary
was linked with -z nodeflib linker option, this step is
skipped.
When linking, to set
DT_RUNPATH: use -Wl,--enable-new-dtags -Wl,-R$(RUNPATH)
DT_RPATH: use -Wl,--disable-new-dtags -Wl,-R$(RPATH)
In theory, it is better to use DT_RUNPATH as the LD_LIBRARY_PATH, on which the user has a control, has precedence. But here you want to avoid the user control, so use the DT_RPATH. In you link line:
-Wl,--disable-new-dtags -Wl,-R/opt/my_prog/lib
You can always launch your binary (here called foo) with
$ LD_LIBRARY_PATH=/opt/my_prog/lib foo
or make a shell script with the line above.
While compiling your source code use the below command
gcc -o [desired_executable_file_name] -L [Your shared library path] -l [your shared library name] -I [Header file path]
for example in your case
gcc -o my_word_exe -L /opt/my_prog/lib -lmyworld -I [header path if their]
Then it"ll take libmyworld.so in /opt/my_prog/lib this path
Use LD_PRELOAD.
LD_PRELOAD=/home/lib/libmyworld.so mybinary
The advantage is that you don't fiddle with LD_LIBRARY_PATH - your binary may depend on other shared libraries and it may need proper LD_LIBRARY_PATH/ld.so.conf/whatever.
PS. This is the least invasive and flexible solution, because does not affect loading of other libraries and does not hardcode paths in the user executable.
I have a make file for a program that links to libdl.so with the following line -ldl. There are no calls to dlopen or any of the related functions. What is the effect of linking to this library in this way even though you do not use any of the functions?
You'll have to read the documentation for your linker. From info ld on my Linux/ELF/GNU Binutils system (emphasis added):
`--as-needed'
`--no-as-needed'
This option affects ELF DT_NEEDED tags for dynamic libraries
mentioned on the command line after the `--as-needed' option.
Normally the linker will add a DT_NEEDED tag for each dynamic
library mentioned on the command line, regardless of whether the
library is actually needed or not. `--as-needed' causes a
DT_NEEDED tag to only be emitted for a library that satisfies an
undefined symbol reference from a regular object file or, if the
library is not found in the DT_NEEDED lists of other libraries
linked up to that point, an undefined symbol reference from
another dynamic library. `--no-as-needed' restores the default
behaviour.
You can check yourself by running ldd on a test program. On a simple test program, I get:
linux-vdso.so.1 => (0x00007fffd8305000)
libc.so.6 => /lib/libc.so.6 (0x00007f646c669000)
/lib/ld-linux-x86-64.so.2 (0x00007f646c9ca000)
However, if I link with -ldl, I get this:
linux-vdso.so.1 => (0x00007fff644f1000)
libdl.so.2 => /lib/libdl.so.2 (0x00007fb9b1375000)
libc.so.6 => /lib/libc.so.6 (0x00007fb9b1014000)
/lib/ld-linux-x86-64.so.2 (0x00007fb9b1579000)
Even though libdl isn't used by my program. However, if I run GCC with -Wl,--as-needed, libdl will not be linked in. According to my tests, this only works if -Wl,--as-needed is listed on the command line before -ldl.
What are the effects? It means that your binary won't run on systems without the shared library, even though you don't use it. It also means that your binary will break if you upgrade the shared library and uninstall the old one. It's not a big deal because binary compatibility is a bear anyway, but I see no reason not to turn on -Wl,--as-needed for projects in general.
I have coded a small application which uses nothing but STL. It has 8275 bytes of size without linking to any specific libraries:
linux-gate.so.1 => (0x00e1e000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x0015a000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0x0044b000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00741000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x00250000)
/lib/ld-linux.so.2 (0x00a75000)
Now, when I compile it and link it with boost_thread, it grows to 8290 bytes:
linux-gate.so.1 => (0x009d9000)
libboost_thread.so.1.40.0 => /usr/lib/libboost_thread.so.1.40.0 (0x00e59000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x003a3000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0x00bc5000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00f8a000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0x00110000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0x00bf0000)
librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0x00dd8000)
/lib/ld-linux.so.2 (0x00ba3000)
Note that there is no function call on my code to features of boost_thread. However, boost_thread is added anyway as a dependency of my application (as you can see on the output of ldd).
Linking to a shared library is different than linking a static library. The main differences were explained already by #Dietrich Epp, but there's another important detail. Shared libraries define functions void _init() and void _fini(void) which are called upon loading/unloading a shared library; if you don't define them yourself the linker will add default stubs.
Those will be called also if you link your program against a shared library, but don't reference any symbol from the library (and don't add --as-needed linker flag).
Most linkers will simply omit the unused objects from the final binary, exactly as if you hadn't linked with the library in the first place.
I came across a reference to it recently on proggit and (as of now) it is not explained.
I suspect this might be it, but I don't know for sure.
If you set LD_PRELOAD to the path of a shared object, that file will be loaded before any other library (including the C runtime, libc.so). So to run ls with your special malloc() implementation, do this:
$ LD_PRELOAD=/path/to/my/malloc.so /bin/ls
You can override symbols in the stock libraries by creating a library with the same symbols and specifying the library in LD_PRELOAD.
Some people use it to specify libraries in nonstandard locations, but LD_LIBRARY_PATH is better for that purpose.
As many people mentioned, using LD_PRELOAD to preload library. By the way, you can CHECK if the setting is available by ldd command.
Example: Suppose you need to preload your own libselinux.so.1.
> ldd /bin/ls
...
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f3927b1d000)
libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007f3927914000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f392754f000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f3927311000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f392710c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f3927d65000)
libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007f3926f07000)
Thus, set your preload environment:
export LD_PRELOAD=/home/patric/libselinux.so.1
Check your library again:
>ldd /bin/ls
...
libselinux.so.1 =>
/home/patric/libselinux.so.1 (0x00007fb9245d8000)
...
With LD_PRELOAD you can give libraries precedence.
For example you can write a library which implement malloc and free. And by loading these with LD_PRELOAD your malloc and free will be executed rather than the standard ones.
LD_PRELOAD lists shared libraries with functions that override the standard set, just as /etc/ld.so.preload does. These are implemented by the loader /lib/ld-linux.so. If you want to override just a few selected functions, you can do this by creating an overriding object file and setting LD_PRELOAD; the functions in this object file will override just those functions leaving others as they were.
For more information on shared libraries visit
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
To export mylib.so to env:
$ export LD_PRELOAD=/path/mylib.so
$ ./mybin
To disable it:
$ unset LD_PRELOAD
Here is a detailed blog post about preloading:
https://blog.cryptomilk.org/2014/07/21/what-is-preloading/
Using LD_PRELOAD path, you can force the application loader to load provided shared object, over the default provided.
Developers uses this to debug their applications by providing different versions of the shared objects.
We've used it to hack certain applications, by overriding existing functions using prepared shared objects.
When LD_PRELOAD is used, that file will be loaded before any other. Use
$export LD_PRELOAD=/path/lib for lib to be pre-loaded. This can even be used in programs too.