What is the difference between LD_PRELOAD_PATH and LD_LIBRARY_PATH?
I understand what they do, but not the particulars of how they differ.
From
http://en.wikipedia.org/wiki/Dynamic_linker
The dynamic linker can be influenced into modifying its behavior
during either the program's execution or the program's linking.
Examples of this can be seen in the run-time linker manual pages for
various Unix-like systems. A typical modification of
this behavior is the use of the LD_LIBRARY_PATH and LD_PRELOAD
environment variables. These variables adjust the runtime linking
process by searching for shared libraries at alternate locations and
by forcibly loading and linking libraries that would otherwise not be,
respectively.
In particular, I am interested in the differences in Linux which has both LD_PRELOAD_PATH and LD_LIBRARY_PATH:
https://linuxgazette.net/issue48/tag/48.html
Update: The author of this 1999 Linux Gazette article notes in his 2013 comment below the accepted answer that LD_PRELOAD_PATH does not in fact exist.
LD_PRELOAD (not LD_PRELOAD_PATH) is a list of specific libraries (files) to be loaded before any other libraries, whether the program wants it or not. LD_LIBRARY_PATH is a list of directories to search when loading libraries that would have been loaded anyway. On linux you can read man ld.so for more information about these and other environment variables that affect the dynamic linker.
LD_PRELOAD is more powerful
The most important difference is that LD_PRELOAD can replace functions statically linked into a binary.
Example
If ever you find that LD_PRELOAD works for you, but LD_LIBRARY_PATH mysteriously does not, this, almost certainly, is the reason why.
For example, I was debugging GNU Readline in bash and was confused at first why my modified libreadline.so was loading with LD_PRELOAD but not LD_LIBRARY_PATH.
$ LD_PRELOAD=shlib/libreadline.so bash -
(worked)
$ LD_LIBRARY_PATH=shlib/ bash -
(failed)
Taking a look with the ldd command, which lists dependencies on dynamically linked libraries, gives the answer:
$ ldd /bin/bash
linux-vdso.so.1
libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
/lib64/ld-linux-x86-64.so.2
Sure enough, libreadline is not listed as one of the shared libraries. Therefore, /bin/bash on my machine must have been statically linked with its own version of libreadline at compile time. The reason LD_LIBRARY_PATH doesn't work is because the binary never asks the dynamic linker (ld.so) to load libreadline.
LD_PRELOAD, on the other hand, loads the library no matter what, allowing a library to override even statically linked functions.
Related
After reading this question, my first reaction was that the user is not seeing the error because he specifies the location of the library with -L.
However, apparently, the -L option only influences where the linker looks, and has no influence over where the loader looks when you try to run the compiled application.
My question then is what's the point of -L? Since you won't be able to run your binary unless you have the proper directories in LD_LIBRARY_PATH anyway, why not just put them there in the first place, and drop the -L, since the linker looks in LD_LIBRARY_PATH automatically?
It might be the case that you are cross-compiling and the linker is targeting a system other than your own. For instance, MinGW can be used to compile Windows binaries on Linux. Here -L will point to the DLLs needed for linking and LD_LIBRARY_PATH will point to any libraries needed by linker to run. This allows compiling and linking of different architectures, OS ABIs, or processor types.
It's also helpful when trying to build special targets. I might be case that one links a static version of program against a different static library. This is the first step in Linux From Scratch, where one creates a separate mini-environment on the main system to become a chroot jail.
Setting LD_LIBRARY_PATH will affect all the commands you run to build your code (including the compiler itself).
That's not desirable in general (e.g. you might not want your compiler to run debug/instrumented libraries while it compiles - it might even go as far as breaking your compiles).
Use -L to tell the compiler where to look, LD_LIBRARY_PATH to influence runtime linking.
Building the binary and running the binary are two completely independent and unrelated processes. You seem to suggest that the running environment should affect the building environment, i.e. you seem to be making an assumption that the code build in some setup (account, machine) will be later run in the same setup. I find this assumption rather strange. I'd even say that in most cases the building and the running are done in different environments. I would actually prefer my compilers not to derive any assumptions about future running environment from the environment these compilers are invoked in. Looking onto the LD_LIBRARY_PATH of the building environment would be a major no-no.
The other answers are all good, but one nobody has mentioned yet is static libraries. Most of the time when you use -L it's with a static library built locally in your build tree that you don't intent to install, and it has nothing to do with LD_LIBRARY_PATH.
Compilers on Solaris support the -R /runtime/path/to/some/libs that adds to the path where libraries are to be searched by the run-time linker. On Linux the same could be achieved with -Wl,-rpath,/runtime/path/to/some/libs. It passes the -rpath /runtime/path/to/some/libs option to ld. GNU ld also supports the -R /path/to/libs for compatibility with other ELF linkers but this should be avoided as -R is normally used to specify symbol files to GNU ld.
I compile a project and run it on my system, it does perfectly fine.
I uploaded this to git and asked my administrator to run it on server and it says undefined reference to 'clock_gettime'. This error can be resolved by linking a library with -lrt option. Similar problem happened to me earlier but that was related to linking math.h library with the help of -lm option.
Is there a way to find which all libraries to link so that ou do not run into such problems and not feel embarrassed in front of your administrator. How to decide what all libraries to link. Can there be a thought process for this?
Some man pages will mention both the headers and libraries that should be used. (This is not true of all pages unfortunately.)
For example, look at man 3 signal. On my system that page has a LIBRARY section at the top with Standard C Library (libc, -lc), followed by a SYNOPSIS section with #include <signal.h>.
Some other strategies include:
Blunt use of strings piped to egrep in a shell loop, to search for references to names among installed libraries. This can return a lot of false positives for common names, but it works very well for elaborate function names that don't match single English words.
Iterative compilation and linking. If you are trying to resolve a single function, write a tiny program that calls the function and use a shell or makefile loop to test each single library on your system (starting with a small list of most likely candidates from /usr/lib or wherever else). At some point the build will succeed and show you the right library. This method takes a little work to set up, but the infrastructure you create is also very easy to reuse in the future.
Inspect binaries iteratively using your linker's tools (e.g. otool on Mac OS X, or objdump or readelf on Solaris or Linux). For instance, try to disassemble a library's text section starting from a particular name (otool -v -t -p _symbol on Mac OS X).
For *nix based machines, ldd on the executable will print shared library dependencies.
I downloaded the glibc source code, modified some portion of the standard library and then used LD_PRELOAD to use that modified standard library (in the form of an .so file) with my program. However, when I copied that .so file to another computer and tried to run the same program using LD_PRELOAD there, I got a segmentation fault.
Notice that both computers have x86-64 processors. Moreover, both computers have gcc 4.4 installed. Although the computer in which it is not running has also gcc 4.1.2 installed besides gcc 4.4. However, one is running Ubuntu 10.04 (where I compiled), while the other is running CentOS 5. Is that the cause of the segmentation fault? How can I solve this problem? Notice that I don't have administrative rights on the computer with CentOS 5.
When you LD_PRELOAD the C library, I believe you're loading it in addition to the default C library. When they're the exact same version, all the symbols match, and yours takes precedence. So it works. When they're different versions, you may well have a mix, on a per-symbol basis.
Also, the NSS (name service switch, e.g., all the stuff from /etc/nsswitch.conf) API is not stable. These modules are separate from the main libc.so, but are dynamically loaded when a program e.g., does a user id to username mapping. Loading the wrong version (because you copied libc.so over) will do all kinds of badness.
Further, Ubuntu may be using eglibc and CentOS glibc. So you could be looking at a different fork of glibc.
If your LD_PRELOAD library included only the symbols you actually need to override, and overrode them to the minimum amount possible (e.g., if possible, call the overridden function), then your library has a much higher chance of being portable.
For an example of how to do this, see (for example) fakeroot.
If you're changing so much of libc that your only choice is to override all of it, then (a) you're doing something very weird; (b) you probably want to use LD_LIBRARY_PATH, not LD_PRELOAD; see the ld.so(8) manpage for details.
It is likely that your libc is not portable between kernel versions.
fakeroot seems to build some libfakeroot.a (but inside a .lib directory).
But I am not sure, that the static linker can indeed replace/rename symbols as the dynamic linker can.
Fakeroot uses the dynamic linker in order to do its magic (specifically, LD_PRELOAD). Unfortunately, the dynamic linker is not involved in loading statically linked binaries (which is how the dynamic linker itself is invoked: /lib/ld-linux.so.2 is statically compiled).
As answered above, your only option, as far as I'm aware, is to use fakeroot-ng, which uses a completely different mechanism to inject into the process, and is, thus, able to work on statically linked libraries without a problem.
In fact, statically linked libraries was part of the reason I set out to write fakeroot-ng in the first place. At the time, there was no way to tell ldconfig to run on a subtree, and ldconfig is statically linked.
Shachar
seems the solution is to use fakeroot-ng, which works for statically linked binaries.
Is it possible to have glibc and uClibc based applications running side-by-side on one system?
Background: We have binary gcc based cross-compiler configured to link with uClibc. We have cross-compiled glibc with it. Now we want to build some applications so they will link with the glibc rather than uClibc. We don't want to rebuild the compiler.
There's no problem with glibc and uClibc living side-by-side with some programs linking to one and other programs linking to the other. However, there is a problem with additional libraries. Each shared library on your system will be built against either glibc or uClibc (using the corresponding headers, which define distinct ABIs for the standard library functions), so for example if both a glibc program and a uClibc program need ncurses, you'll need to have two versions of ncurses built, and have a way of ensuring that the correct one for the given program gets loaded at runtime. Alternatively, you could choose to only use one set of shared libraries, and use static libraries for programs linked to the other libc, but you'd still need to build your 2 sets of libraries.
Yes, it should be perfectly possible, but you might have to play around with LD_PRELOAD_PATH. If you are linking statically, change to dynamic linking.
It is nearly impossible to mix them in the same FHS, as the ABI and include dir are incompatible. However, you could install either of them in an directory offset, by tweaking dynamic-linker field in ELF and exploiting sysroot feature in gcc/binutils. An on going experiment is in Gentoo community[1], known as Prefix/libc.
http://wiki.gentoo.org/wiki/Prefix/libc