XML::LibXML is a Perl library. LibXML.so is a part of the library. I'm trying to make XML::LibXML use custom built libxml2. But providing LD_LIBRARY_PATH doesn't make any difference:
$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f66af5e9000)
$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f2d26ae3000)
$ ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f878cbc8000)
$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
libxml2.so.2 => /home/yuri/_/libxml2/.libs/libxml2.so.2 (0x00007f6f8f5d8000)
What am I doing wrong? Is there a way to handle it? The goal is to make Perl script use custom built libxml2 (to investigate some issue).
UPD
$ find /usr/lib -name libxml2.so.2
/usr/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2/libxml2.so.2
$ find /usr/local/lib -name libxml2.so.2
tl;dr
ExtUtils::MakeMaker hardcodes paths of the shared libraries in the binaries by means of RPATH entry in .dynamic section. But you can preload the needed library,
$ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so
or make the loader ignore RPATH entry,
$ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs /usr/lib/ld-linux-x86-64.so.2 --inhibit-rpath /abs/path/to/LibXML.so /abs/path/to/perl ./1.pl
or convert RPATH entry into RUNPATH one,
$ chrpath --convert blib/arch/auto/XML/LibXML/LibXML.so
or delete RPATH entry,
$ chrpath --delete blib/arch/auto/XML/LibXML/LibXML.so
or specify XMLPREFIX variable,
$ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build
Under the hood
ExtUtils::Liblist::ext() takes a list of libraries to be linked with, like -lxml2 -lz -llzma -licui18n -licuuc -licudata -lm -ldl, and turns them it into four or five variables, that make their way into the generated Makefile. One of those is LD_RUN_PATH. It contains all the paths where the libraries where found.
$is_dyna is true if the library is dynamic (not ends with .a). $is_perl is true if Perl was not built with it, and %ld_run_path_seen is to not duplicate values in LD_RUN_PATH.
The part that calculates LD_RUN_PATH, that generates part of the Makefile with the variable, and the part that passes it to the linker.
Consequences
LD_RUN_PATH and in its turn RPATH entry in the binary make the loader search for libraries (at runtime) in the directories where they where found (at compile time).
-rpath=dir
Add a directory to the runtime library search path. This is used when linking an ELF executable with shared objects. All -rpath arguments are concatenated and passed to the runtime linker, which uses them to locate shared objects at runtime. The -rpath option is also used when locating shared objects which are needed by shared objects explicitly included in the link; see the description of the -rpath-link option. If -rpath is not used when linking an ELF executable, the contents of the environment variable "LD_RUN_PATH" will be used if it is defined.
https://jlk.fjfi.cvut.cz/arch/manpages/man/core/binutils/ld.1.en
If a shared object dependency does not contain a slash, then it is searched for in the following order:
o Using the directories specified in the DT_RPATH dynamic section attribute of the binary if present and DT_RUNPATH attribute does not exist. Use of DT_RPATH is deprecated.
o Using the environment variable LD_LIBRARY_PATH, unless the executable is being run in secure-execution mode (see below), in which case this variable is ignored.
https://jlk.fjfi.cvut.cz/arch/manpages/man/core/man-pages/ld.so.8.en
$ perl Makefile.PL
$ make
Result,
$ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
0x000000000000000f (RPATH) Library rpath: [/usr/lib]
$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
RPATH /usr/lib
$ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f6cfabb2000)
--dynamic - display contents of the .dynamic section, -x - display all headers.
Solution
One way to remedy this is to preload needed instance of libxml2,
$ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)
Another is to tell Makefile.PL search for libxml2 in the desired location (at compile time).
$ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build
$ make
Result,
$ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
0x000000000000000f (RPATH) Library rpath: [path/to/custom/libxml2/build/lib:/usr/lib]
$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
RPATH path/to/custom/libxml2/build/lib:/usr/lib
$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)
On a side note
Should you care to debug XML::LibXML, you can run make with OPTIMIZE variable (add -B to make make rebuild everything in case the library has already been built),
$ make OPTIMIZE='-g3 -O0'
There's also -O2, and no -g in LDDLFLAGS, but not sure if that matters much.
Alternatively, WriteMakeFile takes OPTIMIZE parameter, so you can add a line here,
'OPTIMIZE' => '-g3 -O0'
Or you can add the variable here, and pass it to Makefile.PL,
$ perl Makefile.PL OPTIMIZE='-g3 -O0'
If you install your custom built version of libxml in /usr/local/lib, I think the XML::LibXML module should load the custom one rather than the one in /usr/lib (Assuming you have sudoers or root permission to install it).
I'm not sure why Python seems to use a different lookup path for libraries than Perl does, though. Maybe the full path to libxml library is hardcoded in XML::LibXML's LibXML.so? If the XML::LibXML module was linked with -l/usr/lib/libxml2.so instead of -lxml2 that would be the problem.
Related
I'm building the following file:
int main()
{
return 0;
}
with the following flags:
g++ -o main -Wl,-rpath,'$ORIGIN' main.cpp
but the rpath flag is doing nothing. When I execute the following commands:
objdump -x main | grep -i rpath
readelf -a main | grep -i rpath
I obtain nothing (RPATH is not defined).
What I'm doing wrong?
EDIT
I have tried to do the above with a different binary using the following cmake flags:
set(CMAKE_SKIP_BUILD_RPATH FALSE)
set(CMAKE_BUILD_WITH_INSTALL_RPATH FALSE)
set(CMAKE_INSTALL_RPATH "\$ORIGIN")
set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
I have moved the executable to a different machine, and placed a dynamic library that it needs to 'dlopen' in the same folder. It has worked (and I'm 100% sure this is because rpath, since before applying the above cmake flags the executable didn't worked).
Still, using the above two commands to check rpath (objdump and readelf) I still don't see anything.
If I didnt miss something here, you are not linking any libs in your build command.
Lets say you want to link libusb.so shared library, which is located in libusb sub-folder of your current folder where is main.cpp.
I will not take any details here, about soname, linkname of lib etc, just to make clear about rpath.
rpath will provide runtime linker path to library, not for linktime, cause even shared library need to be present(accessible) in compile/link time. So, to provide your application loader with possibility to look for needed library in start time, relatively to your app folder, there is $ORIGIN variable, you can see it with readelf but only if you link some library with $ORIGIN in rpath.
Here is example based on your question:
g++ main.cpp -o main -L./libusb -Wl,-rpath,'$ORIGIN/libusb' -lusb
As you see, you need to provide -L directory for compile/link time search, and rpath for runtime linker. Now you will be able to examin all needed libs for your app using readelf and location for search.
After spending almost an hour on filtering output of
clang -v hello_world.c
I got the following linking command:
ld -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 \
/usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o \
hello_world.o /usr/lib/x86_64-linux-gnu/libc.so \
/usr/lib/x86_64-linux-gnu/crtn.o
Is there an easier way to find out that -lc will expand to /usr/lib/x86_64-linux-gnu/libc.so?
I need to know which files are used so I can copy them to another system for cross compiling.
edit
Looks like I should use the cross-compile toolchain. From clang docs:
https://clang.llvm.org/docs/CrossCompilation.html
When you have extracted your cross-compiler from a zip file into a
directory, you have to use --sysroot=. The path is the root
directory where you have unpacked your file, and Clang will look for
the directories bin, lib, include in there.
Where can I get that zip file they refer to? I'm interested in x86_64-linux-gnu target.
This will list all the files linked to libc.so.6:
for i in `find . -type f -executable`
do
j=$(ldd $i | grep libc.so.6 | wc -l)
if [ $j -gt 0 ]
then
echo $i
fi
done
In case you want to find out what the path of libc.so.6 is, stated in the original question something similar to:
ldd `which ld` | sed 's/^[[:space:]]libc.so.6[[:space:]]=>[[:space:]]\(.*\)[[:space:]](.*)/\1/p' | grep --color=never libc.so
will type the path, you can obviously replace the expression after ldd with any filename.
From the comments, there is a way with clang directly though it will generate a lot of noise which is significantly harder to exclude compared to the ldd way.
clang -Wl,--verbose hello_world.c
will tell the linker to be verbose and it will eventually tell you all the library paths tried for each library.
I am trying cross compile C code (outside yocto) using toolchain (/home/amruta/Downloads/poky-glibc-x86_64-core-image-weston-sdk-cortexa7hf-neon-toolchain-2.4.2.sh) for renesas G1E module. Procedure followed to cross compile :
Installed given toolchain
Set environment :
amruta#amruta-OptiPlex-3060:~$ . /opt/poky/2.4.2/environment-setup-cortexa7hf-neon-poky-linux-gnueabi
Compiling in same terminal :
amruta#amruta-OptiPlex-3060:~/amruta/amruta_projects/G1E/EnergyMeterApp1/src$ $CC *.c -o Energymeter -L /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib/libmosquitto -I /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/
Source files used for cross compilation :
amruta#amruta-OptiPlex-3060:~/amruta/amruta_projects/G1E/EnergyMeterApp1/src$ ls
client_shared_lib.c EnergyMeterApp1.h GenericFunctions.c modbus ProcessHandler.c
client_shared_lib.h FileLogger.c GenericFunctions.h mosq_pub_sub_client.c ProcessHandler.h
FileLogger.h libconfig.h mosq_pub_sub_client.h ReadAllConfigs.c
EnergyMeterApp1.c GenericDefns.h Makefile mosquitto.h ReadAllConfigs.h
Library files already present in lib dir
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib | grep libmos
libmosquitto.so.1
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib | grep libconf
libconfig.so.9
libconfig++.so.9
libconfig.so.9.2.0
libconfig++.so.9.2.0
amruta#amruta-OptiPlex-3060:~$
Header files present in include dir
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/ | grep mos
mosquitto.h
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/ | grep libconfig
libconfig.h
Build output (partial):
amruta#amruta-OptiPlex-3060:~/amruta/amruta_projects/G1E/EnergyMeterApp1/src$ $CC *.c -o Energymeter -L /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib/libmosquitto -I /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/
/tmp/ccBMhPYg.o: In function client_id_generate':
client_shared_lib.c:(.text+0x64): undefined reference to mosquitto_lib_cleanup'
client_shared_lib.c:(.text+0x144): undefined reference to `mosquitto_lib_cleanup'
/tmp/ccRMPfVp.o: In function ReadEnergyMeterConfigs':
ReadAllConfigs.c:(.text+0x374): undefined reference to config_init'
ReadAllConfigs.c:(.text+0x384): undefined reference to config_read_file'
ReadAllConfigs.c:(.text+0x3c8): undefined reference to config_destroy'
Please suggest for successful cross compilation.
Specifying a library directory (-L <dir>) will not cause any libraries therein to be linked. The -L switch only tells the linker where to look for libraries specified by -l <lib> switches, of which you have none.
You need to add -l mosquitto to link libmosquitto.so for example. Similarly -l config for libconfig.so.
In general for any library libXXX.so or libXXX.a you link it with -l XXX. Where the libraries are versioned as in this case, the latest version will be linked. To link a specific version, you don't use -l <lib> but simply specify the path to the specific .so file as an input without a '-' switch.
I am trying to manually do all the compilation steps of my program. In the last step where I use the linker command ld, I need to specify the object file for library(stdio as I have used printf in my code) then only I can make and .exe file. Where does this object file reside?
I'd suggest that you run gcc -v your_file.c. That will let you see exactly what commands your linker is using. You probably don't have an stdio.o file to link against. Instead this is included in the C runtime library and the exact file will depend on your system configuration.
stdio is part of the C standard library. The exact location of the standard library is system-specific, but it is often in a file called libc.a or libc.so.
In my machine this can be found in:
/usr/lib/x86_64-linux-gnu/libc.a
/usr/lib/x86_64-linux-gnu/libc.so
You can ask ld for the search directory: ld --verbose | grep SEARCH_DIR.
libc.a is an ar (man) archive, and you can extract stdio.o from the libc.a archive using ar command:
$ # list archive member with `ar t`
$ ar t /usr/lib/x86_64-linux-gnu/libc.a | grep ^stdio
stdio.o
$ # extract stdio.o with `ar x`
$ ar x /usr/lib/x86_64-linux-gnu/libc.a stdio.o
its probably under root
Somthing like /usr/lib or /usr/lib***
There are multitudes of articles online that proclaim in strident tones that the use of LD_LIBRARY_PATH is a bad idea, and that one must set library search paths using the -R option. The majority of said articles also mention Solaris in the same breath. The trouble is, on Linux, this does not work with g++.
g++: unrecognized option '-R'
Now what?
You can use -Wl,-rpath=/your/rpath:
$ g++ -o t t.cpp -Wl,-rpath=/my/lib/dir -lwhatever
$ readelf -a t|grep RPATH
0x000000000000000f (RPATH) Library rpath: [/my/lib/dir]