I am trying to manually do all the compilation steps of my program. In the last step where I use the linker command ld, I need to specify the object file for library(stdio as I have used printf in my code) then only I can make and .exe file. Where does this object file reside?
I'd suggest that you run gcc -v your_file.c. That will let you see exactly what commands your linker is using. You probably don't have an stdio.o file to link against. Instead this is included in the C runtime library and the exact file will depend on your system configuration.
stdio is part of the C standard library. The exact location of the standard library is system-specific, but it is often in a file called libc.a or libc.so.
In my machine this can be found in:
/usr/lib/x86_64-linux-gnu/libc.a
/usr/lib/x86_64-linux-gnu/libc.so
You can ask ld for the search directory: ld --verbose | grep SEARCH_DIR.
libc.a is an ar (man) archive, and you can extract stdio.o from the libc.a archive using ar command:
$ # list archive member with `ar t`
$ ar t /usr/lib/x86_64-linux-gnu/libc.a | grep ^stdio
stdio.o
$ # extract stdio.o with `ar x`
$ ar x /usr/lib/x86_64-linux-gnu/libc.a stdio.o
its probably under root
Somthing like /usr/lib or /usr/lib***
Related
Given these:
bar.cpp:
int foo();
int bar()
{
return foo();
}
foo.cpp:
int foo()
{
return 42;
}
The libfoo.so is built by gcc for foo.cpp,i.e. gcc -shared -o libfoo.so -fPIC foo.c
As it's all known that readelf -d could be used to show the dependency of a specific shared library.
$ gcc -shared -o libbar2.so -fPIC bar.c -lfoo -L.
$ gcc -shared -o libbar.so -lfoo -L. -fPIC bar.c
$ readelf -d libbar2.so | grep -i needed
0x0000000000000001 (NEEDED) Shared library: [libfoo.so]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
$ readelf -d libbar.so | grep -i needed
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Why the sequence of the parameters passes to the gcc influence the output of readelf -d for the built shared library?
All these tests are on Ubuntu16.04 with gcc 5.4.0.
Update:
$ ls -l libbar*
-rwxrwxr-x 1 joy joy 8000 Oct 4 23:16 libbar2.so
-rwxrwxr-x 1 joy joy 8000 Oct 4 23:16 libbar.so
$ sum -r libbar*
00265 8 libbar2.so
56181 8 libbar.so
The linking process is sequential and the order in which you specify the files is important. The file are treated in the order they are given. See this extract from the ld manual:
Some of the command-line options to ld may be specified at any point
in the command line. However, options which
refer to files, such as -l or -T, cause the file to be read at the point at which the option appears in the command
line, relative to the object files and other file options.
When you try to link a shared library into another one, the linker will lookup if there is any undefined reference that requires something from the library in all the files considered UP TO NOW(hence in your second example, there is no files prior to the libfoo library ) , and if there is none, the library is left aside, and the linking continue with the remaining files.
Here you also have a behaviour that may be surprising: it is possible (by default) to create shared libraries that still have undefined references (that means they are not self contained). That is what happen in your second example(libbar.so). If you want to avoid this behaviour to be sure you are not in this case you can add the -Wl,-no-undefined option (see https://stackoverflow.com/a/2356393/4871988).
If you add this option the second case will raise an error at link time.
EDIT: I found this other extract in the ld manual that explain this behaviour:
The linker will search an archive only once, at the location where it
is specified on the command line. If the
archive defines a symbol which was undefined in some object which appeared before the archive on the command
line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an
object appearing later on the command line will not cause the linker to search the archive again.
See the -( option for a way to force the linker to search archives multiple times.
You may list the same archive multiple times on the command line.
This also applies to shared libraries
I am looking to figure out which C library to include when compiling a program that includes it as a header, in this case #include <pcre2.h>. The only way I've been able to figure out where the file is I need is to check for a specific symbol that I know needs to be exported. For example:
$ ls
CMakeCache.txt Makefile install_manifest.txt libpcre2-posix.pc pcre2_grep_test.sh
CMakeFiles a.out libpcre2-8.a pcre2-config pcre2_test.sh
CTestCustom.ctest cmake_install.cmake libpcre2-8.pc pcre2.h pcre2grep
CTestTestfile.cmake config.h libpcre2-posix.a pcre2_chartables.c pcre2test
$ objdump -t libpcre2-8.a|grep pcre2_compile
pcre2_compile.c.o: file format elf64-x86-64
0000000000000000 l df *ABS* 0000000000000000 pcre2_compile.c
00000000000100bc g F .text 00000000000019dd pcre2_compile_8
0000000000000172 g F .text 00000000000000e3 pcre2_compile_context_create_8
0000000000000426 g F .text 0000000000000055 pcre2_compile_context_copy_8
0000000000000557 g F .text 0000000000000032 pcre2_compile_context_free_8
And because the symbol pcre2_compile_8 exists in that file (after trying every other file...) I know that the library I need to include is pcre2-8, that is, I compile my code with:
$ gcc myfile.c -lpcre2-8 -o myfile; ./myfile
Two questions related to this:
Is there a simpler way to find a symbols in a batch of files (some of which are not elf files)? For example, something like objdump -t *? Or what's the closest thing to doing that?
Is there a better way to find out what the library value of -l<library> is? Or, what's the common way when someone downloads a new C program that they know what to add to their command-line so that the program works? (For me, I've just spent the last hour figuring out that it's -lpcre2-8 and not -lpcre or -lpcre2.
Usually, the function you call from the library will be a symbol defined by that library. But in PCRE2, due to different code unit sizes, the function you call (e.g. pcre2_compile) actually becomes a different symbol through preprocessor macros (e.g. pcre2_compile_8). You can find the symbol you need from the library by compiling your program and checking the undefined symbols:
$ cat test.c
#define PCRE2_CODE_UNIT_WIDTH 8
#include <pcre2.h>
int main() {
pcre2_compile("",0,0,NULL,NULL,NULL);
}
$ gcc -c test.c
$ nm -u test.o
U _GLOBAL_OFFSET_TABLE_
U pcre2_compile_8
Is there a simpler way to find a symbols in a batch of files?
You can search a directory (/usr/lib/ below) for the library files (.a or .so extension below), running nm for each and search for the undefined symbol (adapted from this question):
$ for lib in $(find /usr/lib/ -name \*.a -o -name \*.so)
> do
> nm -A --defined-only $lib 2>/dev/null| grep pcre2_compile_8
> done
/usr/lib/x86_64-linux-gnu/libpcre2-8.a:libpcre2_8_la-pcre2_compile.o:0000000000007f40 T pcre2_compile_8
Is there a better way to find out what the library value of -l is?
It is usually conveyed through the library documentation. For PCRE2, the second page of the documentation talks about the pcre-config tool that gives the appropriate flags:
pcre2-config returns the configuration of the installed PCRE2 libraries and the options required to compile a program to use them. Some of the options apply only to the 8-bit, or 16-bit, or 32-bit libraries, respectively, and are not available for libraries that have not been built.
[...]
--libs8 Writes to the standard output the command line options required to link with the 8-bit PCRE2 library (-lpcre2-8 on many systems).
[...]
--cflags Writes to the standard output the command line options required to compile files that use PCRE2 (this may include some -I options, but is blank on many systems).
So for this particular library, the recommended way to build and link is:
gcc -c $(pcre2-config --cflags) test.c -o test.o
gcc test.o -o test $(pcre2-config --libs8)
I am trying cross compile C code (outside yocto) using toolchain (/home/amruta/Downloads/poky-glibc-x86_64-core-image-weston-sdk-cortexa7hf-neon-toolchain-2.4.2.sh) for renesas G1E module. Procedure followed to cross compile :
Installed given toolchain
Set environment :
amruta#amruta-OptiPlex-3060:~$ . /opt/poky/2.4.2/environment-setup-cortexa7hf-neon-poky-linux-gnueabi
Compiling in same terminal :
amruta#amruta-OptiPlex-3060:~/amruta/amruta_projects/G1E/EnergyMeterApp1/src$ $CC *.c -o Energymeter -L /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib/libmosquitto -I /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/
Source files used for cross compilation :
amruta#amruta-OptiPlex-3060:~/amruta/amruta_projects/G1E/EnergyMeterApp1/src$ ls
client_shared_lib.c EnergyMeterApp1.h GenericFunctions.c modbus ProcessHandler.c
client_shared_lib.h FileLogger.c GenericFunctions.h mosq_pub_sub_client.c ProcessHandler.h
FileLogger.h libconfig.h mosq_pub_sub_client.h ReadAllConfigs.c
EnergyMeterApp1.c GenericDefns.h Makefile mosquitto.h ReadAllConfigs.h
Library files already present in lib dir
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib | grep libmos
libmosquitto.so.1
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib | grep libconf
libconfig.so.9
libconfig++.so.9
libconfig.so.9.2.0
libconfig++.so.9.2.0
amruta#amruta-OptiPlex-3060:~$
Header files present in include dir
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/ | grep mos
mosquitto.h
amruta#amruta-OptiPlex-3060:~$ ls /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/ | grep libconfig
libconfig.h
Build output (partial):
amruta#amruta-OptiPlex-3060:~/amruta/amruta_projects/G1E/EnergyMeterApp1/src$ $CC *.c -o Energymeter -L /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/lib/libmosquitto -I /opt/poky/2.4.2/sysroots/cortexa7hf-neon-poky-linux-gnueabi/usr/include/
/tmp/ccBMhPYg.o: In function client_id_generate':
client_shared_lib.c:(.text+0x64): undefined reference to mosquitto_lib_cleanup'
client_shared_lib.c:(.text+0x144): undefined reference to `mosquitto_lib_cleanup'
/tmp/ccRMPfVp.o: In function ReadEnergyMeterConfigs':
ReadAllConfigs.c:(.text+0x374): undefined reference to config_init'
ReadAllConfigs.c:(.text+0x384): undefined reference to config_read_file'
ReadAllConfigs.c:(.text+0x3c8): undefined reference to config_destroy'
Please suggest for successful cross compilation.
Specifying a library directory (-L <dir>) will not cause any libraries therein to be linked. The -L switch only tells the linker where to look for libraries specified by -l <lib> switches, of which you have none.
You need to add -l mosquitto to link libmosquitto.so for example. Similarly -l config for libconfig.so.
In general for any library libXXX.so or libXXX.a you link it with -l XXX. Where the libraries are versioned as in this case, the latest version will be linked. To link a specific version, you don't use -l <lib> but simply specify the path to the specific .so file as an input without a '-' switch.
XML::LibXML is a Perl library. LibXML.so is a part of the library. I'm trying to make XML::LibXML use custom built libxml2. But providing LD_LIBRARY_PATH doesn't make any difference:
$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f66af5e9000)
$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f2d26ae3000)
$ ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f878cbc8000)
$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
libxml2.so.2 => /home/yuri/_/libxml2/.libs/libxml2.so.2 (0x00007f6f8f5d8000)
What am I doing wrong? Is there a way to handle it? The goal is to make Perl script use custom built libxml2 (to investigate some issue).
UPD
$ find /usr/lib -name libxml2.so.2
/usr/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2/libxml2.so.2
$ find /usr/local/lib -name libxml2.so.2
tl;dr
ExtUtils::MakeMaker hardcodes paths of the shared libraries in the binaries by means of RPATH entry in .dynamic section. But you can preload the needed library,
$ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so
or make the loader ignore RPATH entry,
$ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs /usr/lib/ld-linux-x86-64.so.2 --inhibit-rpath /abs/path/to/LibXML.so /abs/path/to/perl ./1.pl
or convert RPATH entry into RUNPATH one,
$ chrpath --convert blib/arch/auto/XML/LibXML/LibXML.so
or delete RPATH entry,
$ chrpath --delete blib/arch/auto/XML/LibXML/LibXML.so
or specify XMLPREFIX variable,
$ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build
Under the hood
ExtUtils::Liblist::ext() takes a list of libraries to be linked with, like -lxml2 -lz -llzma -licui18n -licuuc -licudata -lm -ldl, and turns them it into four or five variables, that make their way into the generated Makefile. One of those is LD_RUN_PATH. It contains all the paths where the libraries where found.
$is_dyna is true if the library is dynamic (not ends with .a). $is_perl is true if Perl was not built with it, and %ld_run_path_seen is to not duplicate values in LD_RUN_PATH.
The part that calculates LD_RUN_PATH, that generates part of the Makefile with the variable, and the part that passes it to the linker.
Consequences
LD_RUN_PATH and in its turn RPATH entry in the binary make the loader search for libraries (at runtime) in the directories where they where found (at compile time).
-rpath=dir
Add a directory to the runtime library search path. This is used when linking an ELF executable with shared objects. All -rpath arguments are concatenated and passed to the runtime linker, which uses them to locate shared objects at runtime. The -rpath option is also used when locating shared objects which are needed by shared objects explicitly included in the link; see the description of the -rpath-link option. If -rpath is not used when linking an ELF executable, the contents of the environment variable "LD_RUN_PATH" will be used if it is defined.
https://jlk.fjfi.cvut.cz/arch/manpages/man/core/binutils/ld.1.en
If a shared object dependency does not contain a slash, then it is searched for in the following order:
o Using the directories specified in the DT_RPATH dynamic section attribute of the binary if present and DT_RUNPATH attribute does not exist. Use of DT_RPATH is deprecated.
o Using the environment variable LD_LIBRARY_PATH, unless the executable is being run in secure-execution mode (see below), in which case this variable is ignored.
https://jlk.fjfi.cvut.cz/arch/manpages/man/core/man-pages/ld.so.8.en
$ perl Makefile.PL
$ make
Result,
$ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
0x000000000000000f (RPATH) Library rpath: [/usr/lib]
$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
RPATH /usr/lib
$ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f6cfabb2000)
--dynamic - display contents of the .dynamic section, -x - display all headers.
Solution
One way to remedy this is to preload needed instance of libxml2,
$ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)
Another is to tell Makefile.PL search for libxml2 in the desired location (at compile time).
$ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build
$ make
Result,
$ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
0x000000000000000f (RPATH) Library rpath: [path/to/custom/libxml2/build/lib:/usr/lib]
$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
RPATH path/to/custom/libxml2/build/lib:/usr/lib
$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
libxml2.so.2 => path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)
On a side note
Should you care to debug XML::LibXML, you can run make with OPTIMIZE variable (add -B to make make rebuild everything in case the library has already been built),
$ make OPTIMIZE='-g3 -O0'
There's also -O2, and no -g in LDDLFLAGS, but not sure if that matters much.
Alternatively, WriteMakeFile takes OPTIMIZE parameter, so you can add a line here,
'OPTIMIZE' => '-g3 -O0'
Or you can add the variable here, and pass it to Makefile.PL,
$ perl Makefile.PL OPTIMIZE='-g3 -O0'
If you install your custom built version of libxml in /usr/local/lib, I think the XML::LibXML module should load the custom one rather than the one in /usr/lib (Assuming you have sudoers or root permission to install it).
I'm not sure why Python seems to use a different lookup path for libraries than Perl does, though. Maybe the full path to libxml library is hardcoded in XML::LibXML's LibXML.so? If the XML::LibXML module was linked with -l/usr/lib/libxml2.so instead of -lxml2 that would be the problem.
I have 25 object files which are combined to form one shared library. I have generated the shared library with gcc and while I was looking for exposed symbols with nm -D libmylib.so, I found two undesirable exposed symbols with the name of i and y which are in .bss section. I have tried to find them in my source files but i cant find them so if anyone can tell me whether there is some way to find that which .o file exactly has these undesired exposed symbols? Can I do it with nm or do I need another tool?
Any help would be appreciated.
Once the shared library is linked, you can no longer tell which parts of it came from which object file.
You can search the individual objects from which you build the library:
find . -name '*.o' -print0 | xargs -0 nm -A | egrep ' (i|y)$'
You can ask the linker to tell you when they are defined:
$(CC) -fPIC -shared -o libmy.so $(OBJS) -Wl,-y,i,-y,y
If you built the library from objects compiled with -g, you may ask GDB where i and y came from:
gdb -q libmy.so
(gdb) info var ^i$
(gdb) info var ^y$
In the directory with your object files you can run:
find . -name '*.o' -exec nm -D {} \; -print
This should print symbols and then file name