Recently I got a test binary. When I checked it using objdump, I observed that it includes hard coded library path. Why it is needed to to hardcode the path like that? Shouldn't the path be taken from SHELL environment variables or -L parameter instead ?
objdump -p testprog
The output includes the hardcoded path to shared libraries:
....
NEEDED /home/test/lib/liba.so
NEEDED /home/test/lib/libb.so
NEEDED /home/test/lib/libc.so
....
This is probably because those three .so files had no SONAME on the host where your test program was built. Tell the person who built it to rebuild liba.so with -Wl,soname,liba.so and similar for the other two, then relink the main program.
Related
I'm trying to gather all the dependencies needed by some .so file. I use the Recursive ldd script, but that doesn't really matter for the manner of sake.
I want to put all the .so files in one directory, say it's in
/home/user/project/lib
I'm having a weird experience: Say there's a file libmat.so which I want to gather all its dependencies. So I ran
/home/user/project/lib$ ldd libmat.so
...
libmwboost_system.so.1.65.1 => /home/user/project/lib/libmwboost_system.so.1.65.1
libmwboost_filesystem.so.1.65.1 => /home/user/project/lib/libmwboost_filesystem.so.1.65.1
...
So we see that ldd recognized the libmwboost_system.so.1.65.1 file in the current directory.
Turns out that libmwboost_filesystem.so.1.65.1.so also depends on libmwboost_system.so.1.65.1,
But when I run:
/home/user/project/lib$ ldd libmwboost_filesystem.so.1.65.1
...
libmwboost_system.so.1.65.1 => not found
...
How come ldd can find it when I run in on libmat.so and can't when I run it on libmwboost_filesystem.so.1.65.1 ?
I would be glad if someone could provide an explanation in the context of the linking process. As far as I know, when you link a file against a library, you use the following flags:
~$ gcc my_program.c -Lpath/to/solib/for/static/linker -lnameoflib -wl,-rpath=path/to/solib/for/dynamic/linker
This -Wl,-rpath flag embeds in the executable the path of the library that the dynamic linker will search for at run time. In the case of a shared library that depends on other libraries - does it work the same?
Ok so that's how it works:
When linking a binary - whether it's an executable or another shared library - against a shared library, the static linker embeds in the binary the names of the libraries we linked against. Those libraries will be loaded by the dynmaic linker - ld.so - once the program is run by the user.
Now the question is where the dynamic linker will search for these libraries at runtime. Briefly, according to the man page of ld.so, when the dynamic linker first inspects the binary to resolve its dependencies, it goes through the strings of the dependencies. If a string contains a slash "/" then the string is interpreted as a path. This can happen if the library name was specified with a slash at link time, like this:
~$ gcc prog.c ../path/to/library.so
In this case, the string embeded in the executable will be: ../path/to/library.so and the dynamic linker will search it relatively to the location of the binary.
If not, it will search for the library in a list of locations (the whole list is specified in the man page). The first location is the directories specified in the DT_RUNPATH section attribute of the binary. This can be set at link time using the -Wl,-rpath flag:
~$ gcc prog.c -Lpath/of/lib/ -lmylib -Wl,-rpath=path/of/lib
In this case, the path/of/lib will be searched for the library by the dynamic library.
You can inspect this attribute using readelf:
~$ readelf -d binary | grep RUNPATH
In my case, the libmat.so library contained a RUNPATH attribute set to $ORIGIN, meaning that libraries will be searched in the same location of the binary library. Whereas, the libmwboost_filesystem.so.1.65.1 didn't have this attribute set, that is why ldd didn't find the library.
ldd is just using ld.so to try and load the libraries, and shows where it found them, according to the search path specified in the ld.so man page.
I want to create a simple library and after compilation and ar command I can get resulting .a file.
Now I want to add this file as a static library and use it in terminal, I dont know if its possible. but idea is from other library named Ctypes.sh on github. that library can be used to make syscalls from terminal or bash terminal.
I like to know how I can add my mylib.a as static libary and make it usable from terminal.
the library is simple I just want to invoke a few syscalls in linux from terminal.
I also looked into the code of ctypes.sh so my library is also be used to make some syscalls.
the reference I used above is here
https://github.com/taviso/ctypes.sh/wiki
Every command that you run on linux are binary file that you execute.
When you run a command like:
ls -a
it's like running:
./ls -a
Where the ./ls is the binary and -a a parameter.
All the binary used in a terminale is stocked in the bin (included in the default PATH). When you run a command your terminal will check in first, in the folder to find the binary and after, he gonna check every folder in the PATH environement variable. If you wan't to add a specific folder to the PATH to use a personnal folder for different baniry (check this link).
In your probleme you have a library with different function (I suppose) that you wan't to use in a terminale. Have 2 solution:
Split your library in multiple micro programme, that you can execute in the terminale,
Create a programme whit param to run different function.
I have a shared library's libmyworld.so in /opt/my_prog/lib and also in /home/user1/lib
Irrespective of the order I specified in LD_LIBRARY_PATH (LD_LIBRARY_PATH=/home/user1/lib;/opt/myprog/lib); my binary SHOULD always look for libmyworld.so FIRST in /opt/my_prog/lib;
Can this be done using GCC during compilation time? without modifying my_prog binary. Thanks in advance.
The search order for dynamic libraries in Linux (from ld.so man page) is the following
Using the DT_RPATH dynamic section attribute of the binary
if present and DT_RUNPATH attribute does not exist. Use of
DT_RPATH is deprecated.
Using the environment variable LD_LIBRARY_PATH. Except if
the executable is a setuid/setgid binary, in which case it
is ignored.
Using the DT_RUNPATH dynamic section attribute of the binary
if present.
From the cache file /etc/ld.so.cache which contains a
compiled list of candidate libraries previously found in the
augmented library path. If, however, the binary was linked
with -z nodeflib linker option, libraries in the default
library paths are skipped.
In the default path /lib, and then /usr/lib. If the binary
was linked with -z nodeflib linker option, this step is
skipped.
When linking, to set
DT_RUNPATH: use -Wl,--enable-new-dtags -Wl,-R$(RUNPATH)
DT_RPATH: use -Wl,--disable-new-dtags -Wl,-R$(RPATH)
In theory, it is better to use DT_RUNPATH as the LD_LIBRARY_PATH, on which the user has a control, has precedence. But here you want to avoid the user control, so use the DT_RPATH. In you link line:
-Wl,--disable-new-dtags -Wl,-R/opt/my_prog/lib
You can always launch your binary (here called foo) with
$ LD_LIBRARY_PATH=/opt/my_prog/lib foo
or make a shell script with the line above.
While compiling your source code use the below command
gcc -o [desired_executable_file_name] -L [Your shared library path] -l [your shared library name] -I [Header file path]
for example in your case
gcc -o my_word_exe -L /opt/my_prog/lib -lmyworld -I [header path if their]
Then it"ll take libmyworld.so in /opt/my_prog/lib this path
Use LD_PRELOAD.
LD_PRELOAD=/home/lib/libmyworld.so mybinary
The advantage is that you don't fiddle with LD_LIBRARY_PATH - your binary may depend on other shared libraries and it may need proper LD_LIBRARY_PATH/ld.so.conf/whatever.
PS. This is the least invasive and flexible solution, because does not affect loading of other libraries and does not hardcode paths in the user executable.
I am building a project that builds multiple shared libraries and executable files. All the source files that are used to build these binaries are in a single /src directory. So it is not obvious to figure out which source files were used to build each of the binaries (there is many-to-many relation).
My goal is to write a script that would parse a set of C files for each binary and make sure that only the right functions are called from them.
One option seems to be to try to extract this information from Makefile. But this does not work well with generated files and headers (due to dependence on Includes).
Another option could be to simply browse call graphs, but this would get complicated, because a lot of functions are called by using function pointers.
Any other ideas?
You can first compile your project with debug information (gcc -g) and use objdump to get which source files were included.
objdump -W <some_compiled_binary>
Dwarf format should contain the information you are looking for.
<0><b>: Abbrev Number: 1 (DW_TAG_compile_unit)
< c> DW_AT_producer : (indirect string, offset: 0x5f): GNU C 4.4.3
<10> DW_AT_language : 1 (ANSI C)
<11> DW_AT_name : (indirect string, offset: 0x28): test_3.c
<15> DW_AT_comp_dir : (indirect string, offset: 0x36): /home/auselen/trials
<19> DW_AT_low_pc : 0x82f0
<1d> DW_AT_high_pc : 0x8408
<21> DW_AT_stmt_list : 0x0
In this example, I've compiled object file from test_3, and it was located in .../trials directory. Then of course you need to write some script around this to collect related source file names.
First you need to separate the debug symbols from the binary you just compiled. check this question on how to do so:
How to generate gcc debug symbol outside the build target?
Then you can try to parse this file on your own. I know how to do so for Visual Studio but as you are using GCC I won't be able to help you further.
Here is an idea, need to refine based on your specific build. Make a build, log it using script (for example script log.txt make clean all). The last (or one of the last) step should be the linking of object files. (Tip: look for cc -o <your_binary_name>). That line should link all .o files which should have corresponding .c files in your tree. Then grep those .c files for all the included header files.
If you have duplicate names in your .c files in your tree, then we'll need to look at the full path in the linker line or work from the Makefile.
What Mahmood suggests below should work too. If you have an image with symbols, strings <debug_image> | grep <full_path_of_src_directory> should give you a list of C files.
You can use unix nm tool. It shows all symbols that are defined in the object. So you need to:
Run nm on your binary and grab all undefined symbols
Run ldd on your binary to grab list of all its dynamic dependencies (.so files your binary is linked to)
Run nm on each .so file youf found in step 2.
That will give you the full list of dynamic symbols that your binary use.
Example:
nm -C --dynamic /bin/ls
....skipping.....
00000000006186d0 A _edata
0000000000618c70 A _end
U _exit
0000000000410e34 T _fini
0000000000401d88 T _init
U _obstack_begin
U _obstack_newchunk
U _setjmp
U abort
U acl_extended_file
U bindtextdomain
U calloc
U clock_gettime
U closedir
U dcgettext
U dirfd
All those symbols with capital "U" are used by ls command.
If your goal is to analyze C source files, you can do that by customizing the GCC compiler. You could use MELT for that purpose (MELT is a high-level domain specific language to extend GCC) -adding your own analyzing passes coded in MELT inside GCC-, but you should first learn about GCC middle-end internal representations (Gimple, Tree, ...).
Customizing GCC takes several days of work (mostly because GCC internals are quite complex in the details).
Feel free to ask me more about MELT.
Hello Stack Overflow Community,
i am working on a c project to interleave multiple c programs into one binary, which can run the interleaved programs as treads or forks for benchmarking purposes.
Therefore i run make in each program folder of the desired programs and prelink all .o files with "ld -r" to one new .o file. After that i add a specific named function to each of these "big" .o files, which does nothing but run the main() of each program and providing the argc and argv. Then i use objcopy to localize every global Symbol except the unknown ones and the one of my specific function which shall run the main(). At last i link these manipulated .o files together with my program which runs the specific named functions as threads, or forks or after another.
Now to my Question/Problem:
I ran into a problem with static libs. I was using ffmpeg for testing, and it builds static libs such as libavcodc and libavutil and so on. Unfortunately, "ld -r" does not link .a files. So i tried to extract these libs with ar -x and then link the extracted .o files in the way mentioned above to the "big" new .o file. But i did not work because libavcodec and libavutil both include the file ff_inverse.o. That is obviously not a problem when i just build ffmpeg, which will link these static libraries. But still, both libraries include it, so there must be a machanism which makes the choice, which ff_inverse.o to use and to link. So my Question: How does this work? Where is the difference?
The way ld does it with normal linking is to prioritize the libraries. Libraries listed first in the command line are linked in first, and only if symbols still are unresolved does it move on to the next library. When linking static libraries, it ignores the name of each .o file, because the name is unnecessary, only the exported symbols are necessary. You may want to emulate that behavior, by extracting libraries in a sorted order.