From a function in a shared library, inside a running process (written in C), how do I discover where that shared library was loaded from?
All of the answers I've found involve using things such as ldd at the command line, or by peeking in /proc/self/maps.
On Win32, I'd just use GetModuleFileName(GetModuleHandle("foo.dll"), szPath, COUNTOF(szPath)). What's the Linux equivalent?
Bonus question: I need the same information in OS X.
You could use dl_iterate_phdr to iterate all loaded libraries and their segments (similar functionality is available for OSX, see e.g. this question). But most of the projects just parse /proc/self/maps.
As a side note, keep in mind that mappings may change dynamically (if libraries are loaded via dlopen) so reading them at startup may not be enough.
One method to achieve this is to make use of dladdr:
Code for the shared object:
$ cat so.c
#include <stdio.h>
#include <dlfcn.h>
void test_so_func()
{
Dl_info info;
if (dladdr(test_so_func, &info))
{
printf("Loaded from path = %s\n", info.dli_fname);
}
printf("hello\n");
}
Code for the main exec:
$ cat test.c
void test_so_func();
int main() {
test_so_func();
return 0;
}
Makefile:
$ cat Makefile
test: test.o libso.so
gcc test.o -o $# -Wl,-L.,-lso,-rpath,'$$ORIGIN'
clean:
-rm -f libso.so test.o test
libso.so: so.c
gcc -D_GNU_SOURCE=1 -fPIC -shared $< -o $# -lc -ldl
test.o: test.c
gcc -fPIC -c $< -o $#
Let's compile!
$ make
gcc -fPIC -c test.c -o test.o
gcc -D_GNU_SOURCE=1 -fPIC -shared so.c -o libso.so -lc -ldl
gcc test.o -o test -Wl,-L.,-lso,-rpath,'$ORIGIN'
Test this binary.
$ ./test
Loaded from path = /spare/scratch/1564054710/libso.so
hello
Verify that libso.so is indeed speaking the truth.
$ ldd ./test
linux-vdso.so.1 => (0x00007ffdf55d5000)
libso.so => /spare/scratch/1564054710/./libso.so (0x00007fbcc4602000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbcc4238000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbcc4034000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbcc4804000)
The credit for this answer goes to https://github.com/mingwandroid
Related
I have an executable shared_main , a shared library libbar.so and a dynamic load shared library libfoo.so (load in shared_main via dlopen).
shared_main doesn't use any symbols from libbar.so but libfoo.so does.
So gcc -g -Wall -o shared_main shared_main.c libbar.so -ldl doesn't link libbar.so to shared_main.
Checked via ldd shared_main.
How to let gcc force shared_main link libbar.so?
P.S. I know I can link libfoo.so with libbar.so. But I want to try if I can force shared_main to link libbar.so here.
shared_main.c
#include <stdio.h>
#include <dlfcn.h>
#include <stdlib.h>
int main(){
void* libHandle = dlopen("./libfoo.so", RTLD_LAZY);
if(libHandle == NULL){
printf("dlopen:%s", dlerror());
exit(1);
}
int(*xyz)(int);
xyz = (int(*)(int)) dlsym(libHandle, "xyz");
if(xyz == NULL){
printf("dlsym:%s", dlerror());
exit(1);
}
int b = xyz(3);
printf("xyz(3): %d\n", b);
}
foo.c (libfoo.so)
void func_in_bar();
int xyz(int b){
func_in_bar();
return b + 10;
}
bar.c (libbar.so)
//mimic python share library runtime
#include <stdio.h>
void func_in_bar(){
printf("I am a Function in bar\n");
}
void another_func_in_bar(){
printf("I am another function in bar\n");
}
makefile
shared_main:
gcc -g -Wall -o shared_main shared_main.c libbar.so -ldl
shared:
gcc -g -Wall -fPIC -shared -o libfoo.so foo.c
gcc -g -Wall -fPIC -shared -o libbar.so bar.c
You have an XY-problem, where X is: libfoo has unresolved symbols, but the linker doesn't warn about it
So use the -z defs option linkage-time, and when you get the linker error about the unresolved symbol add -lfoo to the linkage command.
That's still not enough, you will have to use a -L and a -Wl,-rpath option too. Here is a complete Makefile:
# Makefile
# LIBDIR should be the final place of the shared libraries
# such as /usr/local/lib or ~/libexec/myproject
LIBDIR := ${PWD}
TARGETS := shared_main libbar.so libfoo.so
all: ${TARGETS}
clean:
rm -f ${TARGETS} 2>/dev/null || true
shared_main: shared_main.c
gcc -g -Wall -o shared_main shared_main.c -ldl
libbar.so: bar.c
gcc -g -Wall -fPIC -shared -o libbar.so bar.c
libfoo.so: foo.c libbar.so
gcc -g -Wall -fPIC -shared -z defs -o libfoo.so foo.c \
-L${LIBDIR} -Wl,-rpath,${LIBDIR} -lbar
Edit: nonetheless, here is a hackish solution for the original question: use option -Wl,--no-as-needed
shared_main:
gcc -g -Wall -o shared_main shared_main.c \
-Wl,--no-as-needed -Wl,-rpath,${PWD} libbar.so -ldl
Everything works just fine for me, with unmodified files from OP.
$ make shared
gcc -g -Wall -fPIC -shared -o libfoo.so foo.c
gcc -g -Wall -fPIC -shared -o libbar.so bar.c
$ make shared_main
gcc -g -Wall -o shared_main shared_main.c libbar.so -ldl
$ LD_LIBRARY_PATH=. ldd shared_main
linux-vdso.so.1 (0x00007ffccb5f2000)
libbar.so => ./libbar.so (0x00007f78f6ce0000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f78f6cd1000)
libc.so.6 => /lib64/libc.so.6 (0x00007f78f6b06000)
/lib64/ld-linux-x86-64.so.2 (0x00007f78f6ce7000)
$ LD_LIBRARY_PATH=. ./shared_main
I am a Function in bar
xyz(3): 13
I only needed to help the library loader out a bit using LD_LIBRARY_PATH.
$ gcc --version
gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ ld --version
GNU ld version 2.37-17.fc35
Copyright (C) 2021 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
The answer is -Wl,--no-as-needed.
For my example, the full command is:
gcc -g -Wall -o shared_main shared_main.c -Wl,--no-as-needed libbar.so -ldl
From https://man7.org/linux/man-pages/man1/ld.1.html :
--as-needed
--no-as-needed
This option affects ELF DT_NEEDED tags for dynamic libraries
mentioned on the command line after the --as-needed option.
Normally the linker will add a DT_NEEDED tag for each dynamic
library mentioned on the command line, regardless of whether
the library is actually needed or not. --as-needed causes a
DT_NEEDED tag to only be emitted for a library that at that
point in the link satisfies a non-weak undefined symbol
reference from a regular object file or, if the library is
not found in the DT_NEEDED lists of other needed libraries, a
non-weak undefined symbol reference from another needed
dynamic library. Object files or libraries appearing on the
command line after the library in question do not affect
whether the library is seen as needed. This is similar to
the rules for extraction of object files from archives.
--no-as-needed restores the default behaviour.
I have one program and it uses static or shared library. Now I want to do something like
make static
or
make shared
to compile the project in two cases.
My Makefile looks like
shared: main.o libresult.so
gcc -o shared main.o -L. -lresult -Wl,-rpath,.
main.o: main.c
gcc -c main.c
libresult.so: func.o
gcc -shared -o libresult.so func.o
func.o: func.c
gcc -c -fPIC func.c
static: main.o libresult.a
gcc -o static main.o -L. -lresult
main.o: main.c
gcc -c main.c
libresult.a: func.o
ar cr libresult.a func.o
func.o: func.c
gcc -c func.c
clean:
rm -f *.o *.a *.so static shared
Off course, terminal gives some warnings. It works well, but I dont think that
is much beautiful :). How to make it better?
1) You have two copies of the main.o rule:
main.o: main.c
gcc -c main.c
Delete one of them.
2) You have two versions of the func.o rule:
func.o: func.c
gcc -c -fPIC func.c
func.o: func.c
gcc -c func.c
This is a more serious problem. The object you put in the shared library (libresult.so) must be compiled with -fPIC; the object you put in the static library (libresult.a) may be compiled with -fPIC, but there is no reason to do so and it may prevent the compiler form performing some optimisation. Make doesn't know your intentions, so it is better to make two versions of the object with different names:
func_so.o: func.c
gcc -c -fPIC func.c -o func_so.o
func_a.o: func.c
gcc -c func.c -o func_a.o
(Don't forget to modify the rules that rely on these objects.)
3) Use automatic variables to reduce redundancy and make your makefile cleaner. For example:
main.o: main.c
gcc -c $< -o $#
Further improvements are possible, once you are comfortable with these.
I have problem link libgcc into a static linked .so
it only happens when linking 64bit module with -m64
Ubuntu 64bit 12.10 gcc 4.7
also failed on Ubuntu 64bit 12.04 gcc 4.6
32bit no problem
$gcc -fPIC -c -o hello.o hello.c -m32
$gcc -shared -m32 hello.o -o libhello.so -static-libgcc -Wl,-Bstatic -lc
$ ldd libhello.so
statically linked
64bit failed
$ make
gcc -fPIC -c -o hello.o hello.c
gcc -shared -m64 hello.o -o libhello.so -static-libgcc -Wl,-Bstatic -lc
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/libc.a(iofclose.o): relocation R_X86_64_32 against `__gcc_personality_v0' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../x86_64-linux-gnu/libc.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
make: *** [libhello.so] Error 1
hello.c
#include <stdio.h>
int f(){
FILE *out = fopen("/tmp/x.log", "wb");
fclose(out);
return 1;
}
Makefile
all: libhello.so
libhello.so: hello.o
gcc -shared -m64 hello.o -o libhello.so -static-libgcc -Wl,-Bstatic -lc
hello.o: hello.c
gcc -fPIC -c -o hello.o hello.c
clean:
rm -f hello.o libhello.so
The answer is basically "you can't do that." You're trying to link non-PIC code into a shared library, which is simply impossible on the x86_64 (amd64) architecture. You would need a static but PIC version of libgcc, and I suspect that would be only the start of your problems.
One of the reasons why libgcc is normally shared is that a given running executable has to have one and only one copy of some of the key data structures that libgcc maintains. Static linking makes sense for a final executable, since that one and only one copy will be the one statically linked into the executable, but the whole point of a dynamic object is to be loaded into another executable (which in turn will have its own copy of libgcc, either shared or static).
If you have a simple C program, like
int main(void) {return 0;}
It can be compiled with gcc -o test test.c.
As I understand, gcc performs compiling, assembling then linking. The latter two steps are achieved by it running as and ld.
I can generate the assembly code by using gcc -S test.c.
What would you type into a terminal, to convert the assembly code into an executable?
(the reason for doing so is to learn assembly)
These are the different stages using gcc
gcc -E --> Preprocessor, but don't compile
gcc -S --> Compile but don't assemble
gcc -c --> Preprocess, compile, and assemble, but don't link
gcc with no switch will link your object files and generate the executable
// main.c
#include <stdio.h>
int main(void)
{
printf("Hello World !\n");
return 0;
}
For preprocessing, compiling, assembling and then finally linking the simple aforementioned hello world program, follow the steps below:
Step 1/4) Preprocess main.c to generate main.i:
$: gcc -E main.c -o main.i
NOTE: You could call the C preprocessor directly as well:
$: cpp main.c -o main.i
Step 2/4) Compile main.i to generate main.s:
$: gcc -S main.i -o main.s
Step 3/4) Assemble main.s to generate main.o:
$: as main.s -o main.o
NOTE: You can combine the aforementioned steps 1, 2 and 3 by using the -c (small C) flag of gcc:
$: gcc -c main.s -o main.o // OR $: gcc -c main.c -o main.o
Step 4/4) Link main.o with other necessary object files namely, crti.o & crtn.o (they define function prologs & epilogs, respectively), crt1.o (contains _start symbol for bootstrapping the initial execution of the program), libc.so path or -lc flag for libc and then finally set the name of the dynamic linker, to generate a dynamically linked ELF executable:
On x86_64:
$: ld /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/x86_64-linux-gnu/crtn.o /usr/lib/x86_64-linux-gnu/crt1.o -lc main.o -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o main_ELF_executable
OR (if you'd like to specify path to libc.so)
$: ld /usr/lib/x86_64-linux-gnu/crti.o /usr/lib/x86_64-linux-gnu/crtn.o /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/libc.so main.o -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o main_ELF_executable
On 32-bit ARM:
$: ld /usr/lib/arm-linux-gnueabihf/crti.o /usr/lib/arm-linux-gnueabihf/crtn.o /usr/lib/arm-linux-gnueabihf/crt1.o -lc main.o -dynamic-linker /lib/ld-linux.so.3 -o main_ELF_executable
OR (if you'd like to specify path to libc.so)
$: ld /usr/lib/arm-linux-gnueabihf/crti.o /usr/lib/arm-linux-gnueabihf/crtn.o /usr/lib/arm-linux-gnueabihf/crt1.o /usr/lib/arm-linux-gnueabihf/libc.so main.o -dynamic-linker /lib/ld-linux-armhf.so.3 -o main_ELF_executable
You can then run the ELF executable 'main_ELF_executable':
$: ./main_ELF_executable
Hello World !
Sources:
https://linux.die.net/man/1/gcc
https://linux.die.net/man/1/ld
https://dev.gentoo.org/~vapier/crt.txt
gcc test.s -o test will compile the test from test.s for you.
NASM might also be worth your time -- it might be easier / more friendly than gcc for compiling assembly.
After you do gcc -S -o test.s test.c, type gcc -o test test.s.
As you may or may not know, the four stages of compilation are to preprocess (-E), compile to assembly (-S), assemble to object code (-c), and finally link. The hardest for me to figure out was how to use the preprocessor output. Here's how to do it:
gcc -E hello.c | gcc -S -xc -o hello.s -
gcc -c hello.s -o hello.o
gcc hello.o -o hello
You can have gcc start and stop the compilation process wherever you want. gcc test.s -o test will have it compile test.s from assembly into an executable.
what I did was first I run the preprocessor by
clang++ test.cpp -E > test.i
then compiled it with ...
clang++ -S test.i
it should create a assembly file test.s ... then make the machine insturction file by
as test.s -o test.o
now you need to link it which is kinda confusing for dumb peoples like me ...
so we don't know the arguments for our last process which is linking ... to find out ... run
clang++ -v test.s
it should give you some big text of something ... find this line "-dynamic-linker" ... there's definately a -dynamic-linker in your output text ... now copy the text from -dynamic-linker to rest of the output ... just copy everything afterwards including "-dynamic-linker" ... now what i got is ...
-dynamic-linker /system/bin/linker -o a.out /data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib/crtbegin_dynamic.o -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0 -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../.. -L/data/data/com.termux/files/usr/lib -L/system/lib /data/data/com.termux/files/usr/tmp/test-169b42.o -lc++_shared -lgcc -ldl -lm -lc -lgcc -ldl /data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib/crtend_android.o
in this what you have to change is where your object file is ... in my case it is /data/data/com.termux/files/usr/tmp/test-169b42.o ... i need to change it to where my test.o file is ... /data/data/com.termux/files/home/CPP/Cpp_Log/hello_world/test.o ... this is where my test.o file is ...
so the argument we have to pass is ...
-dynamic-linker /system/bin/linker -o a.out /data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib/crtbegin_dynamic.o -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0 -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../.. -L/data/data/com.termux/files/usr/lib -L/system/lib /data/data/com.termux/files/home/CPP/Cpp_Log/hello_world/main.o -lc++_shared -lgcc -ldl -lm -lc -lgcc -ldl /data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib/crtend_android.o
now to link ... use the ld ... so the command is ld args -o test or in our case ...
ld -dynamic-linker /system/bin/linker -o a.out /data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib/crtbegin_dynamic.o -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0 -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib -L/data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../.. -L/data/data/com.termux/files/usr/lib -L/system/lib /data/data/com.termux/files/home/CPP/Cpp_Log/hello_world/main.o -lc++_shared -lgcc -ldl -lm -lc -lgcc -ldl /data/data/com.termux/files/usr/lib/gcc/arm-linux-androideabi/11.1.0/../../../../lib/crtend_android.o -pie -o test ...
since andriod 5+ can only run pie elf executables ... i also added the "-pie" (position independent executable) before the -o test ...
now it should give you a executable file test ... just run it by ./test
it should work
I am new to g++ and Makefile. I am trying to link this BeBOP SMC library, which is in my lib directory. Under the lib directory are bebop_util and sparse_matrix_converter, both of which have already been built without errors. I see libbebop_util.a, libbebop_util.so under bebop_util and libsparse_matrix_converter.a, libsparse_matrix_converter.so under sparse_matrix_converter. Below is the source:
Makefile
CC=g++
CFLAGS=-c
# CFLAGS=-c -Wall
INCLUDE_DIRS=-Ilib/bebop_util/include -Ilib/sparse_matrix_converter/include
LIB_DIRS=-Llib/bebop_util -Llib/sparse_matrix_converter
LIBS=-lbebop_util -lsparse_matrix_converter
test.out: test.o
$(CC) -o test.out $(LIB_DIRS) $(LIBS) test.o
test.o: test.cpp
$(CC) $(CFLAGS) $(INCLUDE_DIRS) test.cpp
clean:
rm -f test.o test.out
test.cpp
extern "C" {
#include <bebop/smc/sparse_matrix.h>
#include <bebop/smc/sparse_matrix_ops.h>
}
int main(int argc, const char* argv[])
{
struct sparse_matrix_t* A = load_sparse_matrix (MATRIX_MARKET, "sample_i
nput");
destroy_sparse_matrix(A);
return 0;
}
As a safeguard, I also have the LD_LIBRARY_PATH set:
login4% setenv | grep LD_LIBRARY_PATH
LD_LIBRARY_PATH=/share/apps/teragrid/globus-4.0.8-r1/myproxy-3.4/lib:/share/apps/teragrid/globus-4.0.8-r1/lib:/share/apps/teragrid/srb-client-3.4.1-r1/lib:/opt/apps/pgi7_2/mvapich/1.0.1/lib:/opt/apps/pgi7_2/mvapich/1.0.1/lib/shared:/opt/apps/pgi/7.2-5/linux86-64/7.2-5/libso:/opt/gsi-openssh-4.3/lib:/opt/apps/binutils-amd/070220/lib64:/share/home/01355/tomwang/cs380p_assn3/lib:/share/home/01355/tomwang/cs380p_assn3/lib/bebob_util:/share/home/01355/tomwang/cs380p_assn3/lib/sparse_matrix_converter
Output
login3% make
g++ -c -Ilib/bebop_util/include -Ilib/sparse_matrix_converter/include test.cpp
g++ -o test.out -Llib/bebop_util -Llib/sparse_matrix_converter -lbebop_util -lsparse_matrix_converter test.o
login3% ./test.out
./test.out: error while loading shared libraries: libbebop_util.so: cannot open shared object file: No such file or directory
Please suggest what may be wrong or additional info for me to provide. Thanks.
Tom
Are you sure the directory that libbebop_util.so is in is mentioned in your LD_LIBRARY_PATH? Based on your build line, the following should work:
env LD_LIBRARY_PATH=./lib:${LD_LIBRARY_PATH} ./test_out
It looks like you're not having problems with linking. Instead, the problem is that your built executable has a reference to libbebop_util.so that is invalid.
Try running ldd test.out to see where it's looking for the shared libraries.