What is the purpose of specifying library dependencies with gcc?
I have written a shared library, libPulse_IO.so, that calls functions from both libusb-1.0.so and libpcap.so.
i.e. in pulse_IO.c there is stuff like:
#include <libusb-1.0/libusb.h>
#include <pcap.h>
ret = libusb_init(&pdw_io->context);
pdw_io->pcap=pcap_open_live(pdw_io->eth_name, ETH_SNAPLEN, ETH_PROMISCUOUS, ETH_TIMEOUT, pcap_errbuf);
Now when I come to build my library, I notice that it seems to make no difference if I specify -l libusb-1.0 -l pcap in the call to gcc
i.e. I can run (no mention of -l usb-1.0 or -l pcap):
gcc -fPIC -g -Wall -fvisibility=hidden -I../../../g2/src -I../../core/src pulse_IO.c -c -o ../build/linux/debug/pulse_IO.o
gcc -fPIC -g -Wall -fvisibility=hidden -I../../../g2/src -I../../core/src -shared -Wl,-soname,libPulse_IO_dbg.so ../build/linux/debug/pulse_IO.o -o ../build/linux/debug/libPulse_IO_dbg.so
or (-l usb-1.0 or -l pcap specified):
gcc -fPIC -g -Wall -fvisibility=hidden -I../../../g2/src -I../../core/src -lusb-1.0 -lpcap pulse_IO.c -c -o ../build/linux/debug/pulse_IO.o
gcc -fPIC -g -Wall -fvisibility=hidden -I../../../g2/src -I../../core/src -lusb-1.0 -lpcap -shared -Wl,-soname,libPulse_IO_dbg.so ../build/linux/debug/pulse_IO.o ../build/linux/debug/shared.o -o ../build/linux/debug/libPulse_IO_dbg.so
Both instances build without error.
Running ldd in both cases yields the same output:
ldd ../build/linux/debug/libPulse_IO_dbg.so
linux-vdso.so.1 => (0x00007ffea93ee000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd43af68000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd43b53f000)
No mention of a dependency on libusb-1.0.so or libpcap.so - how?
Examining the shared library produced in both instances shows that there are undefined references to the libusb and pcap functions, i.e.
nm ../build/linux/debug/libPulse_IO_dbg.so
U __assert_fail##GLIBC_2.2.5
U atof##GLIBC_2.2.5
U atoi##GLIBC_2.2.5
...
U libusb_init
U libusb_kernel_driver_active
U libusb_open_device_with_vid_pid
...
U pcap_open_live
U pcap_perror
U pcap_sendpacket
U pcap_setfilter
So the library is built in both instances, with or without the -l library specifiers, and in both instances we can see calls to the libusb-1.0 and libpcap functions in the symbol table.
Which brings me back to my original question.. what is the purpose of specifying library dependencies with gcc?
My expectation was that the calls to gcc would have failed without specifying the libusb-1.0 and libpcap library dependencies?
Related
I have an executable shared_main , a shared library libbar.so and a dynamic load shared library libfoo.so (load in shared_main via dlopen).
shared_main doesn't use any symbols from libbar.so but libfoo.so does.
So gcc -g -Wall -o shared_main shared_main.c libbar.so -ldl doesn't link libbar.so to shared_main.
Checked via ldd shared_main.
How to let gcc force shared_main link libbar.so?
P.S. I know I can link libfoo.so with libbar.so. But I want to try if I can force shared_main to link libbar.so here.
shared_main.c
#include <stdio.h>
#include <dlfcn.h>
#include <stdlib.h>
int main(){
void* libHandle = dlopen("./libfoo.so", RTLD_LAZY);
if(libHandle == NULL){
printf("dlopen:%s", dlerror());
exit(1);
}
int(*xyz)(int);
xyz = (int(*)(int)) dlsym(libHandle, "xyz");
if(xyz == NULL){
printf("dlsym:%s", dlerror());
exit(1);
}
int b = xyz(3);
printf("xyz(3): %d\n", b);
}
foo.c (libfoo.so)
void func_in_bar();
int xyz(int b){
func_in_bar();
return b + 10;
}
bar.c (libbar.so)
//mimic python share library runtime
#include <stdio.h>
void func_in_bar(){
printf("I am a Function in bar\n");
}
void another_func_in_bar(){
printf("I am another function in bar\n");
}
makefile
shared_main:
gcc -g -Wall -o shared_main shared_main.c libbar.so -ldl
shared:
gcc -g -Wall -fPIC -shared -o libfoo.so foo.c
gcc -g -Wall -fPIC -shared -o libbar.so bar.c
You have an XY-problem, where X is: libfoo has unresolved symbols, but the linker doesn't warn about it
So use the -z defs option linkage-time, and when you get the linker error about the unresolved symbol add -lfoo to the linkage command.
That's still not enough, you will have to use a -L and a -Wl,-rpath option too. Here is a complete Makefile:
# Makefile
# LIBDIR should be the final place of the shared libraries
# such as /usr/local/lib or ~/libexec/myproject
LIBDIR := ${PWD}
TARGETS := shared_main libbar.so libfoo.so
all: ${TARGETS}
clean:
rm -f ${TARGETS} 2>/dev/null || true
shared_main: shared_main.c
gcc -g -Wall -o shared_main shared_main.c -ldl
libbar.so: bar.c
gcc -g -Wall -fPIC -shared -o libbar.so bar.c
libfoo.so: foo.c libbar.so
gcc -g -Wall -fPIC -shared -z defs -o libfoo.so foo.c \
-L${LIBDIR} -Wl,-rpath,${LIBDIR} -lbar
Edit: nonetheless, here is a hackish solution for the original question: use option -Wl,--no-as-needed
shared_main:
gcc -g -Wall -o shared_main shared_main.c \
-Wl,--no-as-needed -Wl,-rpath,${PWD} libbar.so -ldl
Everything works just fine for me, with unmodified files from OP.
$ make shared
gcc -g -Wall -fPIC -shared -o libfoo.so foo.c
gcc -g -Wall -fPIC -shared -o libbar.so bar.c
$ make shared_main
gcc -g -Wall -o shared_main shared_main.c libbar.so -ldl
$ LD_LIBRARY_PATH=. ldd shared_main
linux-vdso.so.1 (0x00007ffccb5f2000)
libbar.so => ./libbar.so (0x00007f78f6ce0000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f78f6cd1000)
libc.so.6 => /lib64/libc.so.6 (0x00007f78f6b06000)
/lib64/ld-linux-x86-64.so.2 (0x00007f78f6ce7000)
$ LD_LIBRARY_PATH=. ./shared_main
I am a Function in bar
xyz(3): 13
I only needed to help the library loader out a bit using LD_LIBRARY_PATH.
$ gcc --version
gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ ld --version
GNU ld version 2.37-17.fc35
Copyright (C) 2021 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.
The answer is -Wl,--no-as-needed.
For my example, the full command is:
gcc -g -Wall -o shared_main shared_main.c -Wl,--no-as-needed libbar.so -ldl
From https://man7.org/linux/man-pages/man1/ld.1.html :
--as-needed
--no-as-needed
This option affects ELF DT_NEEDED tags for dynamic libraries
mentioned on the command line after the --as-needed option.
Normally the linker will add a DT_NEEDED tag for each dynamic
library mentioned on the command line, regardless of whether
the library is actually needed or not. --as-needed causes a
DT_NEEDED tag to only be emitted for a library that at that
point in the link satisfies a non-weak undefined symbol
reference from a regular object file or, if the library is
not found in the DT_NEEDED lists of other needed libraries, a
non-weak undefined symbol reference from another needed
dynamic library. Object files or libraries appearing on the
command line after the library in question do not affect
whether the library is seen as needed. This is similar to
the rules for extraction of object files from archives.
--no-as-needed restores the default behaviour.
I am using GNU g++ 4.9.2 compiler both on Solaris and Linux.
On Solaris platform, to create a shared library from a source file (a.c), I use the following command:
g++ -G a.c -o a
a becomes a shared library
a.c contains the following code:
void libfn1()
{
}
If I try not to use -G option i.e. compile as:
g++ a.c -o a
It gets a linker error: Undefined Symbol main
But, on Linux, if I do the same thing: it says:
g++: error: unrecognized command line option -G
How to create a shared library on Linux? What is the g++ option for that?
The g++ documentation says this:
These additional options are available on System V Release 4 for
compatibility with other compilers on those systems:
-G Create a shared object. It is recommended that -symbolic or -shared be
used instead.
Normally you want to generate position independent code too, for a shared library, with the -fPIC flag.
So you'd want to run:
g++ -fPIC -shared a.c -o liba.so
The process to create a shared library on a Linux system is a bit different.
Shared libraries on Linux are .so (for "shared object") files, not .g.
You do it like this:
First, you need to generate position-independent code from your C++ source. That is so your library works from wherever it is called. To do that, you should use g++'s -fPIC flag.
So, for each source file you want to be included in your library, you should only compile it to position-independent code. We'll handle linking later.
For each source file:
g++ -c -fPIC file.cpp
(The -c flag tells g++ "compile, don't link").
for each file.cpp, g++ will generate file.o, an object file containing position-independent code.
To then build the object files into a shared library, you should use
g++ -o -shared myLibrary.so {all_object_files}
So if you have file1.o, file2.o and file3.o, the command would be:
g++ -shared -o myLibrary.so file1.o file2.o file3.o
Of course, if you have a lot of files this can get pretty tedious, so you should write a Makefile to automate this process for you! Here's an example:
myLibrary.so: file1.o file2.o file3.o
$(CXX) -shared $^ -o $#
file1.o file2.o file3.o : CXXFLAGS+=-fPIC
I want to create a static library implementing a TUI interface using ncurses. I'm new to this topic and applying these answers didn't work
out for me.
rpath
combine multiple libraries
I want to link varViewer.c in a library. I use this code to compile a example file.
Compiling src/example.c and src/viewer/varViewer.c:
gcc -Wall -Wextra -Wnonnull -Isrc/viewer/
-c src/example.c -o obs/example.o -lncurses
-fsanitize=undefined -fprofile-arcs -ftest-coverage -Winline -O2
gcc -Wall -Wextra -Wnonnull -Isrc/viewer/
-c src/viewer/varViewer.c -o obs/viewer/varViewer.o -lncurses
-fsanitize=undefined -fprofile-arcs -ftest-coverage -Winline -O2
Linking obs/example.o to bin/example:
gcc -o bin/example -Isrc/viewer/
obs/example.o obs/viewer/varViewer.o -lncurses
-fsanitize=undefined -fprofile-arcs -ftest-coverage -Winline -O2
bin/example works as expected. I tried to compile a suitable object file:
Compiling src/viewer/varViewer.c:
gcc -Wall -Wextra -Wnonnull -Isrc/viewer/
-c src/viewer/varViewer.c -o lib/objects/varViewer.o
-lncurses -fPIC -O2
and put it into library:
ar -cvr lib/libvarViewer.a lib/objects/varViewer.o
when I try to use it in other projects, a include it using
-L ..../src/viewer/lib -lvarViewer
flags but all references to functions I used from ncurses library and
string.h are undefined:
nm /lib/libvarViewer.a
....
U __snprintf_chk
U __stack_chk_fail
U start_color
U stdscr
U strchr
U strlen
U strncmp
U strncpy
U strnlen
U waddnstr
U wborder
U wclear
U wcolor_set
U wgetnstr
U winsdelln
U winsnstr
U wmove
U wrefresh
What is the correct way to call ar? I already tried to produce a "thin" library using -T option.
The command
gcc -Wall -Wextra -Wnonnull -Isrc/viewer/
-c src/viewer/varViewer.c -o lib/objects/varViewer.o
-lncurses -fPIC -O2
will simply compile the varViewer.c file and thus, will ignore the libraries because it won't link to a final executable.
Consequently, the object file varViewer.o will have undefined references (U reported in nm as you have shown) to the symbols not defined within varViewer.c. These not only include the libncurses library but also the regular libc library (which provides symbols such as strlen, strncpy, ...). However, remember that libc will be automatically added into your link stage by the compiler unless you explicitly state otherwise.
You could use the command ar x to extract the object files (.o) from other static libraries and then use ar again to generate a new library. While this is possible I don't think that including all the libraries into yours is a good idea. I think that it is better to link against all the libraries needed because that ensures that the application is linked against the latest version available -- while if you embed one library into yours, then will stick to that version until you upgrade it.
a.so defines function A(), b.so defines function B() and calls A() that is defined in a.so. b.so is linked to a.so as below
gcc -fPIC -shared B.c -o libb.so -la
Now I created a binary which is calling B() that is defined in b.so
gcc mybin.c -o mybin -lb
gcc is checking for every symbol in b.so and throws error saying A() is not defined.
gcc mybin.c -o mybin -lb -la
The above works but I had to link mybin with a.so even though it is not directly related to a.so. My requirement is that if b.so is properly linked with a.so, linking mybin with just b.so should work.
Is this possible ??
When linking the executable the linker wants to know that the libraries needed by libb.so exist so it can check they resolve any undefined references in libb.so
The linker looks for the required libraries in the usual places, so if that wouldn't find liba.so then you can tell the linker where to look by using LD_LIBRARY_PATH as shown by another answer, or by using the linker option that exists specifically for this purpose, -rpath-link
gcc mybin.c -o mybin -lb -Wl,-rpath-link=.
(The -Wl, prefix is how you tell GCC to pass an option to the linker, which is needed for linker options that GCC doesn't know about directly.)
Alternatively, you can tell the linker to allow undefined symbols in libb.so in which case it doesn't try to find liba.so to resolve the reference to A(), and trusts that you'll link correctly and ensure the library is available at run-time. That is done with the --allow-shlib-undefined option:
gcc mybin.c -o mybin -lb -Wl,--allow-shlib-undefined
However, it's generally better to let the linker check that all symbols are defined, and tell it how to find the required libraries, as it finds real problems sooner.
The -rpath-link option only tells the linker where to look for additional shared libraries, it has no visible effect on the linked executable (i.e. it doesn't record the path to the library in the executable anywhere). Another alternative is to create libb.so so that it contains the knowledge of how to find liba.so embedded within it. That is done with the -rpath linker option, e.g. if liba.so is in /some/path you could do:
gcc -fPIC -shared B.c -o libb.so -la -Wl,-rpath=/some/path
This puts a DT_RPATH tag in libb.so and the linker will use that to find its dependencies:
readelf -d libb.so | fgrep RPATH
0x000000000000000f (RPATH) Library rpath: [/some/path]
Now you can link the executable without needing any knowledge of liba.so, the linker will see that libb.so needs liba.so and will use the RPATH to find it:
gcc mybin.c -o mybin -lb
This works fine if the path /some/path/liba.so is fixed, but it might not be true that the library is found in the same location during development and after your executable is deployed. In that case you can still use -rpath-link for the executable to tell the linker where to look for it during linking, and rely on the RPATH to find it at run-time. Or you can use the special string $ORIGIN in the RPATH which the dynamic linker expands to the location of the object containing the RPATH, e.g. if liba.so and libb.so are always in the same directory, you can link libb.so like this (note the quotes to prevent the shell expanding $ORIGIN):
gcc -fPIC -shared B.c -o libb.so -la '-Wl,-rpath=$ORIGIN'
readelf -d libb.so | fgrep RPATH
0x000000000000000f (RPATH) Library rpath: [$ORIGIN]
The missing bit was the LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=.
gcc -fPIC -shared a.c -o liba.so
gcc -fPIC -shared b.c -o libb.so -la
gcc mybin.c -o mybin -lb -L.
The reason -L does not work seems it that this flag it used to « Add directory dir to the list of directories to be searched for -l » , see the GCC doc. And libba.so is not specified in a -l.
Here are the steps:
gcc -fPIC -shared a.c -o liba.so
gcc -fPIC -shared b.c -o libb.so
gcc mybin.c -o mybin -lb -la -L. # -L to look for the directory to load .so files
Then make sure you do:
export LD_LIBRARY_PATH=<your_directory>
You need to have liba.so and libb.so mentioned in gcc mybin.c -o mybin -lb -la -L. as the function A() needs to be loaded at runtime. b.so IS NOT LINKED with a.so. Linking happens that runtime in shared library. They are compiled and any reference to functions or extern variables are not resolved.
If you want to include b.so but not a.so then compile the b as static library thereby it loads a into its binary. And then when you compile mybin you will only need b.lib
I'm trying to compile my project and I use the lib ncurse. And I've got some errors when compiler links files.
Here is my flags line in Makefile:
-W -Wall -Werror -Wextra -lncurses
I've included ncurses.h
Some layouts :
prompt$> dpkg -S curses.h
libslang2-dev:amd64: /usr/include/slcurses.h
libncurses5-dev: /usr/include/ncurses.h
libncurses5-dev: /usr/include/curses.h
prompt$> dpkg -L libncurses5-dev | grep .so
/usr/lib/x86_64-linux-gnu/libncurses.so
/usr/lib/x86_64-linux-gnu/libcurses.so
/usr/lib/x86_64-linux-gnu/libmenu.so
/usr/lib/x86_64-linux-gnu/libform.so
/usr/lib/x86_64-linux-gnu/libpanel.s
And here are my erros :
gcc -W -Wall -Werror -Wextra -I./Includes/. -lncurses -o Sources/NCurses/ncurses_init.o -c Sources/NCurses/ncurses_init.c
./Sources/NCurses/ncurses_init.o: In function `ncruses_destroy':
ncurses_init.c:(.text+0x5): undefined reference to `endwin'
./Sources/NCurses/ncurses_init.o: In function `ncurses_write_line':
ncurses_init.c:(.text+0xc5): undefined reference to `mvwprintw'
./Sources/NCurses/ncurses_init.o: In function `ncurses_init':
ncurses_init.c:(.text+0xee): undefined reference to `initscr'
collect2: error: ld returned 1 exit status
Thanks a lot
You need to change your makefile so that the -lncurses directive comes after your object code on the gcc command line, i.e. it needs to generate the command:
gcc -W -Wall -Werror -Wextra -I./Includes/. -o Sources/NCurses/ncurses_init.o -c Sources/NCurses/ncurses_init.c -lncurses
This is because object files and libraries are linked in order in a single pass.
In C++ , I fixed it just by linking the ncurses library .
Here is the command :
g++ main.cpp -lncurses
I got flags to correct order by using LDLIBS variable:
ifndef PKG_CONFIG
PKG_CONFIG=pkg-config
endif
CFLAGS+=-std=c99 -pedantic -Wall
LDLIBS=$(shell $(PKG_CONFIG) --libs ncurses)
man gcc | grep -A10 "\-l library"
-l library
Search the library named library when linking. (The second alternative with the library as a separate argument is only for POSIX
compliance and is not recommended.)
It makes a difference where in the command you write this option; the linker searches and processes libraries and object files
in the order they are specified. Thus, foo.o -lz bar.o searches
library z after file foo.o but
before bar.o. If bar.o refers to functions in z, those functions may not be loaded.