Why does the order in which libraries are linked sometimes cause errors in GCC?
(See the history on this answer to get the more elaborate text, but I now think it's easier for the reader to see real command lines).
Common files shared by all below commands
// a depends on b, b depends on d
$ cat a.cpp
extern int a;
int main() {
return a;
}
$ cat b.cpp
extern int b;
int a = b;
$ cat d.cpp
int b;
Linking to static libraries
$ g++ -c b.cpp -o b.o
$ ar cr libb.a b.o
$ g++ -c d.cpp -o d.o
$ ar cr libd.a d.o
$ g++ -L. -ld -lb a.cpp # wrong order
$ g++ -L. -lb -ld a.cpp # wrong order
$ g++ a.cpp -L. -ld -lb # wrong order
$ g++ a.cpp -L. -lb -ld # right order
The linker searches from left to right, and notes unresolved symbols as it goes. If a library resolves the symbol, it takes the object files of that library to resolve the symbol (b.o out of libb.a in this case).
Dependencies of static libraries against each other work the same - the library that needs symbols must be first, then the library that resolves the symbol.
If a static library depends on another library, but the other library again depends on the former library, there is a cycle. You can resolve this by enclosing the cyclically dependent libraries by -( and -), such as -( -la -lb -) (you may need to escape the parens, such as -\( and -\)). The linker then searches those enclosed lib multiple times to ensure cycling dependencies are resolved. Alternatively, you can specify the libraries multiple times, so each is before one another: -la -lb -la.
Linking to dynamic libraries
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -L. -ld -o libb.so # specifies its dependency!
$ g++ -L. -lb a.cpp # wrong order (works on some distributions)
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong order
$ g++ -Wl,--as-needed a.cpp -L. -lb # right order
It's the same here - the libraries must follow the object files of the program. The difference here compared with static libraries is that you need not care about the dependencies of the libraries against each other, because dynamic libraries sort out their dependencies themselves.
Some recent distributions apparently default to using the --as-needed linker flag, which enforces that the program's object files come before the dynamic libraries. If that flag is passed, the linker will not link to libraries that are not actually needed by the executable (and it detects this from left to right). My recent archlinux distribution doesn't use this flag by default, so it didn't give an error for not following the correct order.
It is not correct to omit the dependency of b.so against d.so when creating the former. You will be required to specify the library when linking a then, but a doesn't really need the integer b itself, so it should not be made to care about b's own dependencies.
Here is an example of the implications if you miss specifying the dependencies for libb.so
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -o libb.so # wrong (but links)
$ g++ -L. -lb a.cpp # wrong, as above
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong, as above
$ g++ a.cpp -L. -lb # wrong, missing libd.so
$ g++ a.cpp -L. -ld -lb # wrong order (works on some distributions)
$ g++ -Wl,--as-needed a.cpp -L. -ld -lb # wrong order (like static libs)
$ g++ -Wl,--as-needed a.cpp -L. -lb -ld # "right"
If you now look into what dependencies the binary has, you note the binary itself depends also on libd, not just libb as it should. The binary will need to be relinked if libb later depends on another library, if you do it this way. And if someone else loads libb using dlopen at runtime (think of loading plugins dynamically), the call will fail as well. So the "right" really should be a wrong as well.
The GNU ld linker is a so-called smart linker. It will keep track of the functions used by preceding static libraries, permanently tossing out those functions that are not used from its lookup tables. The result is that if you link a static library too early, then the functions in that library are no longer available to static libraries later on the link line.
The typical UNIX linker works from left to right, so put all your dependent libraries on the left, and the ones that satisfy those dependencies on the right of the link line. You may find that some libraries depend on others while at the same time other libraries depend on them. This is where it gets complicated. When it comes to circular references, fix your code!
Here's an example to make it clear how things work with GCC when static libraries are involved. So let's assume we have the following scenario:
myprog.o - containing main() function, dependent on libmysqlclient
libmysqlclient - static, for the sake of the example (you'd prefer the shared library, of course, as the libmysqlclient is huge); in /usr/local/lib; and dependent on stuff from libz
libz (dynamic)
How do we link this? (Note: examples from compiling on Cygwin using gcc 4.3.4)
gcc -L/usr/local/lib -lmysqlclient myprog.o
# undefined reference to `_mysql_init'
# myprog depends on libmysqlclient
# so myprog has to come earlier on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# we have to link with libz, too
gcc myprog.o -lz -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# libz is needed by libmysqlclient
# so it has to appear *after* it on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient -lz
# this works
If you add -Wl,--start-group to the linker flags it does not care which order they're in or if there are circular dependencies.
On Qt this means adding:
QMAKE_LFLAGS += -Wl,--start-group
Saves loads of time messing about and it doesn't seem to slow down linking much (which takes far less time than compilation anyway).
Another alternative would be to specify the list of libraries twice:
gcc prog.o libA.a libB.a libA.a libB.a -o prog.x
Doing this, you don't have to bother with the right sequence since the reference will be resolved in the second block.
A quick tip that tripped me up: if you're invoking the linker as "gcc" or "g++", then using "--start-group" and "--end-group" won't pass those options through to the linker -- nor will it flag an error. It will just fail the link with undefined symbols if you had the library order wrong.
You need to write them as "-Wl,--start-group" etc. to tell GCC to pass the argument through to the linker.
You may can use -Xlinker option.
g++ -o foobar -Xlinker -start-group -Xlinker libA.a -Xlinker libB.a -Xlinker libC.a -Xlinker -end-group
is ALMOST equal to
g++ -o foobar -Xlinker -start-group -Xlinker libC.a -Xlinker libB.a -Xlinker libA.a -Xlinker -end-group
Careful !
The order within a group is important !
Here's an example: a debug library has a debug routine, but the non-debug
library has a weak version of the same. You must put the debug library
FIRST in the group or you will resolve to the non-debug version.
You need to precede each library in the group list with -Xlinker
Link order certainly does matter, at least on some platforms. I have seen crashes for applications linked with libraries in wrong order (where wrong means A linked before B but B depends on A).
I have seen this a lot, some of our modules link in excess of a 100 libraries of our code plus system & 3rd party libs.
Depending on different linkers HP/Intel/GCC/SUN/SGI/IBM/etc you can get unresolved functions/variables etc, on some platforms you have to list libraries twice.
For the most part we use structured hierarchy of libraries, core, platform, different layers of abstraction, but for some systems you still have to play with the order in the link command.
Once you hit upon a solution document it so the next developer does not have to work it out again.
My old lecturer used to say, "high cohesion & low coupling", it’s still true today.
Related
Why does the order in which libraries are linked sometimes cause errors in GCC?
(See the history on this answer to get the more elaborate text, but I now think it's easier for the reader to see real command lines).
Common files shared by all below commands
// a depends on b, b depends on d
$ cat a.cpp
extern int a;
int main() {
return a;
}
$ cat b.cpp
extern int b;
int a = b;
$ cat d.cpp
int b;
Linking to static libraries
$ g++ -c b.cpp -o b.o
$ ar cr libb.a b.o
$ g++ -c d.cpp -o d.o
$ ar cr libd.a d.o
$ g++ -L. -ld -lb a.cpp # wrong order
$ g++ -L. -lb -ld a.cpp # wrong order
$ g++ a.cpp -L. -ld -lb # wrong order
$ g++ a.cpp -L. -lb -ld # right order
The linker searches from left to right, and notes unresolved symbols as it goes. If a library resolves the symbol, it takes the object files of that library to resolve the symbol (b.o out of libb.a in this case).
Dependencies of static libraries against each other work the same - the library that needs symbols must be first, then the library that resolves the symbol.
If a static library depends on another library, but the other library again depends on the former library, there is a cycle. You can resolve this by enclosing the cyclically dependent libraries by -( and -), such as -( -la -lb -) (you may need to escape the parens, such as -\( and -\)). The linker then searches those enclosed lib multiple times to ensure cycling dependencies are resolved. Alternatively, you can specify the libraries multiple times, so each is before one another: -la -lb -la.
Linking to dynamic libraries
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -L. -ld -o libb.so # specifies its dependency!
$ g++ -L. -lb a.cpp # wrong order (works on some distributions)
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong order
$ g++ -Wl,--as-needed a.cpp -L. -lb # right order
It's the same here - the libraries must follow the object files of the program. The difference here compared with static libraries is that you need not care about the dependencies of the libraries against each other, because dynamic libraries sort out their dependencies themselves.
Some recent distributions apparently default to using the --as-needed linker flag, which enforces that the program's object files come before the dynamic libraries. If that flag is passed, the linker will not link to libraries that are not actually needed by the executable (and it detects this from left to right). My recent archlinux distribution doesn't use this flag by default, so it didn't give an error for not following the correct order.
It is not correct to omit the dependency of b.so against d.so when creating the former. You will be required to specify the library when linking a then, but a doesn't really need the integer b itself, so it should not be made to care about b's own dependencies.
Here is an example of the implications if you miss specifying the dependencies for libb.so
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -o libb.so # wrong (but links)
$ g++ -L. -lb a.cpp # wrong, as above
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong, as above
$ g++ a.cpp -L. -lb # wrong, missing libd.so
$ g++ a.cpp -L. -ld -lb # wrong order (works on some distributions)
$ g++ -Wl,--as-needed a.cpp -L. -ld -lb # wrong order (like static libs)
$ g++ -Wl,--as-needed a.cpp -L. -lb -ld # "right"
If you now look into what dependencies the binary has, you note the binary itself depends also on libd, not just libb as it should. The binary will need to be relinked if libb later depends on another library, if you do it this way. And if someone else loads libb using dlopen at runtime (think of loading plugins dynamically), the call will fail as well. So the "right" really should be a wrong as well.
The GNU ld linker is a so-called smart linker. It will keep track of the functions used by preceding static libraries, permanently tossing out those functions that are not used from its lookup tables. The result is that if you link a static library too early, then the functions in that library are no longer available to static libraries later on the link line.
The typical UNIX linker works from left to right, so put all your dependent libraries on the left, and the ones that satisfy those dependencies on the right of the link line. You may find that some libraries depend on others while at the same time other libraries depend on them. This is where it gets complicated. When it comes to circular references, fix your code!
Here's an example to make it clear how things work with GCC when static libraries are involved. So let's assume we have the following scenario:
myprog.o - containing main() function, dependent on libmysqlclient
libmysqlclient - static, for the sake of the example (you'd prefer the shared library, of course, as the libmysqlclient is huge); in /usr/local/lib; and dependent on stuff from libz
libz (dynamic)
How do we link this? (Note: examples from compiling on Cygwin using gcc 4.3.4)
gcc -L/usr/local/lib -lmysqlclient myprog.o
# undefined reference to `_mysql_init'
# myprog depends on libmysqlclient
# so myprog has to come earlier on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# we have to link with libz, too
gcc myprog.o -lz -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# libz is needed by libmysqlclient
# so it has to appear *after* it on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient -lz
# this works
If you add -Wl,--start-group to the linker flags it does not care which order they're in or if there are circular dependencies.
On Qt this means adding:
QMAKE_LFLAGS += -Wl,--start-group
Saves loads of time messing about and it doesn't seem to slow down linking much (which takes far less time than compilation anyway).
Another alternative would be to specify the list of libraries twice:
gcc prog.o libA.a libB.a libA.a libB.a -o prog.x
Doing this, you don't have to bother with the right sequence since the reference will be resolved in the second block.
A quick tip that tripped me up: if you're invoking the linker as "gcc" or "g++", then using "--start-group" and "--end-group" won't pass those options through to the linker -- nor will it flag an error. It will just fail the link with undefined symbols if you had the library order wrong.
You need to write them as "-Wl,--start-group" etc. to tell GCC to pass the argument through to the linker.
You may can use -Xlinker option.
g++ -o foobar -Xlinker -start-group -Xlinker libA.a -Xlinker libB.a -Xlinker libC.a -Xlinker -end-group
is ALMOST equal to
g++ -o foobar -Xlinker -start-group -Xlinker libC.a -Xlinker libB.a -Xlinker libA.a -Xlinker -end-group
Careful !
The order within a group is important !
Here's an example: a debug library has a debug routine, but the non-debug
library has a weak version of the same. You must put the debug library
FIRST in the group or you will resolve to the non-debug version.
You need to precede each library in the group list with -Xlinker
Link order certainly does matter, at least on some platforms. I have seen crashes for applications linked with libraries in wrong order (where wrong means A linked before B but B depends on A).
I have seen this a lot, some of our modules link in excess of a 100 libraries of our code plus system & 3rd party libs.
Depending on different linkers HP/Intel/GCC/SUN/SGI/IBM/etc you can get unresolved functions/variables etc, on some platforms you have to list libraries twice.
For the most part we use structured hierarchy of libraries, core, platform, different layers of abstraction, but for some systems you still have to play with the order in the link command.
Once you hit upon a solution document it so the next developer does not have to work it out again.
My old lecturer used to say, "high cohesion & low coupling", it’s still true today.
Why does the order in which libraries are linked sometimes cause errors in GCC?
(See the history on this answer to get the more elaborate text, but I now think it's easier for the reader to see real command lines).
Common files shared by all below commands
// a depends on b, b depends on d
$ cat a.cpp
extern int a;
int main() {
return a;
}
$ cat b.cpp
extern int b;
int a = b;
$ cat d.cpp
int b;
Linking to static libraries
$ g++ -c b.cpp -o b.o
$ ar cr libb.a b.o
$ g++ -c d.cpp -o d.o
$ ar cr libd.a d.o
$ g++ -L. -ld -lb a.cpp # wrong order
$ g++ -L. -lb -ld a.cpp # wrong order
$ g++ a.cpp -L. -ld -lb # wrong order
$ g++ a.cpp -L. -lb -ld # right order
The linker searches from left to right, and notes unresolved symbols as it goes. If a library resolves the symbol, it takes the object files of that library to resolve the symbol (b.o out of libb.a in this case).
Dependencies of static libraries against each other work the same - the library that needs symbols must be first, then the library that resolves the symbol.
If a static library depends on another library, but the other library again depends on the former library, there is a cycle. You can resolve this by enclosing the cyclically dependent libraries by -( and -), such as -( -la -lb -) (you may need to escape the parens, such as -\( and -\)). The linker then searches those enclosed lib multiple times to ensure cycling dependencies are resolved. Alternatively, you can specify the libraries multiple times, so each is before one another: -la -lb -la.
Linking to dynamic libraries
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -L. -ld -o libb.so # specifies its dependency!
$ g++ -L. -lb a.cpp # wrong order (works on some distributions)
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong order
$ g++ -Wl,--as-needed a.cpp -L. -lb # right order
It's the same here - the libraries must follow the object files of the program. The difference here compared with static libraries is that you need not care about the dependencies of the libraries against each other, because dynamic libraries sort out their dependencies themselves.
Some recent distributions apparently default to using the --as-needed linker flag, which enforces that the program's object files come before the dynamic libraries. If that flag is passed, the linker will not link to libraries that are not actually needed by the executable (and it detects this from left to right). My recent archlinux distribution doesn't use this flag by default, so it didn't give an error for not following the correct order.
It is not correct to omit the dependency of b.so against d.so when creating the former. You will be required to specify the library when linking a then, but a doesn't really need the integer b itself, so it should not be made to care about b's own dependencies.
Here is an example of the implications if you miss specifying the dependencies for libb.so
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -o libb.so # wrong (but links)
$ g++ -L. -lb a.cpp # wrong, as above
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong, as above
$ g++ a.cpp -L. -lb # wrong, missing libd.so
$ g++ a.cpp -L. -ld -lb # wrong order (works on some distributions)
$ g++ -Wl,--as-needed a.cpp -L. -ld -lb # wrong order (like static libs)
$ g++ -Wl,--as-needed a.cpp -L. -lb -ld # "right"
If you now look into what dependencies the binary has, you note the binary itself depends also on libd, not just libb as it should. The binary will need to be relinked if libb later depends on another library, if you do it this way. And if someone else loads libb using dlopen at runtime (think of loading plugins dynamically), the call will fail as well. So the "right" really should be a wrong as well.
The GNU ld linker is a so-called smart linker. It will keep track of the functions used by preceding static libraries, permanently tossing out those functions that are not used from its lookup tables. The result is that if you link a static library too early, then the functions in that library are no longer available to static libraries later on the link line.
The typical UNIX linker works from left to right, so put all your dependent libraries on the left, and the ones that satisfy those dependencies on the right of the link line. You may find that some libraries depend on others while at the same time other libraries depend on them. This is where it gets complicated. When it comes to circular references, fix your code!
Here's an example to make it clear how things work with GCC when static libraries are involved. So let's assume we have the following scenario:
myprog.o - containing main() function, dependent on libmysqlclient
libmysqlclient - static, for the sake of the example (you'd prefer the shared library, of course, as the libmysqlclient is huge); in /usr/local/lib; and dependent on stuff from libz
libz (dynamic)
How do we link this? (Note: examples from compiling on Cygwin using gcc 4.3.4)
gcc -L/usr/local/lib -lmysqlclient myprog.o
# undefined reference to `_mysql_init'
# myprog depends on libmysqlclient
# so myprog has to come earlier on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# we have to link with libz, too
gcc myprog.o -lz -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# libz is needed by libmysqlclient
# so it has to appear *after* it on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient -lz
# this works
If you add -Wl,--start-group to the linker flags it does not care which order they're in or if there are circular dependencies.
On Qt this means adding:
QMAKE_LFLAGS += -Wl,--start-group
Saves loads of time messing about and it doesn't seem to slow down linking much (which takes far less time than compilation anyway).
Another alternative would be to specify the list of libraries twice:
gcc prog.o libA.a libB.a libA.a libB.a -o prog.x
Doing this, you don't have to bother with the right sequence since the reference will be resolved in the second block.
A quick tip that tripped me up: if you're invoking the linker as "gcc" or "g++", then using "--start-group" and "--end-group" won't pass those options through to the linker -- nor will it flag an error. It will just fail the link with undefined symbols if you had the library order wrong.
You need to write them as "-Wl,--start-group" etc. to tell GCC to pass the argument through to the linker.
You may can use -Xlinker option.
g++ -o foobar -Xlinker -start-group -Xlinker libA.a -Xlinker libB.a -Xlinker libC.a -Xlinker -end-group
is ALMOST equal to
g++ -o foobar -Xlinker -start-group -Xlinker libC.a -Xlinker libB.a -Xlinker libA.a -Xlinker -end-group
Careful !
The order within a group is important !
Here's an example: a debug library has a debug routine, but the non-debug
library has a weak version of the same. You must put the debug library
FIRST in the group or you will resolve to the non-debug version.
You need to precede each library in the group list with -Xlinker
Link order certainly does matter, at least on some platforms. I have seen crashes for applications linked with libraries in wrong order (where wrong means A linked before B but B depends on A).
I have seen this a lot, some of our modules link in excess of a 100 libraries of our code plus system & 3rd party libs.
Depending on different linkers HP/Intel/GCC/SUN/SGI/IBM/etc you can get unresolved functions/variables etc, on some platforms you have to list libraries twice.
For the most part we use structured hierarchy of libraries, core, platform, different layers of abstraction, but for some systems you still have to play with the order in the link command.
Once you hit upon a solution document it so the next developer does not have to work it out again.
My old lecturer used to say, "high cohesion & low coupling", it’s still true today.
I am trying to understand static libraries and shared objects in C. I am trying to understand whether one type of library can depend on other type.
Consider a scenario:
libA.so has a function foo_A_dyn():
libA.so ---> foo_A_dyn()
foo_A_dyn() uses a function foo_B_static() which is defined in libB.a which is a static library.
libB.a ---> foo_B_static()
I have built my libraries in the following way:
gcc -c foo_B.c -o foo_B.o
ar -cvq libB.a foo_B.o
gcc -fPIC -c foo_A.c -o foo_A.o
gcc -shared libA.so foo_A.o -I.
gcc main.c -lA -lB -L. -I. -o EXE
Note: main.c makes call to foo_A_dyn() and does NOT call foo_B_static() directly.
And now when I am trying to build my executable EXE, I am getting the error "undefined reference to foo_B_static".
I think the error seems genuine but I am not able to decode the rationale behind this and put it to words.
Can someone please help?
From gcc link options:
-llibrary
-l library
...
It makes a difference where in the command you write this option; the linker searches and processes libraries and object files in the order they are specified. Thus, ‘foo.o -lz bar.o’ searches library ‘z’ after file foo.o but before bar.o. If bar.o refers to functions in ‘z’, those functions may not be loaded.
Try:
gcc main.c -lB -lA -L. -I. -o EXE
Here's what the linker is doing. When we link our executable ('EXE' above) it has some symbols (functions and other things) that are unresolved. It will look down the list of libraries that follow in sequential order, trying to resolve unresolved symbols. Along the way, it finds that some of the symbols are provided by libB.so, so it notes that they are now resolved by this library. While going through libB.so it finds some symbols which are unresolved and it tries to resolve them by looking up the library that follows.
When we are ordering the libraries like:
gcc main.c -lA -lB -L. -I. -o EXE
Linker is not able to lookup for the definition of symbols used in libB into libA. Reason could be that backward reference is not available.
I have also figured out that:
shared object can depend on a static archive,
a static archive can depend on a shared object, and
one static archive can depend on another static archive
Please let me know if I have erred somewhere.
Why does the order in which libraries are linked sometimes cause errors in GCC?
(See the history on this answer to get the more elaborate text, but I now think it's easier for the reader to see real command lines).
Common files shared by all below commands
// a depends on b, b depends on d
$ cat a.cpp
extern int a;
int main() {
return a;
}
$ cat b.cpp
extern int b;
int a = b;
$ cat d.cpp
int b;
Linking to static libraries
$ g++ -c b.cpp -o b.o
$ ar cr libb.a b.o
$ g++ -c d.cpp -o d.o
$ ar cr libd.a d.o
$ g++ -L. -ld -lb a.cpp # wrong order
$ g++ -L. -lb -ld a.cpp # wrong order
$ g++ a.cpp -L. -ld -lb # wrong order
$ g++ a.cpp -L. -lb -ld # right order
The linker searches from left to right, and notes unresolved symbols as it goes. If a library resolves the symbol, it takes the object files of that library to resolve the symbol (b.o out of libb.a in this case).
Dependencies of static libraries against each other work the same - the library that needs symbols must be first, then the library that resolves the symbol.
If a static library depends on another library, but the other library again depends on the former library, there is a cycle. You can resolve this by enclosing the cyclically dependent libraries by -( and -), such as -( -la -lb -) (you may need to escape the parens, such as -\( and -\)). The linker then searches those enclosed lib multiple times to ensure cycling dependencies are resolved. Alternatively, you can specify the libraries multiple times, so each is before one another: -la -lb -la.
Linking to dynamic libraries
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -L. -ld -o libb.so # specifies its dependency!
$ g++ -L. -lb a.cpp # wrong order (works on some distributions)
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong order
$ g++ -Wl,--as-needed a.cpp -L. -lb # right order
It's the same here - the libraries must follow the object files of the program. The difference here compared with static libraries is that you need not care about the dependencies of the libraries against each other, because dynamic libraries sort out their dependencies themselves.
Some recent distributions apparently default to using the --as-needed linker flag, which enforces that the program's object files come before the dynamic libraries. If that flag is passed, the linker will not link to libraries that are not actually needed by the executable (and it detects this from left to right). My recent archlinux distribution doesn't use this flag by default, so it didn't give an error for not following the correct order.
It is not correct to omit the dependency of b.so against d.so when creating the former. You will be required to specify the library when linking a then, but a doesn't really need the integer b itself, so it should not be made to care about b's own dependencies.
Here is an example of the implications if you miss specifying the dependencies for libb.so
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -o libb.so # wrong (but links)
$ g++ -L. -lb a.cpp # wrong, as above
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong, as above
$ g++ a.cpp -L. -lb # wrong, missing libd.so
$ g++ a.cpp -L. -ld -lb # wrong order (works on some distributions)
$ g++ -Wl,--as-needed a.cpp -L. -ld -lb # wrong order (like static libs)
$ g++ -Wl,--as-needed a.cpp -L. -lb -ld # "right"
If you now look into what dependencies the binary has, you note the binary itself depends also on libd, not just libb as it should. The binary will need to be relinked if libb later depends on another library, if you do it this way. And if someone else loads libb using dlopen at runtime (think of loading plugins dynamically), the call will fail as well. So the "right" really should be a wrong as well.
The GNU ld linker is a so-called smart linker. It will keep track of the functions used by preceding static libraries, permanently tossing out those functions that are not used from its lookup tables. The result is that if you link a static library too early, then the functions in that library are no longer available to static libraries later on the link line.
The typical UNIX linker works from left to right, so put all your dependent libraries on the left, and the ones that satisfy those dependencies on the right of the link line. You may find that some libraries depend on others while at the same time other libraries depend on them. This is where it gets complicated. When it comes to circular references, fix your code!
Here's an example to make it clear how things work with GCC when static libraries are involved. So let's assume we have the following scenario:
myprog.o - containing main() function, dependent on libmysqlclient
libmysqlclient - static, for the sake of the example (you'd prefer the shared library, of course, as the libmysqlclient is huge); in /usr/local/lib; and dependent on stuff from libz
libz (dynamic)
How do we link this? (Note: examples from compiling on Cygwin using gcc 4.3.4)
gcc -L/usr/local/lib -lmysqlclient myprog.o
# undefined reference to `_mysql_init'
# myprog depends on libmysqlclient
# so myprog has to come earlier on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# we have to link with libz, too
gcc myprog.o -lz -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# libz is needed by libmysqlclient
# so it has to appear *after* it on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient -lz
# this works
If you add -Wl,--start-group to the linker flags it does not care which order they're in or if there are circular dependencies.
On Qt this means adding:
QMAKE_LFLAGS += -Wl,--start-group
Saves loads of time messing about and it doesn't seem to slow down linking much (which takes far less time than compilation anyway).
Another alternative would be to specify the list of libraries twice:
gcc prog.o libA.a libB.a libA.a libB.a -o prog.x
Doing this, you don't have to bother with the right sequence since the reference will be resolved in the second block.
A quick tip that tripped me up: if you're invoking the linker as "gcc" or "g++", then using "--start-group" and "--end-group" won't pass those options through to the linker -- nor will it flag an error. It will just fail the link with undefined symbols if you had the library order wrong.
You need to write them as "-Wl,--start-group" etc. to tell GCC to pass the argument through to the linker.
You may can use -Xlinker option.
g++ -o foobar -Xlinker -start-group -Xlinker libA.a -Xlinker libB.a -Xlinker libC.a -Xlinker -end-group
is ALMOST equal to
g++ -o foobar -Xlinker -start-group -Xlinker libC.a -Xlinker libB.a -Xlinker libA.a -Xlinker -end-group
Careful !
The order within a group is important !
Here's an example: a debug library has a debug routine, but the non-debug
library has a weak version of the same. You must put the debug library
FIRST in the group or you will resolve to the non-debug version.
You need to precede each library in the group list with -Xlinker
Link order certainly does matter, at least on some platforms. I have seen crashes for applications linked with libraries in wrong order (where wrong means A linked before B but B depends on A).
I have seen this a lot, some of our modules link in excess of a 100 libraries of our code plus system & 3rd party libs.
Depending on different linkers HP/Intel/GCC/SUN/SGI/IBM/etc you can get unresolved functions/variables etc, on some platforms you have to list libraries twice.
For the most part we use structured hierarchy of libraries, core, platform, different layers of abstraction, but for some systems you still have to play with the order in the link command.
Once you hit upon a solution document it so the next developer does not have to work it out again.
My old lecturer used to say, "high cohesion & low coupling", it’s still true today.
a.so defines function A(), b.so defines function B() and calls A() that is defined in a.so. b.so is linked to a.so as below
gcc -fPIC -shared B.c -o libb.so -la
Now I created a binary which is calling B() that is defined in b.so
gcc mybin.c -o mybin -lb
gcc is checking for every symbol in b.so and throws error saying A() is not defined.
gcc mybin.c -o mybin -lb -la
The above works but I had to link mybin with a.so even though it is not directly related to a.so. My requirement is that if b.so is properly linked with a.so, linking mybin with just b.so should work.
Is this possible ??
When linking the executable the linker wants to know that the libraries needed by libb.so exist so it can check they resolve any undefined references in libb.so
The linker looks for the required libraries in the usual places, so if that wouldn't find liba.so then you can tell the linker where to look by using LD_LIBRARY_PATH as shown by another answer, or by using the linker option that exists specifically for this purpose, -rpath-link
gcc mybin.c -o mybin -lb -Wl,-rpath-link=.
(The -Wl, prefix is how you tell GCC to pass an option to the linker, which is needed for linker options that GCC doesn't know about directly.)
Alternatively, you can tell the linker to allow undefined symbols in libb.so in which case it doesn't try to find liba.so to resolve the reference to A(), and trusts that you'll link correctly and ensure the library is available at run-time. That is done with the --allow-shlib-undefined option:
gcc mybin.c -o mybin -lb -Wl,--allow-shlib-undefined
However, it's generally better to let the linker check that all symbols are defined, and tell it how to find the required libraries, as it finds real problems sooner.
The -rpath-link option only tells the linker where to look for additional shared libraries, it has no visible effect on the linked executable (i.e. it doesn't record the path to the library in the executable anywhere). Another alternative is to create libb.so so that it contains the knowledge of how to find liba.so embedded within it. That is done with the -rpath linker option, e.g. if liba.so is in /some/path you could do:
gcc -fPIC -shared B.c -o libb.so -la -Wl,-rpath=/some/path
This puts a DT_RPATH tag in libb.so and the linker will use that to find its dependencies:
readelf -d libb.so | fgrep RPATH
0x000000000000000f (RPATH) Library rpath: [/some/path]
Now you can link the executable without needing any knowledge of liba.so, the linker will see that libb.so needs liba.so and will use the RPATH to find it:
gcc mybin.c -o mybin -lb
This works fine if the path /some/path/liba.so is fixed, but it might not be true that the library is found in the same location during development and after your executable is deployed. In that case you can still use -rpath-link for the executable to tell the linker where to look for it during linking, and rely on the RPATH to find it at run-time. Or you can use the special string $ORIGIN in the RPATH which the dynamic linker expands to the location of the object containing the RPATH, e.g. if liba.so and libb.so are always in the same directory, you can link libb.so like this (note the quotes to prevent the shell expanding $ORIGIN):
gcc -fPIC -shared B.c -o libb.so -la '-Wl,-rpath=$ORIGIN'
readelf -d libb.so | fgrep RPATH
0x000000000000000f (RPATH) Library rpath: [$ORIGIN]
The missing bit was the LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=.
gcc -fPIC -shared a.c -o liba.so
gcc -fPIC -shared b.c -o libb.so -la
gcc mybin.c -o mybin -lb -L.
The reason -L does not work seems it that this flag it used to « Add directory dir to the list of directories to be searched for -l » , see the GCC doc. And libba.so is not specified in a -l.
Here are the steps:
gcc -fPIC -shared a.c -o liba.so
gcc -fPIC -shared b.c -o libb.so
gcc mybin.c -o mybin -lb -la -L. # -L to look for the directory to load .so files
Then make sure you do:
export LD_LIBRARY_PATH=<your_directory>
You need to have liba.so and libb.so mentioned in gcc mybin.c -o mybin -lb -la -L. as the function A() needs to be loaded at runtime. b.so IS NOT LINKED with a.so. Linking happens that runtime in shared library. They are compiled and any reference to functions or extern variables are not resolved.
If you want to include b.so but not a.so then compile the b as static library thereby it loads a into its binary. And then when you compile mybin you will only need b.lib