I have created a .c file which is being converted to a .o file along with around 300 other .c files and included in a .a static library. This library, along with many others is being used to create a .so dynamic library. On analyzing both the .a and the .so file with nm, I found that for some reason the symbols defined in the .c file are present in the .a file but not in the .so file. I can think of no reason this should happen. Can somebody please help me out here? The steps used to create the two binaries are:
gcc -fvisibility=hidden -c foo.c -o foo.c.o
ar cr libbar.a foo.c.o ...
gcc -fvisibility=hidden -fPIC -o libfinal.so libbar.a x.o y.a ...
The reason I have specified visibility hidden here is that I want to expose only a few selected symbols. To expose the symbols from foo.c I have specified the visibility attribute so that the functions signatures in the header foo.h look like:
extern int _____attribute_____ ((visibility ("default"))) func();
EDIT: The command nm libbar.a | grep Ctx gives:
000023c5 T CtxAcquireBitmap
000026e9 T CtxAcquireArray
00001e77 T CtxCallMethod
However, nm libfinal.so | grep Ctx does not show anything.
UPDATE: Found another post which discusses the uses of the --whole-archive option. Also, stumbled across the --export-dynamicoption which apparently tells the linker to retain unreferenced symbols. Investigating further.
Try using --whole-archive linker option to include all objects into your shared library when linking
gcc -o libfinal.so -Wl,--whole-archive libbar.a x.o y.a -Wl,--no-whole-archive
From man ld:
--whole-archive
For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive in the
link, rather than searching the archive for the required object files. This is normally used to turn an archive file into a shared
library, forcing every object to be included in the resulting shared library. This option may be used more than once.
Two notes when using this option from gcc: First, gcc doesn't know about this option, so you have to use -Wl,-whole-archive.
Second, don't forget to use -Wl,-no-whole-archive after your list of archives, because gcc will add its own list of archives to your
link and you may not want this flag to affect those as well.
As far as I know, when compiling against a .a, gcc will only pull out the objects that are referenced by the other modules. If your intent is to include the whole content of the .a in the .so, a plain "compile/link x.c into libfinal.so using content in libbar.a" is not what you want.
Creating a dummy reference for the required symbols in my main file did not solve the problem. The referenced symbols appeared in the binary dump (obtained using nm) with a U (= undefined) marker. I managed to solve the problem by linking the object file directly when creating the .so file instead of including it in the .a library first. As these functions were marked extern they were included in the .so even though they were not being referenced within the library. Had they not been marked extern, they would not have been included just like sylvainulg said.
Thanks to Dmitry for pointing out the --whole-archive option. I did not know that such an option exists.
Related
I've been playing around with GCC lately and have been experimenting with the linking options. I'm somewhat confused why the link option -l is necessary when statically linking to an archive file. It seems like you can just toss the .a file as if it were an ordinary object file.
For example, take the following make file:
test1 : main.c libfunc.a
gcc main.c -L. -lfunc -o main.out
test2 : main.c libfunc.a
gcc main.c libfunc.a -o main.out
libfunc.a : func1.c func2.c
gcc func1.c -c
gcc func2.c -c
ar cr libfunc.a func1.o func2.o
Make target test1 uses GCC's linking options to link to the archive file. Target test2 instead just includes the archive file direct. Building and running each output seem to result in the same executable.
There are several ways you can tell gcc what file(s) to use. An argument of the form -lname (or the two arguments -l name) says “Search for a library named name”. Per the GCC documentation, this argument is passed to the linker (typically the ld command). The linker looks for a file with a name like libname.extension, where extension is one of the known library files extensions such as .a or .so, and it looks for files with those names in a list of library directories it has. You can add directories to search with the -L switch.
When the linker finds the library, it uses it just as if you had specified the path, so the end result is the same whether you specify the library with -l or with its path.
By using the path, you can specify libraries that are not in the known library directories or that have unusual names.
Note that the linker does not process libraries the same way as object files. When the linker processes an object file, it incorporates everything in the object file into the output file being constructed. When the linker processes a library file, it incorporates only those modules within the library that provide a symbol definition for a symbol referenced by a prior module and not yet resolved. For example, if you write a program that uses sqrt but does not use sin, then, when the linker processes libm.a after reading your object module, it will take the sqrt module from the library but not the sin module.
I am trying to understand more about linking and shared library.
Ultimately, I wonder if it's possible to add a method to a shared library. For instance, suppose one has a source file a.c, and a library lib.so (without the source file). Let's furthermore assume, for simplicity, that a.c declares a single method, whose name is not present in lib.so. I thought maybe it might be possible to, at linking time, link a.o to lib.so while instructing to create newLib.so, and forcing the linker to export all methods/variable in lib.so to that the newLib.so is now basically lib.so with the added method from a.so.
More generally, if one has some source file depending on a shared library, can one create a single output file (library or executable) that is not dependent on the shared library anymore ? (That is, all the relevant methods/variable from the library would have been exported/linked/inlined to the new executable, hence making the dependency void). If that's not possible, what is technically preventing it ?
A somehow similar question has been asked here: Merge multiple .so shared libraries.
One of the reply includes the following text: "If you have access to either source or object files for both libraries, it is straightforward to compile/link a combined SO from them.: without explaining the technical details. Was it a mistake or does it hold ? If so, how to do it ?
Once you have a shared library libfoo.so the only ways you can use it
in the linkage of anything else are:-
Link a program that dynamically depends on it, e.g.
$ gcc -o prog bar.o ... -lfoo
Or, link another shared library that dynamically depends on it, e.g.
$ gcc -shared -o libbar.so bar.o ... -lfoo
In either case the product of the linkage, prog or libbar.so
acquires a dynamic dependency on libfoo.so. This means that prog|libfoo.so
has information inscribed in it by the linker that instructs the
OS loader, at runtime, to find libfoo.so, load it into the
address space of the current process and bind the program's references to libfoo's exported symbols to
the addresses of their definitions.
So libfoo.so must continue to exist as well as prog|libbar.so.
It is not possible to link libfoo.so with prog|libbar.so in
such a way that libfoo.so is physically merged into prog|libbar.so
and is no longer a runtime dependency.
It doesn't matter whether or not you have the source code of the
other linkage input files - bar.o ... - that depend on libfoo.so. The
only kind of linkage you can do with a shared library is dynamic linkage.
This is in complete contrast with the linkage of a static library
You wonder about the statement in this this answer where it says:
If you have access to either source or object files for both libraries, it is straightforward to compile/link a combined SO from them.
The author is just observing that if I have source files
foo_a.c foo_b.c... bar_a.c bar_b.c
which I compile to the corresponding object files:
foo_a.o foo_b.o... bar_a.o bar_b.o...
or if I simply have those object files. Then as well as - or instead of - linking them into two shared libraries:
$ gcc -shared -o libfoo.so foo_a.o foo_b.o...
$ gcc -shared -o libbar.so bar_a.o bar_b.o...
I could link them into one:
$ gcc -shared -o libfoobar.so foo_a.o foo_b.o... bar_a.o bar_b.o...
which would have no dependency on libfoo.so or libbar.so even if they exist.
And although that could be straightforward it could also be false. If there is
any symbol name that is globally defined in any of foo_a.o foo_b.o... and
also globally defined in any of bar_a.o bar_b.o... then it will not matter
to the linkage of either libfoo.so or libbar.so (and it need not be dynamically
exported by either of them). But the linkage of libfoobar.so will fail for
multiple definition of name.
If we build a shared library libbar.so that depends on libfoo.so and has
itself been linked with libfoo.so:
$ gcc -shared -o libbar.so bar.o ... -lfoo
and we then want to link a program with libbar.so, we can do that in such a way
that we don't need to mention its dependency libfoo.so:
$ gcc -o prog main.o ... -lbar -Wl,-rpath=<path/to/libfoo.so>
See this answer to follow that up. But
this doesn't change the fact that libbar.so has a runtime dependency on libfoo.so.
If that's not possible, what is technically preventing it?
What technically prevents linking a shared library with some program
or shared library targ in a way that physically merges it into targ is that a
shared library (like a program) is not the sort of thing that a linker knows
how to physically merge into its output file.
Input files that the linker can physically merge into targ need to
have structural properties that guide the linker in doing that merging. That is the structure of object files.
They consist of named input sections of object code or data that are tagged with various attributes.
Roughly speaking, the linker cuts up the object files into their sections and distributes them into
output sections of the output file according to their attributes, and makes
binary modifications to the merged result to resolve static symbol references
or enable the OS loader to resolve dynamic ones at runtime.
This is not a reversible process. The linker can't consume a program or
shared library and reconstruct the object files from which it was made to
merge them again into something else.
But that's really beside the point. When input files are physically
merged into targ, that is called static linkage.
When input files are just externally referenced in targ to
make the OS loader map them into a process it has launched for targ,
that is called dynamic linkage. Technical development has given us
a file-format solution to each of these needs: object files for static linkage, shared libraries
for dynamic linkage. Neither can be used for the purpose of the other.
I have one .a file ( ar command ) which I want to bind it with my .so file during GCC compilation.
How can I do this.
If I run this command :
gcc /usr/local/apr/lib/libapr-1.a ../../ndagentlibc/obj/*.o tideways_xhprof.o tracing.o -shared -o libhello.so
nm libhello.so | grep apr_term
output: U apr_terminate
apr_terminate is not getting its defination
If your .so file needs the .a file, link your .so with the .a file and all of the needed code from the .a archive will be available in the .so file.
Reorder your command and put the .a library at the end...
gcc ../../ndagentlibc/obj/*.o tideways_xhprof.o tracing.o -shared -o libhello.so /usr/local/apr/lib/libapr-1.a
as the ld(1) linker only selects the object modules (*.o) included in a library archive that it knows have unsolved references for at the moment of reading it (as you put it first in your command line, no unsolved references appear by that time, so no library .o component is selected and included at the time of library processing)
In the case of an archive of object modules, the linker tries to do its best, selecting only the ones that appear to be necessary, and putting an archive at first makes the linker to select no files from it to be linked.
note
BTW, the -shared option is used to create a shared object (a .so module) which probably is not what you want. If you want to create a final executable program, don't use -shared. I point at this, because the first time I had to fight with this, I assumed the kind of linking (shared or static) was specified with some option (common mistake, I think) but the kind of linking is actually given, object by object, by the kind of file you feed to the linker, and not with a command line option. Apart of other things, it makes the linker not to comply when some references are missing in the program (it assumes those will be resolved in a later linking)
I create some files:
file1.c
file2.c
file3.c
I compile them using gcc -c file1.c and i did the same for other files, and i get object files. Later i used ar tool to create static library.
Everythink works correctly, but ar has option
ar -m -a file.o lib.a filetomove.o
to move object files in library, why order of object files is important? Please, show me example where object files must be in correct order.
This is less and less of a problem as time goes on, but for a long time linkers were single pass. That means if a symbol was defined in a.o and referenced in b.o, the linker had to "see" b.o before a.o or it would never find a definition for the reference.
In other circumstances, sometimes a "default" function is provided in a library that is linked last. This is a popular technique in embedded systems development. You can provide an override function by linking it in a static library or object module, but if you don't, the last library will provide a symbol that satisfies the linker.
I know that have been already asked a lot of time but I can't really solve it...
so I have a src folder where my main.c source is, an srclib where my lib.c file is stored and an include directory where my lib.h file is stored. now the makefile compiles the lib correctly and put the lib.a file in lib folder. the main.cincludes the lib like this:
#include "lib.h"
and it's compiled with -I ../include option, but when i compile it I get the undefined reference to xxx error for every function in the library
so what am I missing?
Nope. -I is for including the header files. You also need to link with the library using -l option.
Note: You may need to provide the path-to-library using -L option.
To quote the online gcc manual
-llibrary
Search the library named library when linking...... The linker searches a standard list of directories for the library, which is actually a file named liblibrary.a. The linker then uses this file as if it had been specified precisely by name.
EDIT:
To quote the remaining part of the same manual
It makes a difference where in the command you write this option; the linker searches and processes libraries and object files in the order they are specified. Thus, foo.o -lz bar.o searches library z after file foo.o but before bar.o. If bar.o refers to functions in z, those functions may not be loaded.
So, you need to put the -l<libanme> at the last of your compilation statement, as s_echo.c uses functions defined in that particular library.