I am compiling files a.c & b.c with flag -fstack-protector-strong which results to introduce new symbol __stack_chk_fail in object files a.o, b.o
nm a.o | grep __stack_chk_fail
U __stack_chk_fail
nm b.o | grep __stack_chk_fail
U __stack_chk_fail
I am creating one static library libstat.a using above object files :
ar rc libstat.a a.o b.o
Finally, I am trying to create dynamic library libtest.so using above static library. The above symbol __stack_chk_fail is defined in library libssp.so which is in gcc tool chain path /home/test_usr/gcc-10.3.0/aarch64-unknown-linux-gnu/lib64
objdump -T /home/test_usr/gcc-10.3.0/aarch64-unknown-linux-gnu/lib64/libssp.so.0.0.0 | grep __stack_chk_fail
0000000000000ff0 g DF .text 0000000000000020 LIBSSP_1.0 __stack_chk_fail
Build command:-
bin/gcc -include ... -L /home/test_usr/gcc-10.3.0/aarch64-unknown-linux-gnu/lib64/ -lssp -o libtest.so libstat.a
i was expecting to have the symbol defined in my final library libtest.so, But no luck.
nm libtest.so | grep __stack_chk_fail
U __stack_chk_fail#LIBSSP_1.0
Am i missing some thing here? Why this symbols is not getting defined though i try to link with -lssp which has this symbol definition. Please help!
No, this is not how shared libraries work. If the symbol is defined in some .so it does not "migrate" to every library or executable that gets linked with it. You are thinking of static libraries, not dynamic libraries.
A static library, a .a file, consists of multiple object modules with exported symbols. Any symbol that's referenced by whatever you link with the shared library results in its module being copied out of a static library and into an executable or another library that gets linked with it.
A shared library (or a dynamic library as you call them), a .so file, does not work that way.
The library gets loaded at run time, and at runtime all unresolved symbols in your executable or library get resolved with the symbols exported from the shared library, so nothing needs to get copied out of the shared library.
A static library does not get loaded at runtime, and does not need to be installed, because everything that's needed gets extracted from the static library, by your linker, and included in the executable or library that gets linekd with the static library.
The whole reason to use shared libraries, .sos, is to not bloat everything that gets linked with the library with the code from the linked library. The code, and all exported symbols, remain in the .so file, they do not get "defined" or added into whatever gets linked with it.
Related
I am trying to understand more about linking and shared library.
Ultimately, I wonder if it's possible to add a method to a shared library. For instance, suppose one has a source file a.c, and a library lib.so (without the source file). Let's furthermore assume, for simplicity, that a.c declares a single method, whose name is not present in lib.so. I thought maybe it might be possible to, at linking time, link a.o to lib.so while instructing to create newLib.so, and forcing the linker to export all methods/variable in lib.so to that the newLib.so is now basically lib.so with the added method from a.so.
More generally, if one has some source file depending on a shared library, can one create a single output file (library or executable) that is not dependent on the shared library anymore ? (That is, all the relevant methods/variable from the library would have been exported/linked/inlined to the new executable, hence making the dependency void). If that's not possible, what is technically preventing it ?
A somehow similar question has been asked here: Merge multiple .so shared libraries.
One of the reply includes the following text: "If you have access to either source or object files for both libraries, it is straightforward to compile/link a combined SO from them.: without explaining the technical details. Was it a mistake or does it hold ? If so, how to do it ?
Once you have a shared library libfoo.so the only ways you can use it
in the linkage of anything else are:-
Link a program that dynamically depends on it, e.g.
$ gcc -o prog bar.o ... -lfoo
Or, link another shared library that dynamically depends on it, e.g.
$ gcc -shared -o libbar.so bar.o ... -lfoo
In either case the product of the linkage, prog or libbar.so
acquires a dynamic dependency on libfoo.so. This means that prog|libfoo.so
has information inscribed in it by the linker that instructs the
OS loader, at runtime, to find libfoo.so, load it into the
address space of the current process and bind the program's references to libfoo's exported symbols to
the addresses of their definitions.
So libfoo.so must continue to exist as well as prog|libbar.so.
It is not possible to link libfoo.so with prog|libbar.so in
such a way that libfoo.so is physically merged into prog|libbar.so
and is no longer a runtime dependency.
It doesn't matter whether or not you have the source code of the
other linkage input files - bar.o ... - that depend on libfoo.so. The
only kind of linkage you can do with a shared library is dynamic linkage.
This is in complete contrast with the linkage of a static library
You wonder about the statement in this this answer where it says:
If you have access to either source or object files for both libraries, it is straightforward to compile/link a combined SO from them.
The author is just observing that if I have source files
foo_a.c foo_b.c... bar_a.c bar_b.c
which I compile to the corresponding object files:
foo_a.o foo_b.o... bar_a.o bar_b.o...
or if I simply have those object files. Then as well as - or instead of - linking them into two shared libraries:
$ gcc -shared -o libfoo.so foo_a.o foo_b.o...
$ gcc -shared -o libbar.so bar_a.o bar_b.o...
I could link them into one:
$ gcc -shared -o libfoobar.so foo_a.o foo_b.o... bar_a.o bar_b.o...
which would have no dependency on libfoo.so or libbar.so even if they exist.
And although that could be straightforward it could also be false. If there is
any symbol name that is globally defined in any of foo_a.o foo_b.o... and
also globally defined in any of bar_a.o bar_b.o... then it will not matter
to the linkage of either libfoo.so or libbar.so (and it need not be dynamically
exported by either of them). But the linkage of libfoobar.so will fail for
multiple definition of name.
If we build a shared library libbar.so that depends on libfoo.so and has
itself been linked with libfoo.so:
$ gcc -shared -o libbar.so bar.o ... -lfoo
and we then want to link a program with libbar.so, we can do that in such a way
that we don't need to mention its dependency libfoo.so:
$ gcc -o prog main.o ... -lbar -Wl,-rpath=<path/to/libfoo.so>
See this answer to follow that up. But
this doesn't change the fact that libbar.so has a runtime dependency on libfoo.so.
If that's not possible, what is technically preventing it?
What technically prevents linking a shared library with some program
or shared library targ in a way that physically merges it into targ is that a
shared library (like a program) is not the sort of thing that a linker knows
how to physically merge into its output file.
Input files that the linker can physically merge into targ need to
have structural properties that guide the linker in doing that merging. That is the structure of object files.
They consist of named input sections of object code or data that are tagged with various attributes.
Roughly speaking, the linker cuts up the object files into their sections and distributes them into
output sections of the output file according to their attributes, and makes
binary modifications to the merged result to resolve static symbol references
or enable the OS loader to resolve dynamic ones at runtime.
This is not a reversible process. The linker can't consume a program or
shared library and reconstruct the object files from which it was made to
merge them again into something else.
But that's really beside the point. When input files are physically
merged into targ, that is called static linkage.
When input files are just externally referenced in targ to
make the OS loader map them into a process it has launched for targ,
that is called dynamic linkage. Technical development has given us
a file-format solution to each of these needs: object files for static linkage, shared libraries
for dynamic linkage. Neither can be used for the purpose of the other.
I have an libfoo.a which contains _start and all required symbols for an executable. ld.bfd -o foo libfoo.a works smoothly in my case. However, ld.gold -o foo libfoo.a fails silently generating an executable with no symbols from libfoo.a. Creating an empty a.o and link it with ld.gold -o foo a.o libfoo.a works.
I was wondering is there any way to directly link a static library into an executable using ld.gold without creating a redundant empty object files?
You can specify the entry symbol explicitly with the -e _start option, and the linker will use that to decide that it needs to load the object that defines it.
Unfortunately, gold will not use the implicit start symbol to load an object from the archive library.
With GCC on Linux, is it possible to link a .a into another .a and then only link the resultant .a to my application? Or must my application know of the dependence between one archive and another and link them both?
My understanding is that I must know of the dependencies and link all archives at the end, not in an intermediary step, which seems a little ugly.
This is slightly different than How to merge two "ar" static libraries into one as I'm after a clear description that this is only possible by working around the problem and that linking the two archives together in the naive way is incorrect and will not work, along with the reason as to why.
Yes, your application has to know the dependencies between your different static libraries.
Let's say you have two static libraries a and b.
a has a function void print_a(), and b has a function void print_b() that is calling to print_a(). So, b depends on a.
Their binaries will look like liba.a and libb.a.
Let's say that library b has a reference to a function defined in library a - void print_b(void).
When compiling library b only its symbols are defined in the binary's code section while the others are still undefined:
host$ nm libb.a | grep print
U _print_a <--- Undefined
0000000000000000 T _print_b <--- Defined, in code section
0000000000000068 S _print_b.eh
U _printf
Therefore, when compiling the application that wants to use both of the libraries, linking only to libb.a won't be enough. You'll have to link your application to both libraries. Each library will provide its own symbols addresses in the code section and then your application will be able to link to both.
Something like:
gcc -o main main.c libb.a liba.a
BTW: When compiling library b that uses a, you can but it's not necessary to link to a. The result will be just the same.
Why is this the behavior
When compiling + linking the application that uses static libraries, the symbols in the application source files have to be defined somewhere (with the exception of dynamic linking, but this is done only with dynamic libraries/shared objects. Here we deal with static ones).
Now, remember that a static library is just an archive of objects. When it's created there's no linking phase. Just:
Compiling source code (*.c) to objects (*.o)
Archiving them together in a libXXXX.a file.
It means that if this library (library b in my example) uses some function (void print_a(void)) that is defined in another library (a), this symbol won't be resolved (not as a compilation error, but as the normal behavior). It will be set as Undefined symbol (as we see in the output of nm command) after the library creation, and it will wait to be linked later to its definition. And it's OK because a static library is not executable.
Now returning to application - the linking phase of the application needs to find all the definitions of all the symbols. If you just gave it libb.a as an argument, it wouldn't be able to find the definition to print_a(), because it's not there, it's still undefined. It exists only in liba.a.
Therefore, you must provide both of the libraries.
Let libx.a and liby.a be the modules you want to combine. You can try:-
mkdir tmp # create temporary directory for extracting
cd tmp
ar x ../libx.a # extract libx.a
cp ../liby.a ../libxy.a
ar -q ../libxy.a * # add extracted files to libxy.a
cd ..
rm -rf tmp
libxy.a thus created contains .o files from both .a files
I have shared object A.so which statically links to libssl.a & another shared object B.so which also statically links libssl.a .
A.so & B.so has symbols from libssl.a in GLOBAL scope. I checked this by readelf -s A.so
I have an executable a.out which loads A.so and B.so. When a.out terminated I get a
double free error in one of the symbols from libssl.a in A.so.
Even though libssl.a is statically linked to each shared object, since they are exposed
globally is it possible that the same symbol is shared instead of picking it's local copy.
What is the workaround this ? How to make the symbols local here ?
Please help
This is indeed expected. One instance of libssl.a interposes (likely a subset of) the other, and the results are not pretty. You can use a version script (--version-script to ld, with -Wl, for cc) to control what is exported from A.so and B.so. If something is not exported, it cannot be interposed either.
Alternatively, you could compile libssl.a with visibility flags like -fvisibility=hidden. These flags only affect the dynamic linker and not static linking. You likely needed to compile it yourself anyway because shipped .a files tend to contain position-dependent code, meant for linking into executables. Only some platforms such as 32-bit x86 let you get away with linking such code into shared objects and only at the cost of text relocations.
The dlopen with RTLD_LOCAL as suggested in a comment should also work but it seems hackish to use dlopen for this purpose.
Another option is to use the same shared libssl.so in both libraries.
I have created a .c file which is being converted to a .o file along with around 300 other .c files and included in a .a static library. This library, along with many others is being used to create a .so dynamic library. On analyzing both the .a and the .so file with nm, I found that for some reason the symbols defined in the .c file are present in the .a file but not in the .so file. I can think of no reason this should happen. Can somebody please help me out here? The steps used to create the two binaries are:
gcc -fvisibility=hidden -c foo.c -o foo.c.o
ar cr libbar.a foo.c.o ...
gcc -fvisibility=hidden -fPIC -o libfinal.so libbar.a x.o y.a ...
The reason I have specified visibility hidden here is that I want to expose only a few selected symbols. To expose the symbols from foo.c I have specified the visibility attribute so that the functions signatures in the header foo.h look like:
extern int _____attribute_____ ((visibility ("default"))) func();
EDIT: The command nm libbar.a | grep Ctx gives:
000023c5 T CtxAcquireBitmap
000026e9 T CtxAcquireArray
00001e77 T CtxCallMethod
However, nm libfinal.so | grep Ctx does not show anything.
UPDATE: Found another post which discusses the uses of the --whole-archive option. Also, stumbled across the --export-dynamicoption which apparently tells the linker to retain unreferenced symbols. Investigating further.
Try using --whole-archive linker option to include all objects into your shared library when linking
gcc -o libfinal.so -Wl,--whole-archive libbar.a x.o y.a -Wl,--no-whole-archive
From man ld:
--whole-archive
For each archive mentioned on the command line after the --whole-archive option, include every object file in the archive in the
link, rather than searching the archive for the required object files. This is normally used to turn an archive file into a shared
library, forcing every object to be included in the resulting shared library. This option may be used more than once.
Two notes when using this option from gcc: First, gcc doesn't know about this option, so you have to use -Wl,-whole-archive.
Second, don't forget to use -Wl,-no-whole-archive after your list of archives, because gcc will add its own list of archives to your
link and you may not want this flag to affect those as well.
As far as I know, when compiling against a .a, gcc will only pull out the objects that are referenced by the other modules. If your intent is to include the whole content of the .a in the .so, a plain "compile/link x.c into libfinal.so using content in libbar.a" is not what you want.
Creating a dummy reference for the required symbols in my main file did not solve the problem. The referenced symbols appeared in the binary dump (obtained using nm) with a U (= undefined) marker. I managed to solve the problem by linking the object file directly when creating the .so file instead of including it in the .a library first. As these functions were marked extern they were included in the .so even though they were not being referenced within the library. Had they not been marked extern, they would not have been included just like sylvainulg said.
Thanks to Dmitry for pointing out the --whole-archive option. I did not know that such an option exists.