I have multiple objects files (lets say obj1.o, obj2.o) with which i am building a shared library (libapp.so). All associated objects so not had visibility flag as hidden when they were built, but in final so (libapp.so, see below), if i use visibility flag as hidden, will it hide all internal functions from outside libraries to use?
gcc -shared -fPIC -fvisibility=hidden obj1.o obj2.o -o libapp.so
EDIT:
As wRAR mentioned, i tried using versions cript to hide all symbol on final shared library. version script was written like-
file:libapp.version
{
local: *;
};
and i built my shared library as:
gcc -shared -fPIC obj1.o obj2.o -o libapp.so -Wl,--version-script=libapp.version
Even after putting all symbol as local, i still can see in nm that all symbols are present. What i am doing wrong here?
-fvisibility is a compile-time option, so it has no effect at the link time. You can write a version script for ld to hide symbols at the link time.
Related
I am trying to understand more about linking and shared library.
Ultimately, I wonder if it's possible to add a method to a shared library. For instance, suppose one has a source file a.c, and a library lib.so (without the source file). Let's furthermore assume, for simplicity, that a.c declares a single method, whose name is not present in lib.so. I thought maybe it might be possible to, at linking time, link a.o to lib.so while instructing to create newLib.so, and forcing the linker to export all methods/variable in lib.so to that the newLib.so is now basically lib.so with the added method from a.so.
More generally, if one has some source file depending on a shared library, can one create a single output file (library or executable) that is not dependent on the shared library anymore ? (That is, all the relevant methods/variable from the library would have been exported/linked/inlined to the new executable, hence making the dependency void). If that's not possible, what is technically preventing it ?
A somehow similar question has been asked here: Merge multiple .so shared libraries.
One of the reply includes the following text: "If you have access to either source or object files for both libraries, it is straightforward to compile/link a combined SO from them.: without explaining the technical details. Was it a mistake or does it hold ? If so, how to do it ?
Once you have a shared library libfoo.so the only ways you can use it
in the linkage of anything else are:-
Link a program that dynamically depends on it, e.g.
$ gcc -o prog bar.o ... -lfoo
Or, link another shared library that dynamically depends on it, e.g.
$ gcc -shared -o libbar.so bar.o ... -lfoo
In either case the product of the linkage, prog or libbar.so
acquires a dynamic dependency on libfoo.so. This means that prog|libfoo.so
has information inscribed in it by the linker that instructs the
OS loader, at runtime, to find libfoo.so, load it into the
address space of the current process and bind the program's references to libfoo's exported symbols to
the addresses of their definitions.
So libfoo.so must continue to exist as well as prog|libbar.so.
It is not possible to link libfoo.so with prog|libbar.so in
such a way that libfoo.so is physically merged into prog|libbar.so
and is no longer a runtime dependency.
It doesn't matter whether or not you have the source code of the
other linkage input files - bar.o ... - that depend on libfoo.so. The
only kind of linkage you can do with a shared library is dynamic linkage.
This is in complete contrast with the linkage of a static library
You wonder about the statement in this this answer where it says:
If you have access to either source or object files for both libraries, it is straightforward to compile/link a combined SO from them.
The author is just observing that if I have source files
foo_a.c foo_b.c... bar_a.c bar_b.c
which I compile to the corresponding object files:
foo_a.o foo_b.o... bar_a.o bar_b.o...
or if I simply have those object files. Then as well as - or instead of - linking them into two shared libraries:
$ gcc -shared -o libfoo.so foo_a.o foo_b.o...
$ gcc -shared -o libbar.so bar_a.o bar_b.o...
I could link them into one:
$ gcc -shared -o libfoobar.so foo_a.o foo_b.o... bar_a.o bar_b.o...
which would have no dependency on libfoo.so or libbar.so even if they exist.
And although that could be straightforward it could also be false. If there is
any symbol name that is globally defined in any of foo_a.o foo_b.o... and
also globally defined in any of bar_a.o bar_b.o... then it will not matter
to the linkage of either libfoo.so or libbar.so (and it need not be dynamically
exported by either of them). But the linkage of libfoobar.so will fail for
multiple definition of name.
If we build a shared library libbar.so that depends on libfoo.so and has
itself been linked with libfoo.so:
$ gcc -shared -o libbar.so bar.o ... -lfoo
and we then want to link a program with libbar.so, we can do that in such a way
that we don't need to mention its dependency libfoo.so:
$ gcc -o prog main.o ... -lbar -Wl,-rpath=<path/to/libfoo.so>
See this answer to follow that up. But
this doesn't change the fact that libbar.so has a runtime dependency on libfoo.so.
If that's not possible, what is technically preventing it?
What technically prevents linking a shared library with some program
or shared library targ in a way that physically merges it into targ is that a
shared library (like a program) is not the sort of thing that a linker knows
how to physically merge into its output file.
Input files that the linker can physically merge into targ need to
have structural properties that guide the linker in doing that merging. That is the structure of object files.
They consist of named input sections of object code or data that are tagged with various attributes.
Roughly speaking, the linker cuts up the object files into their sections and distributes them into
output sections of the output file according to their attributes, and makes
binary modifications to the merged result to resolve static symbol references
or enable the OS loader to resolve dynamic ones at runtime.
This is not a reversible process. The linker can't consume a program or
shared library and reconstruct the object files from which it was made to
merge them again into something else.
But that's really beside the point. When input files are physically
merged into targ, that is called static linkage.
When input files are just externally referenced in targ to
make the OS loader map them into a process it has launched for targ,
that is called dynamic linkage. Technical development has given us
a file-format solution to each of these needs: object files for static linkage, shared libraries
for dynamic linkage. Neither can be used for the purpose of the other.
I am trying to compile a program within a docker container built from the Alpine 3.7 base image. The program uses argp.h, and includes it as #include <argp.h>. I have installed argp-standalone and verified that it is making it onto the image. The file argp.h is located in usr/include, however when I compile my program using the following commands:
gcc -W -Wall -Wextra -I/usr/include -c -o progname.o progname.c
gcc -largp -o progname progname.o
I get the following error:
progname.o: In function `parse_opt':
progname.c:(.text+0x4c9): undefined reference to `argp_failure'
progname.c:(.text+0x50f): undefined reference to `argp_failure'
progname.c:(.text+0x555): undefined reference to `argp_failure'
progname.c:(.text+0x59b): undefined reference to `argp_failure'
progname.c:(.text+0x5ce): undefined reference to `argp_error'
progname.c:(.text+0x5f4): undefined reference to `argp_error'
progname.o: In function `main':
progname.c:(.text+0x1397): undefined reference to `argp_parse'
collect2: error: ld returned 1 exit status
make: *** [Makefile:9: progname] Error 1
I have:
Ensured that the version of argp.h which is on the image does in fact include the argp_failure, argp_parse, and argp_error functions.
Tried moving argp.h into different locations on the machine (e.g. into the same directory where compilation is taking place, into /usr/lib)
Tried compiling with -l and -L.
The relevant packages also installed in the image are build-base, make, and gcc.
When compiling on an Ubuntu image these same commands work fine, even without the -largp and -I/usr/include flags. What could be happening differently within an Alpine image which would cause this not to work?
Edit
As per #Pablo's comment, I'm now compiling it as follows:
gcc -W -Wall -Wextra -I/usr/include -L/usr/lib -c -o progname.o progname.c
gcc -largp -o progname progname.o
After having verified that the static library, libargp.a, is located in /usr/lib. However, the same problem still persists.
Edit 2
Compiling as follows (and once again as per #Pablo's suggestion) has resolved the error I was having:
gcc -W -Wall -Wextra -I/usr/include -L/usr/lib -c -o progname.o progname.c
gcc -o progname progname.o /usr/lib/libargp.a
However, I am still curious why, using the exact same library and instructions, this would fail to compile in an Alpine image while compiling without issue in an Ubuntu image.
I am still curious why, using the exact same library and instructions, this would fail to compile in an Alpine image while compiling without issue in an Ubuntu image.
The reason for the linking error on Alpine may be kind of surprising, and is actually not specific to Alpine.
While this fails to link:
gcc -largp -o progname progname.o
This works:
gcc -o progname progname.o -largp
The reason is the order of parameters passed to the linker, and it related to the linking algorithm. Typically, in the linking command line objects are specified first (and possibly user's static libraries in any), then libraries using -l. The standard linker algorithm is explained perfectly in Eli Bendersky's article, Library order in static linking:
Object files and libraries are provided in a certain order on the command-line, from left to right. This is the linking order. Here's what the linker does:
The linker maintains a symbol table. This symbol table does a bunch of things, but among them is keeping two lists:
A list of symbols exported by all the objects and libraries encountered so far.
A list of undefined symbols that the encountered objects and libraries requested to import and were not found yet.
When the linker encounters a new object file, it looks at:
The symbols it exports: these are added to the list of exported symbols mentioned above. If any symbol is in the undefined list, it's removed from there because it has now been found. If any symbol has already been in the exported list, we get a "multiple definition" error: two different objects export the same symbol and the linker is confused.
The symbols it imports: these are added to the list of undefined symbols, unless they can be found in the list of exported symbols.
When the linker encounters a new library, things are a bit more interesting. The linker goes over all the objects in the library. For each one, it first looks at the symbols it exports.
If any of the symbols it exports are on the undefined list, the object is added to the link and the next step is executed. Otherwise, the next step is skipped.
If the object has been added to the link, it's treated as described above - its undefined and exported symbols get added to the symbol table.
Finally, if any of the objects in the library has been included in the link, the library is rescanned again - it's possible that symbols imported by the included object can be found in other objects within the same library.
When -largp appears first, the linker does not include any of its objects in the linking procedure, since it doesn't have any undefined symbols yet. If the static library is provided by path, and not with -l, then all of its objects are added to the linking procedure.
I have an libfoo.a which contains _start and all required symbols for an executable. ld.bfd -o foo libfoo.a works smoothly in my case. However, ld.gold -o foo libfoo.a fails silently generating an executable with no symbols from libfoo.a. Creating an empty a.o and link it with ld.gold -o foo a.o libfoo.a works.
I was wondering is there any way to directly link a static library into an executable using ld.gold without creating a redundant empty object files?
You can specify the entry symbol explicitly with the -e _start option, and the linker will use that to decide that it needs to load the object that defines it.
Unfortunately, gold will not use the implicit start symbol to load an object from the archive library.
I'm creating a bare-metal application for the stm32f407 microcontroller, which has an ARM Cortex M4 core. As such, I'm delivering the implementation of functions like _sbrk myself. I now find that when I try to create a static library containing _sbrk, and link it with my main.c into an application, the linker says
"c:/progra~2/gnutoo~1/4947e~1.920/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/lib/armv7e-m/softfp\libg_nano.a(lib_a-sbrkr.o): In function _sbrk_r: sbrkr.c:(.text._sbrk_r+0xc): undefined reference to `_sbrk'".
If I take that same function out of the static library, and put it into main.c, everything compiles/links/runs just fine.
I am almost certain that this has something to do with the order in which the linker reads in all libraries, and that when my own static library is read, no definition of _sbrk is required yet, and is therefore thrown out, only to find that it was needed anyway when later one of the standard libraries is linked in. However, I do not specify any standard libraries myself, and can therefore not change the order of linking those libraries. I also tried to declare the _sbrk function as __attribute __ ((__ used__ )), thinking that the linker would not throw away that function, but alas, this has not solved my problem.
So my question is, how can I put _sbrk into a static library, without running into unresolved references?
Update: The command to link the final application is:
C:\PROGRA~2\GNUTOO~1\4947E~1.920\bin\AR10B2~1.EXE -g -mcpu=cortex-m4 -mfpu=fpv4-sp-d16 -mfloat-abi=softfp -mthumb -ffunction-sections -fno-rtti -fno-exceptions -std=c++11 -fno-use-cxa-atexit -fno-threadsafe-statics -g -mcpu=cortex-m4 -mfpu=fpv4-sp-d16 -mfloat-abi=softfp -mthumb -Wl,--gc-sections -nostartfiles -Wl,-T"C:/Users/Richard Peters/Documents/Projects/Embedded/http_server/ldscripts/mem.ld" -Wl,-T"C:/Users/Richard Peters/Documents/Projects/Embedded/http_server/ldscripts/sections.ld" "CMakeFiles\http_server.http_server.dir\src\main.cpp.obj" "CMakeFiles\http_server.http_server.dir\src\vectors_stm32f4xx.c.obj" "CMakeFiles\http_server.http_server.dir\http_server.http_server_linker_script_dummy.c.obj" -o "c:\Users\Richard Peters\Documents\Projects\Embedded-install\targets\http_server.http_server\Generic-stm32f4xx\bin\http_server.http_server.elf" "c:\Users\Richard Peters\Documents\Projects\Embedded-install\targets\cmsis_stm.cmsis_stm\Generic-stm32f4xx\lib\libcmsis_stm.cmsis_stm.a"
Where AR10B2~.EXE resolves to arm-none-eabi-g++.exe.
Adding the following makes the thing link:
-lc -lg "c:\Users\Richard Peters\Documents\Projects\Embedded-install\targets\cmsis_stm.cmsis_stm\Generic-stm32f4xx\lib\libcmsis_stm.cmsis_stm.a"
Th libcmsis_stm.cmsis_stm.a library is built with the following command:
C:\PROGRA~2\GNUTOO~1\4947E~1.920\bin\ARM-NO~2.EXE cq "c:\Users\Richard Peters\Documents\Projects\Embedded-install\targets\cmsis_stm.cmsis_stm\Generic-stm32f4xx\lib\libcmsis_stm.cmsis_stm.a" CMakeFiles/cmsis_stm.cmsis_stm.dir/src/cmsis/system_stm32f4xx.c.obj
Where ARM-NO~2.EXE resolves to arm-none-eabi-ar.exe
So one question remains: I would like to place the interrupt vector table, which is a variable, into the static library, but the linker throws that variable away because no source file needs that variable. Is there a mechanism to keep that variable until the linker processes the output sections in the linker file?
When ld links against a library, it will only pick those functions which are required at that time (because of references to functions from translation units which have been linked in before). The linker will forget all other functions (and the library won't be considered later).
Therefore the linking order does matter. Normally you would link in your application object file (which references malloc), then the standard library (which provides malloc and in turn references _sbrk), and then your (application) library which provides _sbrk.
So linking should look like
arm-none-eabi-gcc ... -o out.elf startup.o main.o -lc -lm -lapp
with the _sbrk function being provided by libapp.
So the order of the objects to be linked does matter.
Update
As stated in one of the comments: If you add debug symbols using -g during compilation, then you have to link against libg as well (-lg).
arm-none-eabi-gcc ... -o out.elf startup.o main.o -lc -g -lm -lapp
I have a static library, liborc-0.4.a with no shared library. I have another library, libschroedinger-1.0.a (no shared) that depends on symbols in liborc-0.4.a. If I run nm on liborc-0.4.a, symbols such as orc_init show up as T (meaning they are defined). I built libschroedinger-1.0.a with the command line flag -lorc-0.4 so it saw the symbols and was ok.
However, now I have a small executable that depends on libschroedinger-1.0.a. It compiles fine, but when I run the linker
gcc -lschroedinger-1.0 -lorc-0.4 -o output input.o
It gives errors such as:
/usr/local/lib/libschroedinger-1.0.a(libschroedinger_1.0_la-schro.o):schro.c:(.text+0x21):
undefined reference to `orc_init'
gcc is sensitive to the order of libraries. When it's compiling liborc-0.4.a in, there is no need for orc_init, so it's not included. The solution is to put the LDFLAGS at the end of the command:
gcc -o output input.o -lschroedinger-1.0 -lorc-0.4
You most probably compiled libschroedinger with shared liborc. Static library is the same as bunch of object files in an archive, so they don't need to see more than headers. Write like the following to be sure (the same apples to liborc).
gcc /path/to/libschroedinger-1.0.a /path/to/liborc-0.4.a -o output input.o