Suppose I would like to create a library A that links against another (system) library B of which the following files are installed:
$ ll /usr/lib/libB*
libB.so -> libB.so.3
libB.so.3 -> libB.so.3.0
libB.so.3.0
When creating my own libA.so*, I suppose it'd be good practice to include any of libB.so* on the link line. In fact, the linker flag -Wl,--no-undefined even enforces this.
It doesn't seem to make a difference which of the above libB files are used for linking since eventually they all point to the same file libB.so.3.0, but I'm guessing there are best practices for this, too.
What's recommended and why?
What's recommended and why?
Only linking against libB.so is officially supported. Linking against libB.so.3 or libB.so.3.0 works more or less by accident; don't do it.
You can read about external library versioning here.
Related
I'm compiling a C++ static library using g++ via Cmake. I want to remove symbols relating to the internal implementation so they don't show up in nm. (See here and here for the same with shared libraries.)
This answer tells you how to do it on iOS, and I'm trying to understand what happens under the hood so I can replicate on Linux. They invoke ld with:
-r/--relocatable to Generate relocatable output---i.e., generate an output file that can in turn serve as input to ld.
-x/--discard-all: Delete all local symbols.
AFAICS the -r glues all the modules into one module, and then the -x removes symbols only used inside that module. Is that right?
It's not clear how the linker 'knows' which symbols will be exported externally? Does it rely on __attribute__((visibility("hidden/default"))) as in the .so case?
Edit: clearly I'm confused... I thought cmake invoked ld to link the .os into .a. Googled + clarified above.
Question still stands: how do I modify the build process to exclude most symbols?
I need to compile and, most importantly, link a C program that uses a proprietary function present in a shared library file. Because of lack of communication with the previous development team, there is no proper documentation. I declared a function prototype (because I know the number and type of arguments):
int CustomFunction(unsigned char *in, int size);
Since that function name can be grepped from /customlibs/libcustom.so, I tried to compile the code and link it like this:
gcc -L/customlibs testing.c -o testing -lcustom
Which throws a few error messages looking like this:
/customlibs/libcustom.so: undefined reference to `AnotherCustomFunction'
Obviously, I need to tell linker to include other libraries as well, and, to make things worse, they need to be in certain order. I tried exporting LD_LIBRARY_PATH, using -Wl,-rpath=, -Wl,--no-undefined and -Wl,--start-group. Is there an easy way to give the linker all the .so files without the proper order?
I found the solution (or a workaround) to my problem: adding -Wl,--warn-unresolved-symbols, which turns errors to warnings. Note that this works only if you are ABSOLUTELY certain your function does not depend on the symbols mentioned in undefined refernce to: messages.
Add them on the command line is a way to do it. Something like this below. The LD_LIBRARY_PATH tells gcc where to look for libraries, but you still need to say what libraries to include.
gcc -L/customlibs testing.c -o testing -lcustom -lmylib1 -lmylib2 -lmylib3
You should also include all the header files of your shared library by adding the -I option of gcc, for example : gcc [...] -I/path/to/your/lib/header/files [...]
I am using the DDK to build a project. Several of the build targets generate small internal libraries -- for simplicity, let's call them LibA.lib, LibB.lib, and LibC.lib. But the code for LibA references code from LibB and LibC. To use LibA in a project, you also need to include LibB.lib, and LibC.lib to resolve all of the dependencies. What I want to do is link LibB and LibC into LibA such that a user only needs to include LibA.lib. How can I accomplish this?
The SOURCES file for LibA looks something like this:
TARGETNAME=LibA
TARGETTYPE=LIBRARY
TARGETPATH=..\lib\$(DDKBUILDENV)
INCLUDES = .; \
..\LibB; \
..\LibC; \
$(CRT_INC_PATH) \
$(SDK_INC_PATH)
SOURCES = LibA_main.cpp \
LibA_file2.cpp \
LibA_file3.cpp
I understand that you can manually link libraries with link.exe; e.g.,
link.exe /lib LibA.lib LibB.lib LibC.lib
But if possible, I would like some way to achieve this same effect as a part of the build process for LibA, because some targets at a later point of the build process rely on LibA.
Thanks!
I realize this is a late answer and it may not even be what you want in the end. However, ddkbuild.cmd has a pretty nifty mechanism to run actions before and after a build inside a particular directory.
We use this in one of our driver libraries which necessarily gets built as a number of static libraries and as a final step linked into one big static library much like you want. If you are able to use something like ddkbuild.cmd in your project, this would provide a solution and it's a solution that would work in automated builds, too.
NB: as far as I'm aware you cannot achieve what you want directly with build.exe. However, it's well possible that with some make (NMake) file acrobatics you could achieve a similar result. So the question is whether it's worth reinventing the wheel when there is one already.
I have encountered the same situation as you. Google a lot of and still have no solution. Fortunately, I found a way to resolve it finally. You can try it, add the below statement in your libA sources file.
LIBRARIAN_FLAGS = $(LIBRARIAN_FLAGS) libB.lib libC.lib
The lib utility can combine libraries. Using your example, the command would be:
lib /out:CombinedLib.lib LibA.lib LibB.lib LibC.lib
If one builds static libraries in one's build scripts and one wants to use those static libraries in linking the final executable, the order one mentions the .a files is important:
g++ main.o hw.a gui.a -o executable
If gui.a uses something defined in hw.a the link will fail, because at the time hw.a is processed, the linker doesn't yet know that the definition is needed later, and doesn't include it in the being.generated executable. Manually fiddling around with the linker line is not practical, so a solution is to use --start-group and --end-group which makes the linker run twice through the libraries until no undefined symbols are found anymore.
g++ main.o -Wl,--start-group hw.a gui.a -Wl,--end-group -o executable
However the GNU ld manual says
Using this option has a significant performance cost. It is best to use it only when there are unavoidable circular references between two or more archives.
So I thought that it may be better to take all .a files and put them together into one .a file with an index (-s option of GNU ar) which says in what order the files need to be linked. Then one gives only that one .a file to g++.
But I wonder whether that's faster or slower than using the group commands. And are there any problems with that approach? I also wonder, is there better way to solve these interdependency problems?
EDIT: I've written a program that takes a list of .a files and generates a merged .a file. Works with the GNU common ar format. Packing together all static libs of LLVM works like this
$ ./arcat -o combined.a ~/usr/llvm/lib/libLLVM*.a
I compared the speed against unpacking all .a files manually and then putting them into a new .a file using ar, recomputing the index. Using my arcat tool, I get consistent runtimes around 500ms. Using the manual way, time varies greatly, and takes around 2s. So I think it's worth it.
Code is here. I put it into the public domain :)
You can determine the order using the lorder and tsort utilities, for example
libs='/usr/lib/libncurses.a /usr/lib/libedit.a'
libs_ordered=$(lorder $libs | tsort)
resulting in /usr/lib/libedit.a /usr/lib/libncurses.a because libedit depends on libncurses.
This is probably only a benefit above --start-group if you do not run lorder and tsort again for each link command. Also, it does not allow mutual/cyclic dependencies like --start-group does.
Is there a third option where you just build a single library to begin with? I had a similar problem and I eventually decided to go with the third option.
In my experience, group is slower than just unifying the .a files. You can extract all files from the archive, then create a new .a file from from the smaller files
However, you have to be careful about a circumstance where both files happen to contain the same definition (you can explicitly check for this by using nm to see what definitions are contained in each library)
In my SConscript I have the following line:
Program("xtest", Split("main.cpp"), LIBS="mylib fltk Xft Xinerama Xext X11 m")
How do I get scons to use mylib.a instead of mylib.so, while linking dynamically with the other libraries?
EDIT: Looking to use as few platform specific hacks as possible.
Passing the full filepath wrapped in a File node will force static linking. For example:
lib = File('/usr/lib/libfoo.a')
Program('bar', 'main.c', LIBS = [lib])
Will produce the following linker command line
g++ -o bar main.o /usr/lib/libfoo.a
Notice how the "-l" flag is not passed to the linker for this LIBS entry. This effectively forces static linking. The alternative is to modify LINKFLAGS to get what you want with the caveat that you are bypassing the library dependency scanner -- the status of the library will not be checked for rebuilds.
To make this platform independent you append the env['SHLIBSUFFIX'] onto the library you want to use. env['SHLIBSUFFIX'] gives you this environments suffix for shared libraries.
You also have the ['SHLIBPREFIX'], ['LIBPREFIX'], ['LIBSUFFIX'] and ['PROGSUFFIX'], all useful for situations like this.
Edit:
I obviously haven't made myself understood, so I will clarify.
The return value of these lookups are strings to the pre/suffixes that platform uses. In that way you can refer to the file you need on each platform. Note that you cannot use it as a pure string, it has to be embedded as a file node as BennyG suggests. Working with nodes are anyway the best solution as file nodes are much more versatile than a string.
Hope this helps.