Suppose I have a static library libx.a. How to I make some symbols (not all) from this library to be always present in any binary I link with my library? Reason is that I need these symbols to be available via dlopen+dlsym. I'm aware of --whole-archive linker switch, but it forces all object files from library archive to linked into resulting binary, and that is not what I want...
Observations so far (CentOS 5.4, 32bit) (upd: this paragraph is wrong; I could not reproduce this behaviour)
ld main.o libx.a
will happily strip all non-referenced symbols, while
ld main.o -L. -lx
will link whole library in. I guess this depends on version of binutils used, however, and newer linkers will be able to cherry-pick individual objects from a static library.
Another question is how can I achieve the same effect under Windows?
Thanks in advance. Any hints will be greatly appreciated.
Imagine you have a project which consists of the following three C files in the same folder;
// ---- jam.h
int jam_badger(int);
// ---- jam.c
#include "jam.h"
int jam_badger(int a)
{
return a + 1;
}
// ---- main.c
#include "jam.h"
int main()
{
return jam_badger(2);
}
And you build it with a boost-build bjam file like this;
lib jam : jam.c <link>static ;
lib jam_badger : jam ;
exe demo : jam_badger main.c ;
You will get an error like this.
undefined reference to `jam_badger'
(I have used bjam here because the file is easier to read, but you could use anything you want)
Removing the 'static' produces a working binary, as does adding static to the other library, or just using the one library (rather than the silly wrapping on inside the other)
The reason this happens is because ld is clever enough to only select the parts of the archive which are actually used, which in this case is none of them.
The solution is to surround the static archives with -Wl,--whole-archive and -Wl,--no-whole-archive, like so;
g++ -o "libjam_candle_badger.so" -Wl,--whole-archive libjam_badger.a Wl,--no-whole-archive
Not quite sure how to get boost-build to do this for you, but you get the idea.
First things first: ld main.o libx.a does not build a valid executable. In general, you should never use ld to link anything directly; always use proper compiler driver (gcc in this case) instead.
Also, "ld main.o libx.a" and "ld main.o -L. -lx" should be exactly equivalent. I am very doubtful you actually got different results from these two commands.
Now to answer your question: if you want foo, bar and baz to be exported from your a.out, do this:
gcc -Wl,-u,foo,-u,bar,-u,baz main.o -L. -lx -rdynamic
Update:
your statement: "symbols I want to include are used by library internally only" doesn't make much sense: if the symbols are internal to the library, why do you want to export them? And if something else uses them (via dlsym), then they are not internal to the library -- they are part of the library public API.
You should clarify your question and explain what you really are trying to achieve. Providing sample code will not hurt either.
I would start with splitting off those symbols you always need into a seperate library, retaining only the optional ones in libx.a.
Take an address of the symbol you need to include.
If gcc's optimiser anyway eliminates it, do something with this address - should be enough.
Related
I'm writing a program using libraries foo1.a and foo2.a.
Inside foo2.a, it uses foo3.a, which implements a function funcfoo. But foo1.a contains the same function that implements its own funcfoo. My main program wants to use funcfoo from foo1.a, and some other functions from foo2.a, while also makes sure that foo2.a only uses funcfoo from foo3.a.
Is there anyway I can enforce this to happen?
Short answer : you can't
Long answer : you still can't because name (like function name as well as global variable or enum name ... any name) have to be unique across your binary[1]
but
there is maybe action you can do.
If you own the source code of any library involved in this mess, make a complete new library with the same stuff but renamed.
Basically, i say "make a new major version of you library, since you will change the function name present in the library, thus breaking the retro-compatibility".
If you library is "foo1.a" containing the function "min", then make a "foo2.a" with "foo2_min" function.
It's usually a good pratice to prefix/suffix your internal function (be it in a library or directly in your binary) with something. Like if your company name is "My Little Pony", "MLP_" sound a good prefix (do a google search, just in case).
That's what user694733 said in the comment.
If you're using an IDE, then it should be easy and quick to do so with the "rename refactoring" feature.
I advise to do the most renaming possible in order to avoid further scenario like the one your stuck with now.
You don't have the source code, but the library licence allow you to modify it.
If it's format is open (like a good old so), you can use objcopy.
I read it can do that, but I never do it myself so .... good luck
If it's a close format, either you have the documentation about it and you have to do a lot of work, or you're completly stuck.
A third party own the code source, and/or the library licence don't allow you to modify it
If it's a third party library, maybe your company has subscribed to his support, so you better contact them directly.
.
As far as I know, there is no possibility to "encapsulate" a library into a "spacename" nativly in C.
I hope this answer an help, and I hope it's accurate and complete.
[1] Well, this is not completly rigth as you can create a local variable with a global variable's name, and this will compile and run. In the local context, it's the local variable that will be used. gcc can warn about this kind of scenario with -Wshadow option.
But unless you're doing some shady hack, this situation is usually something that you want to avoid.
Most linkers obey the order of the libraries as you provide them on the command line. If you link your program first with "foo1.a", it will resolve the references of funcfoo in your program with the implemention of "foo1.a". Place "foo2.a" second, which will leave an open reference to funcfoo. With "foo3.a" placed third, this will be resolved with the second implementation.
EDIT 1:
I'm afraid that I was wrong. A quick check (just before going to sleep) revealed that when "foo2.a" is linked, the reference to funcfoo is resolved with the implementation of the already loaded "foo1.a". :-(
I will do some more research, but please don't hold your breath.
EDIT 2:
OK, it took some time, but with the help of "objcopy" is works as proposed. You can use its option --redefine-sym old=new to "rename" symbols even in a library.
I have prepared a small example to follow the steps as a prove of concept. In the real project, the libraries are already built, so just the last commands of the shown list are needed.
Let's start with the main source:
#include "foo1.h"
#include "foo2.h"
int main(void) {
funcfoo();
funcbar();
return 0;
}
It includes the following header files, first "foo1.h" and second "foo2.h":
#ifndef FOO1_H
#define FOO1_H
void funcfoo(void);
#endif
#ifndef FOO2_H
#define FOO2_H
void funcbar(void);
#endif
Of course there are implementations of both, again first "foo1.c" and second "foo2.c":
#include <stdio.h>
#include "foo1.h"
void funcfoo(void) {
puts("funcfoo() in foo1");
}
#include "foo2.h"
#include "foo3.h"
void funcbar(void) {
funcfoo();
}
The third library "foo3" implements the same function as "foo1", first the header file and then the implementation file:
#ifndef FOO3_H
#define FOO3_H
void funcfoo(void);
#endif
#include <stdio.h>
#include "foo3.h"
void funcfoo(void) {
puts("funcfoo() in foo3");
}
These are the commands to build the application:
gcc -c -Wall -Wextra foo1.c -o foo1.o
ar cr libfoo1.a foo1.o
gcc -c -Wall -Wextra foo2.c -o foo2.o
ar cr libfoo2.a foo2.o
gcc -c -Wall -Wextra foo3.c -o foo3.o
ar cr libfoo3.a foo3.o
# In the real prject, only the following steps are needed:
gcc -c -Wall -Wextra main.c -o main.o
objcopy --redefine-sym funcfoo=funcfoo2 libfoo2.a libfoo2n.a
objcopy --redefine-sym funcfoo=funcfoo2 libfoo3.a libfoo3n.a
gcc main.o -L. -lfoo1 -lfoo2n -lfoo3n -o app
As you can see, the tool "objcopy" of the common "binutils" is used to redefine the name of the conflicting function from funcfoo to funcfoo2. I also let it create a new output library to save the original.
And when run, the application prints:
funcfoo() in foo1
funcfoo() in foo3
I need to compile and, most importantly, link a C program that uses a proprietary function present in a shared library file. Because of lack of communication with the previous development team, there is no proper documentation. I declared a function prototype (because I know the number and type of arguments):
int CustomFunction(unsigned char *in, int size);
Since that function name can be grepped from /customlibs/libcustom.so, I tried to compile the code and link it like this:
gcc -L/customlibs testing.c -o testing -lcustom
Which throws a few error messages looking like this:
/customlibs/libcustom.so: undefined reference to `AnotherCustomFunction'
Obviously, I need to tell linker to include other libraries as well, and, to make things worse, they need to be in certain order. I tried exporting LD_LIBRARY_PATH, using -Wl,-rpath=, -Wl,--no-undefined and -Wl,--start-group. Is there an easy way to give the linker all the .so files without the proper order?
I found the solution (or a workaround) to my problem: adding -Wl,--warn-unresolved-symbols, which turns errors to warnings. Note that this works only if you are ABSOLUTELY certain your function does not depend on the symbols mentioned in undefined refernce to: messages.
Add them on the command line is a way to do it. Something like this below. The LD_LIBRARY_PATH tells gcc where to look for libraries, but you still need to say what libraries to include.
gcc -L/customlibs testing.c -o testing -lcustom -lmylib1 -lmylib2 -lmylib3
You should also include all the header files of your shared library by adding the -I option of gcc, for example : gcc [...] -I/path/to/your/lib/header/files [...]
My question is fairly OS X on x86-64 specific but a universal solution that works on other POSIX OSes is even more appreciated.
Given a list of symbol names of some shared library (called original library in the following) and I want my shared library to re-export these symbols. Re-export as in if someone tries to resolve the symbol against my library I either provide my version of this symbol or (if my library doesn't have this symbol) forward to the original library's symbol.
I don't know the types of the symbols, I only know whether they are functions (type T in nm output) or other symbols (type S in nm output).
For functions, I already have a solution: For every function I want to re-export I generate an assembly stub that does dynamically resolve the symbol (using dlsym()) and then jumps into the resolved function with the very same environment (registers rdi, rsi, rdx, rcx, r8, r9, stack pointer, ...). I'm basically generating universal proxy functions. Using some macro trickery that can be generated fairly easy without writing code for each and every symbol.
For non-function symbols the problem seems to be harder because I cannot generate this universal proxy function, because the resolving party does never call a function.
Using a constructor function static void init(void) __attribute__((constructor)); I can execute code whenever someone loads my library, that would be a good point to resolve and re-export all non-function symbols if that's possible.
In other words, I'd like to write the symbol table of my library to point to the respective symbols of another shared library. Doing the rewriting at compile or run time is okay (run time preferred). Or put yet another way, the behaviour of DYLD_INSERT_LIBRARIES (LD_PRELOAD) is exactly what I need but I don't want to insert a new library, I want to replace one (in the file system). EDIT: The reason I don't want/can't use DYLD_INSERT_LIBRARIES or any other environment variable of the DYLD_* family is that they are ignored for code signed, restricted, ... binaries.
I'm aware of the -reexport-l, -reexport_library and -reexported_symbols_list linker flags but I could not get them to work, especially when my library is a "replacement" for frameworks that are part of umbrella frameworks (example: /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/SearchKit) because ld forbids to link directly against parts of umbrella frameworks.
EDIT: Because I explained it somewhat ambiguously: I can't change the way the actual program is linked. The goal is to produce a shared library that is a replacement for the original library. (Apparently called filter library.)
Found it out now (OS X specific): clang -o replacement-lib.dylib ... -Xlinker -reexport_library PATH_TO_ORIGINAL_LIB does the trick. PATH_TO_ORIGINAL_LIB could for example be /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/Versions/Current/SearchKit.
If PATH_TO_ORIGINAL_LIB is a library that is part of an umbrella framework (as in the example above), then replace PATH_TO_ORIGINAL_LIB by the path of some other lib (I created a lib empty.dylib for that) and as a second step do
install_name_tool -change /usr/local/lib/empty.dylib PATH_TO_ORIGINAL_LIB replacement-lib.dylib
To see if the actual reexporting worked use:
otool -l replacement-lib.dylib | grep -A2 LC_REEXPORT_DYLIB
The output should look like
cmd LC_REEXPORT_DYLIB
cmdsize XX
name empty.dylib (offset YY)
After launching the install_name_tool it could be
cmd LC_REEXPORT_DYLIB
cmdsize XX
name /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/Versions/Current/SearchKit (offset YY)
You could link against both libraries and use the link order to make sure to link against the right symbols. This works on both OS X and Linux:
cc -o executable -lmylib -loriglib
Where origlib is the original library and mylib contains symbols that are supposed to overwrite symbols in origlib. Then the executable will be linked against your symbols from mylib first and all unresolved symbols will be linked against origlib.
This works in the same way when linking against OS X frameworks. Just link against your library that replaces symbols first and against the framework after.
cc -o executable -lmylib -framework SomeFramework
Edit: If you just want to replace symbols at runtime then you can use LD_PRELOAD in the same way:
cc -o executable -framework SomeFramework
LD_PRELOAD=libmylib.dylib ./executable
Now I am writing a program to call a web service. I write testMain.c. The others are generated by wsdl2h and soapcpp2.
My compiling command is like this:
gcc -Wall -g -c -L. soapC.c soapClient.c stdsoap2.c testMain.c
gcc -o testMain -L/usr/lib -lgsoap -lgsoapck -lgsoapssl soapC.o soapClient.o stdsoap2.o testMain.o
And I get these errors. Please help me.
stdsoap2.o: In function `soap_print_fault':
/test/stdsoap2.c:16279: undefined reference to `soap_check_faultsubcode'
/test/stdsoap2.c:16281: undefined reference to `soap_check_faultdetail'
stdsoap2.o: In function `soap_sprint_fault':
/test/stdsoap2.c:16341: undefined reference to `soap_check_faultdetail'
collect2: ld returned 1 exit status
Recent versions of GCC/ld/the GNU toolchain require that the object and library files be specified in a certain order, so that symbols can be found by the linker in the same order they depend on each other. This means that libraries should go to the end of the command line; your second line (when you're linking) should be
gcc -o testMain -L/usr/lib soapC.o soapClient.o stdsoap2.o testMain.o -lgsoap -lgsoapck -lgsoapssl
instead.
I search the web, and found a post which is very similar with my problem. I use this solution and have solved the problem. http://www.mail-archive.com/gsoap#yahoogroups.com/msg01022.html
You should not need to link stdsoap2.o to your project because it's already included in libgsoap (given through the gcc linker option -lgsoap). Try to exclude stdsoap2.c from your project. From the gSOAP FAQ:
I get a link error with gcc/g++ (GNU GCC). What should I do? For C
apps: use soapcpp2 option -c to generate C code, use only the
package's .c files, link with libgsoap.a (-lgsoap) or use the lib's
source stdsoap2.c (and dom.c when applicable).
I had the same problem with gsoap-2.8.16 compiled from source. (That version was shipped with CentOS 6.)
First I checked for a missing library. According to nm used on all static libraries provided by gsoap-2.8.16:
for X in /usr/local/lib/libgsoap*.a ; do echo $X; nm $X | grep soap_check_faultdetail; done`
it turned out that none of the libraries provided the missing symbols.
A brief look at the source code revealed that the expected return type of both methods soap_check_faultdetail and soap_check_faultsubcode was const char*, and that these were used to generate error messages.
It looked to me as if these are meant to be callbacks that the client must provide. Maybe their implementation is WSDL-dependent and would be supplied by the gsoap code generation utilities - that I don't know, see the answer from #ChristianAmmer above or below.
Anyway, since I knew the symbols were nowhere supplied, and that null-terminated strings were probably acceptable here, I just supplied my own no-op implementation:
// gsoap-missing-symbols.cpp
extern "C" {
const char* soap_check_faultdetail() { return 0; }
const char* soap_check_faultsubcode() { return 0; }
}
This is a brute-force solution. If you follow this solution, you should maybe check for linker warnings in the future; maybe some mechanism (eg. from the gsoap code generator) will supply conflicting implementations later during development.
For later versions of gsoap, I believe these symbols are no longer used and can be dropped (or renamed), see soap_check_faultX in https://www.genivia.com/changelog.html.
I am trying to compile a program written in c using the expect.h header. I am trying this:
cc -I/usr/include main.c -lexpect -ltcl
cc -I/usr/include main.c
I tried the same with gcc, but to no avail.
I get this:
undefined reference to `exp_spawnl'
undefined reference to `exp_expectl'
Any ideas?
Thanks,
Walter
It means that regardless of what you thought was being used, the expect library being used does not contain those two functions. So, you need to track down which expect library is actually loaded, and then check whether the functions are in it (they aren't, but you should check to see what is defined).
Once you've found your library, you can use nm -g libexpect.so to see the symbols defined. I would expect that you will not see exp_spawnl or exp_expectl in the list of defined symbols.
Then you need to find out which version of the library you do have, and decide whether to upgrade, etc.
I am assuming that the names you quote are indeed spelled correctly.
Is the expect header file installed in /usr/include?
I think the undefined reference error comes from the linker and not the compiler. What directory is the library installed to? Try to link against it using the
-L/path/to/lib -lexpect
option