Let's say I have a binary foo and bar. They both link to library baz.
I want to enable ASAN only for the process foo but not for process bar.
I build two version of baz, one with ASAN instrumented (package it in /pkg/asan/lib/) and other without asan (package it in /pkg/lib).
I want process foo to be link/load to libbaz from /pkg/asan/lib/ , but want process bar to link/load the library from /pkg/lib.
Is there anyways at compile time i can specify foo to first check if the library is present in /pkg/asan/lib , if not go then link with the library from /pkg/lib.
Is there anyways at compile time i can specify foo to first check if the library is present in /pkg/asan/lib , if not go then link with the library from /pkg/lib.
Sure: when linking foo with AddressSanitizer, add -L/pkg/asan/lib -Wl,-rpath=/pkg/asan/lib options.
Related
System spec: Ubuntu 16.04, QtCreator 3.5.1 based on Qt5.5.1
In a C project I'm using dlopen and friends to load a shared library. The library uses some machinery from the main application.
Now, if I compile manually from terminal, everything works just fine. When I use QtCreator to build the project, calls from the library to the main application fail with the following error:
/Programming/C/emme_1/emme_1: symbol lookup error: /Programming/C/emme_1/modtest/test.so: undefined symbol: pop. Here pop is a function in the main application which is named emme_1. Please note that the error happens only if the library's function calls another function inside the main application. If the library's function is empty and is called from the application, it works.
This is my very simple .pro file:
QMAKE_CFLAGS += -DTRACE_ASM
QMAKE_CFLAGS += -rdynamic
QMAKE_CFLAGS += -ldl
LIBS += /usr/lib/x86_64-linux-gnu/libdl.so
I believe it contains all it's needed for the requested functionality, but maybe I'm missing something.
I believe it contains all it's needed for the requested functionality, but maybe I'm missing something.
You must be missing something since the setup doesn't work.
Note that -rdynamic and -ldl are linker flags, not compile flags. You'll want to move them to QMAKE_LFLAGS.
You can look at the final link command, and verify that -rdynamic is not in it. Once you manage to get it there, things will work again.
I am working on a complex C ecosystem where different packages/libraries are developed by different people.
I would like to create a new project named foobar. This project uses two libraries, the library foo and the library bar.
Unfortunately, bar does not require the same version that foo requires. Both use say so there is a conflict.
If all the packages are on Git with submodules, the foobar project cannot be built when cloned recursively because two say functions exist in different translation units. So the submodule strategy doesn't work.
My question is: how is it possible to manage one project that uses two different version of the same static library (*.a)?
Structure
foobar
|
.----'----. <---- (require)
v v
foo bar
(v1.0) | | (v2.0)
'-> say <-'
The project foobar require the library foo and the library bar, both of these libraries uses the say package: foo requires version 1 and bar requires version 2.
Packages
say
// say.h
void say(char *);
foo
// foo.c
#include "say.h"
void foo(void) {
say("I am foo");
}
bar
// bar.c
#include "say.h"
void bar(void) {
say("I am bar");
}
foobar
// main.c
#include <stdlib.h>
#include "foo"
#include "bar"
int main() {
foo();
bar();
return EXIT_SUCCESS;
}
Linkers typically have a mode in which they perform a partial link, which resolves references that can currently be resolved and produces an object module ready for further linking instead of a finished executable file.
For example, the GCC linker ld has a -r switch that allows this. Using this switch, and possibly others, you could link foo.o with one library to make foo.partial.o and separately link bar.o with another library to make bar.partial.o. Then you could link foo.partial.o and bar.partial.o with each other, the main program, and any other libraries and object modules needed.
This can work for static libraries, where the code for each library is included in the resulting executable or object file, and the references to its symbols are fully resolved. For shared dynamic libraries, there may be problems, since dynamic libraries require references to be resolved at run time, and the linker and executable file format might or might not support the ability to distinguish symbols of the same name in different versions of one library.
My question is fairly OS X on x86-64 specific but a universal solution that works on other POSIX OSes is even more appreciated.
Given a list of symbol names of some shared library (called original library in the following) and I want my shared library to re-export these symbols. Re-export as in if someone tries to resolve the symbol against my library I either provide my version of this symbol or (if my library doesn't have this symbol) forward to the original library's symbol.
I don't know the types of the symbols, I only know whether they are functions (type T in nm output) or other symbols (type S in nm output).
For functions, I already have a solution: For every function I want to re-export I generate an assembly stub that does dynamically resolve the symbol (using dlsym()) and then jumps into the resolved function with the very same environment (registers rdi, rsi, rdx, rcx, r8, r9, stack pointer, ...). I'm basically generating universal proxy functions. Using some macro trickery that can be generated fairly easy without writing code for each and every symbol.
For non-function symbols the problem seems to be harder because I cannot generate this universal proxy function, because the resolving party does never call a function.
Using a constructor function static void init(void) __attribute__((constructor)); I can execute code whenever someone loads my library, that would be a good point to resolve and re-export all non-function symbols if that's possible.
In other words, I'd like to write the symbol table of my library to point to the respective symbols of another shared library. Doing the rewriting at compile or run time is okay (run time preferred). Or put yet another way, the behaviour of DYLD_INSERT_LIBRARIES (LD_PRELOAD) is exactly what I need but I don't want to insert a new library, I want to replace one (in the file system). EDIT: The reason I don't want/can't use DYLD_INSERT_LIBRARIES or any other environment variable of the DYLD_* family is that they are ignored for code signed, restricted, ... binaries.
I'm aware of the -reexport-l, -reexport_library and -reexported_symbols_list linker flags but I could not get them to work, especially when my library is a "replacement" for frameworks that are part of umbrella frameworks (example: /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/SearchKit) because ld forbids to link directly against parts of umbrella frameworks.
EDIT: Because I explained it somewhat ambiguously: I can't change the way the actual program is linked. The goal is to produce a shared library that is a replacement for the original library. (Apparently called filter library.)
Found it out now (OS X specific): clang -o replacement-lib.dylib ... -Xlinker -reexport_library PATH_TO_ORIGINAL_LIB does the trick. PATH_TO_ORIGINAL_LIB could for example be /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/Versions/Current/SearchKit.
If PATH_TO_ORIGINAL_LIB is a library that is part of an umbrella framework (as in the example above), then replace PATH_TO_ORIGINAL_LIB by the path of some other lib (I created a lib empty.dylib for that) and as a second step do
install_name_tool -change /usr/local/lib/empty.dylib PATH_TO_ORIGINAL_LIB replacement-lib.dylib
To see if the actual reexporting worked use:
otool -l replacement-lib.dylib | grep -A2 LC_REEXPORT_DYLIB
The output should look like
cmd LC_REEXPORT_DYLIB
cmdsize XX
name empty.dylib (offset YY)
After launching the install_name_tool it could be
cmd LC_REEXPORT_DYLIB
cmdsize XX
name /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/Versions/Current/SearchKit (offset YY)
You could link against both libraries and use the link order to make sure to link against the right symbols. This works on both OS X and Linux:
cc -o executable -lmylib -loriglib
Where origlib is the original library and mylib contains symbols that are supposed to overwrite symbols in origlib. Then the executable will be linked against your symbols from mylib first and all unresolved symbols will be linked against origlib.
This works in the same way when linking against OS X frameworks. Just link against your library that replaces symbols first and against the framework after.
cc -o executable -lmylib -framework SomeFramework
Edit: If you just want to replace symbols at runtime then you can use LD_PRELOAD in the same way:
cc -o executable -framework SomeFramework
LD_PRELOAD=libmylib.dylib ./executable
If I have two libraries, A.lib and B.lib, both of which export foo and bar, how do I tell the linker to use the symbol foo from A.lib and the symbol bar from B.lib?
You cannot. For your example you can have definitions from foo.lib or bar.lib but not both (especially if you cannot recompile the libraries and set symbol visibility so that only the symbols you want are exported). The order you link them against your application will depend on which library's definitions are used (you'll have to use the scientific method, I think the first one linked wins). Niklas Hansson's answer is a great way to do this dynamically but it seems you don't want to modify the original application, either, to dynamically pick/choose what symbols to take out of the libraries.
If you really wanted to you could mangle the symbol tables with a hex editor so that the symbols you don't want exported have different names (hacky, but it would work). I know on Linux there is a tool called objcopy that would let you do this (not sure about Windows).
You can use LIB.EXE /EXTRACT ... to extract only the object files you want to use, and link those files into your own application.
Or you may use LIB to create one new library containing the elements you need:
First, use /REMOVE on A.LIB to remove bar.obj:
LIB.EXE /OUT:ANOBAR.LIB /REMOVE:bar.obj A.LIB
Then combine A.LIB and B.LIB, and make sure to use ANOBAR.LIB as the last on the command line to ensure its foo.obj is used instead of B.LIB's:
LIB.EXE /OUT:COMBINED.LIB B.LIB ANOBAR.LIB
Details are found here: Managing a library, especially the paragraph:
You can use LIB [...] To replace a library member with a new object, specify the library containing the member object to be replaced and the file name for the new object (or the library that contains it). When an object that has the same name exists in more than one input file, LIB puts the last object specified in the LIB command into the output library. When you replace a library member, be sure to specify the new object or library after the library that contains the old object.
I didn't test the command lines given, but I've used similar ones extensively in the past.
If you are using dynamic libraries, you could use dynamic loading and pick foo from A and bar from B when loading.
I am working on Linux environment. I have two 'C' source packages train and test_train.
train package when compiled generates libtrain.so
test_train links to libtrain.so and generates executable train-test
Now I want to generate a call graph using gprof which shows calling sequence of functions in main program as well as those inside libtrain.so
I am compiling and linking both packages with -pg option and debugging level is o0.
After I do ./train-test , gmon.out is generated. Then I do:
$ gprof -q ./train-test gmon.out
Here, output shows call graph of functions in train-test but not in libtrain.so
What could be the problem ?
gprof won't work, you need to use sprof instead. I found these links helpful:
How to use sprof?
http://greg-n-blog.blogspot.com/2010/01/profiling-shared-library-on-linux-using.html
Summary from the 2nd link:
Compile your shared library (libmylib.so) in debug (-g) mode. No -pg.
export LD_PROFILE_OUTPUT=`pwd`
export LD_PROFILE=libmylib.so
rm -f $LD_PROFILE.profile
execute your program that loads libmylib.so
sprof PATH-TO-LIB/$LD_PROFILE $LD_PROFILE.profile -p >log
See the log.
I found that in step 2, it needs to be an existing directory -- otherwise you get a helpful warning. And in step 3, you might need to specify the library as libmylib.so.X (maybe even .X.Y, not sure) -- otherwise you get no warning whatsoever.
I'm loading my library from Python and didn't have any luck with sprof. Instead, I used oprofile, which was in the Fedora repositories, at least:
operf --callgraph /path/to/mybinary
Wait for your application to finish or do Ctl-c to stop profiling. Now let's generate a profile summary:
opreport --callgraph --symbols
See the documentation to interpret it. It's kind of a mess. In the generated report, each symbol is listed in a block of its own. The block's main symbol is the one that's not indented. The items above it are functions that call that function, and the ones below it are the things that get called by it. The percentages in the below section are the relative amount of time it spent in those callees.
If you're not on Linux (like me on Solaris) you simply out of luck as there is no sprof there.
If you have the sources of your library you can solve your problem by linking a static library and making your profiling binary with that one instead.
Another way I manage to trace calls to shared libraries, is by using truss. With the option -u [!]lib,...:[:][!]func, ... one can get a good picture of the call history of a run. It's not completely the same as profiling but can be very usefull in some scenarios.