Problem getting the GNU linker (ld) to export a symbol - linker

I am using certain GNU tools, i.e. the GNU C++ Compiler (g++) and the GNU Linker (ld) to create a shared library (.so) file as well as a binary executable file.
The binary executable file makes use of the dlopen function to dynamically load the shared library file at runtime. In addition to this, the shared library file
needs to invoke a particular class method (called ToolboxManager::registerToolbox) which is defined within the binary executable. This is accommodated by forcing the binary
executable to export the class method, which in turn is accomplished at link time by linking the binary executable with the following command line options ;
-Wl,--dynamic-list=${top_srcdir}/dynamic_symbol_table.txt
where the file ${top_srcdir}/dynamic_symbol_table.txt contains the following content ;
{
extern "C++"
{
"ToolboxManager::registerToolbox*";
};
};
Note the use of the asterisk (*) in the file to force the linker to export all symbols that begin with ToolboxManager::registerToolbox.
When I run the GNU nm utility (nm -C -g ./a.out) on the resulting binary executable, it displays the following information about the afore-mentioned class method ;
08053da0 T ToolboxManager::registerToolbox
(
std::string&,
std::string&,
std::map
<
std::string,
Factory_DSPB_Base*,
std::less
<
std::string
>,
std::allocator
<
std::pair
<
std::string const,
Factory_DSPB_Base*
>
>
>&
)
or, if the nm utility is invoked as above, but this time without the use of the -C command line switch ;
08053da0 T _ZN14ToolboxManager15registerToolboxERSsS0_RSt3mapISsP17Factory_DSPB_BaseSt4lessISsESaISt4pairIKSsS3_EEE
So far, this looks fine. The "T" in front of the definition of the class method ToolboxManager::registerToolbox, denotes that the method resides within the Text/Code section of the file.
Similarly, if I run the nm utility (nm -C -g ./toolbox.so) on the shared library file, it displays the following information about the same afore-mentioned class
method ;
U ToolboxManager::registerToolbox
(
std::string&,
std::string&,
std::map
<
std::string,
Factory_DSPB_Base*,
std::less
<
std::string
>,
std::allocator
<
std::pair
<
std::string const,
Factory_DSPB_Base*
>
>
>&
)
This also looks fine. The "U" in front of the definition of the class method ToolboxManager::registerToolbox, denotes that the method is undefined in the shared library file.
However, a problem occurs when I run the binary exectable from the command line and this problem results in the following error message being displayed ;
./toolbox.so: undefined symbol: _ZN14ToolboxManager15registerToolboxERSsS0_RSt3mapISsP17Factory_DSPB_BaseSt4lessISsESaISt4pairIKSsS3_EEE
The mangled class method name which appears in this runtime message is shown below, as the first of the two lines. For comparison purposes, the mangled class method name from above (and which was generated using the the nm -g command) is shown below as the second of the two lines ;
_ZN14ToolboxManager15registerToolboxERSsS0_RSt3mapISsP17Factory_DSPB_BaseSt4lessISsESaISt4pairIKSsS3_EEE
_ZN14ToolboxManager15registerToolboxERSsS0_RSt3mapISsP17Factory_DSPB_BaseSt4lessISsESaISt4pairIKSsS3_EEE
As can be seen, the two mangled names are identical. Therefore, I cannot understand why the undefined symbol cannot be resolved at runtime.
I then re-linked the binary executable, however this time I replaced the following linker command ;
-Wl,--dynamic-list=${top_srcdir}/dynamic_symbol_table.txt
with the this one ;
-Wl,--export-dynamic
The --export-dynamic linker option instructs the GNU Linker to add all the symbols to the dynamic symbol table.
If then ran the binary executable again. This time it executed
correctly and the call to the dlopen function did not result in an undefined symbol error. This has me utterly perplexed, as it looks as though the symbol is being exported
correctly in the initial version of the binary executable. Is anyone able to see the problem here? Any assistance would be immensely appreciated.
Thanks in advance.

I have managed to solve this problem. I have found that if I remove the quotation marks from the following line ;
"ToolboxManager::registerToolbox*"
within the file ${top_srcdir}/dynamic_symbol_table.txt and then re-link the binary executable, then it works. That is, the dlopen function will not fail anymore.
I can't help but wonder if it would have been more appropriate to ask this question on the GNU binutils mailing list than here on this web-site.

Related

how to print all the undefined function calls along with file name from shared object?

I am trying to print all the Undefined function calls from a shared object file along with file name.
I tried with "nm" command, It print all the undefined function calls .But could not get the file name.
Example:
bash$ nm -u my_test.so
:
U _ZNSs4_Rep20_S_empty_rep_storageE##GLIBCXX_3.4
:
Environment : Ubuntu 18.04 , X86 Arch (Intel processor)
Study in details the specification of the DWARF format (which is the format used by debugging information on Linux). So you could extract the information (but it is not exactly simple) by parsing the DWARF inside your ELF binary.
Consider looking inside the source code of Ian Taylor's libbacktrace. It is doing this extraction of file name from DWARF inside ELF.
Perhaps your real problem is getting precise backtrace information, and then that libbacktrace is exactly what you need!
You might also use gdb : it is extensible and scriptable in Python (or Guile) and you could write your own specialized script.
Perhaps you'll better solve your real problem with some GCC plugin working when you compile your code.
Read How to write shared libraries by Drepper and read more about ELF.
You could for example collect all the undefined symbols in your shared library using nm (or readelf). Then a second script will find the occurrences of these in your source code. It could be even a simple awk script (or some for shell loop using grep), or something as sophisticated as a GCC plugin.
Your example shows (probably) a mangled C++ name. You could use nm -C to get it unmangled. And later write a GCC plugin to find all the GIMPLE CALL instructions using it.
Writing a GCC plugin may take some time, in particular if you are not familiar with GCC internals.

GNU linker get objects through INPUT

I want to use the suggestion made in de GNU linker manual page 40, i.e. INPUT (subr.o), thus specifying object members in a script file.
Eventually I want to specify all the object members of my program that the linker has to use.
The script file looks like this (only the files parts is shown)
SEARCH_DIR(../lib)
STARTUP(boot.o)
ENTRY(_start)
GROUP (libkernel.a libkflib.a)
INPUT (
lowcore.o
init.o
kfalloc.o
kflog.o
kfprintf.o
)
The linker replies with:
attempt to open boot.o failed
attempt to open ../lib/boot.o failed
m68k-rtems4.11-ld: cannot find boot.o
I have specified the search path, the libraries and a list of object members; the object members are definitly in the libraries.
I was expection the linker to look for object members in the working directory and if not there using the search path and libraries.
Obviously there is something wrong but I cannot figure it out.
Suggestions are welcome in order to achieve the desired way of linking: specifying all objects to link an not more than that.
Thanks
Ben
You have misread the manual:
INPUT(file file ...)
The INPUT command directs the linker to include the named files in the link,
as though they were named on the command line.
...
In case a sysroot prefix is configured, and the filename starts with the / character,
and the script being processed was located inside the sysroot prefix, the
filename will be looked for in the sysroot prefix. Otherwise, the linker will
try to open the file in the current directory. If it is not found, the linker
will search through the archive library search path. See the description of
-L in Command Line Options.
If you use INPUT (-lfile), ld will transform the name to libfile.a, as with
the command line argument -l.
...
GROUP(file file ...)
The GROUP command is like INPUT, except that the named files should all be
archives, and they are searched repeatedly until no new undefined references
are created.
STARTUP(filename)
The STARTUP command is just like the INPUT command, except that filename will
become the first input file to be linked, as though it were specified first on the command line
Or perhaps you have read some other documentation that is in error.
You have got wrong the impression that command GROUP(libboo.a...) is complementary to INPUT(foo.o...)
and STARTUP(bar.o), with the effect that files bar.o, foo.o... will be searched for in archives
libboo.a... and - if found - will be extracted from the archives and added to the linkage.
INPUT(foo.o) has the same effect as specifying foo.o on the linker
commandline except that if not found in the current directory it will be searched
for in the same way that a static library specified with the -l option would be, with any SEARCH_DIR(path)
commands in the linker script having the same influence as commandline options
-Lpath.
STARTUP(bar.o) has the same effect as specifying bar.o first in the linker
commandline.
GROUP(libboo.a...) has the same effect as the commandline options
--start-group -lboo.a... --end-group
again with any SEARCH_DIR(path) commands actiing like -Lpath.
INPUT(foo.o) and STARTUP(bar.o) are unconnected with GROUP(libboo.a...), just as in the commandline options:
bar.o --start-group -lboo.a... --end-group foo.o
bar.o and foo.o are an input files unconnected with --start-group -lboo.a... --end-group.
The linker never looks inside static libraries for an input object file that it otherwise
fails to find.
So this command:
INPUT (
lowcore.o
init.o
kfalloc.o
kflog.o
kfprintf.o
)
in your linker script requires the linker to find those object files in the current
directory or in ../lib, and they are not there. Similarly for STARTUP(boot.o). Hence
the linkage errors.
Those object files aren't there, but static libraries libkernel.a and libkflib.a are,
and you tell us they contain all all those object files as members. In that case
you simply don't need:
INPUT (
lowcore.o
init.o
kfalloc.o
kflog.o
kfprintf.o
)
because the linker will search static libraries to find object files that
provide definitions of any symbols for which definitions are called for
by references within object files that it has already linked. You don't have to
tell it to.
But to give the linker any reason to search a static library at all you
must have told it to link some object file that refers to some undefined symbol(s)
before it considers the static library.
That's the reason why, on the commandline, you must place object files before the
libraries to which they refer and it is why, in a self-contained linker script, if you
add libraries to the linkage with GROUP(libboo.a...) or with INPUT (-lboo...),
you must also specify an object file to be linked first, with a STARTUP
command.
So while your failing INPUT command is unnecessary and can just be
deleted, your failing STARTUP(boot.o) must remain, to initiate
any linkage. And must not fail.
To make STARTUP(boot.o) succeed, you have to put the object file boot.o
itself in a place where the script will tell the linker to look for it,
either the current directory or ../lib. Presumably, the current directory.
And when you do that, it becomes pointless to have boot.o remain as a member
of one your static libraries, since it can't be linked from within one. Best
delete it from whichever libary you have it in. Leaving it there is similar
to putting the main function of a program into one of the libraries you
will link it with.

cannot find symbol "Embeddedrcall_Init"

I am trying to create a dll file using swig for an embeddedR C Program in windows environment. I am using the below commands:
C:\swigwin-3.0.12\Examples\r\Z>swig -c++ -tcl embeddedRCall.i
C:\swigwin-3.0.12\Examples\r\Z>gcc -c embeddedRCall.c -I/swigwin-3.0.12/Examples/r/Z
C:\swigwin-3.0.12\Examples\r\Z>gcc -c embeddedRCall_wrap.c -I/Tcl/include/tcl8.6 -I/swigwin-3.0.12/Examples/r/Z
C:\swigwin-3.0.12\Examples\r\Z>gcc -shared embeddedRCall.o embeddedRCall_wrap.o -o embeddedRCall.dll -L/Tcl/lib -L/R/R-3.3.2/bin/i386 -lR -lRblas -lRiconv -lRlapack -ltcl86
% load embeddedRCall
cannot find symbol "Embeddedrcall_Init"
I was able to load other example.dll files with tclsh
However I was unable to figure out the reason-- I am already using tcl 32 bit
My module file name is and module name is embeddedRcall
Am I missing something???
I am relatively new to TCL can someone please help me.
You should have an exported (extern "C") function symbol in your library called something like Embeddedrcall_Init; it is the entry point that lets Tcl install the library into a specific interpreter instance. (It has to be found explicitly because it takes an argument.) By default, the name of the function is found by munging the name of the library (strip version number, case convert, append _Init) but the determination of the name can be overridden by the optional second argument to load.
To be more exact, if the entry is actually called EmbeddedRCall_Init, you would have to load it with:
load embeddedRCall EmbeddedRCall
# The _Init suffix is fixed when loading into a standard interp
Note the case difference! (Also, we recommend using fully qualified path names to loaded libraries, as it avoids some complexities in the dlopen() system.)

Re-export Shared Library Symbols from Other Library (OS X / POSIX)

My question is fairly OS X on x86-64 specific but a universal solution that works on other POSIX OSes is even more appreciated.
Given a list of symbol names of some shared library (called original library in the following) and I want my shared library to re-export these symbols. Re-export as in if someone tries to resolve the symbol against my library I either provide my version of this symbol or (if my library doesn't have this symbol) forward to the original library's symbol.
I don't know the types of the symbols, I only know whether they are functions (type T in nm output) or other symbols (type S in nm output).
For functions, I already have a solution: For every function I want to re-export I generate an assembly stub that does dynamically resolve the symbol (using dlsym()) and then jumps into the resolved function with the very same environment (registers rdi, rsi, rdx, rcx, r8, r9, stack pointer, ...). I'm basically generating universal proxy functions. Using some macro trickery that can be generated fairly easy without writing code for each and every symbol.
For non-function symbols the problem seems to be harder because I cannot generate this universal proxy function, because the resolving party does never call a function.
Using a constructor function static void init(void) __attribute__((constructor)); I can execute code whenever someone loads my library, that would be a good point to resolve and re-export all non-function symbols if that's possible.
In other words, I'd like to write the symbol table of my library to point to the respective symbols of another shared library. Doing the rewriting at compile or run time is okay (run time preferred). Or put yet another way, the behaviour of DYLD_INSERT_LIBRARIES (LD_PRELOAD) is exactly what I need but I don't want to insert a new library, I want to replace one (in the file system). EDIT: The reason I don't want/can't use DYLD_INSERT_LIBRARIES or any other environment variable of the DYLD_* family is that they are ignored for code signed, restricted, ... binaries.
I'm aware of the -reexport-l, -reexport_library and -reexported_symbols_list linker flags but I could not get them to work, especially when my library is a "replacement" for frameworks that are part of umbrella frameworks (example: /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/SearchKit) because ld forbids to link directly against parts of umbrella frameworks.
EDIT: Because I explained it somewhat ambiguously: I can't change the way the actual program is linked. The goal is to produce a shared library that is a replacement for the original library. (Apparently called filter library.)
Found it out now (OS X specific): clang -o replacement-lib.dylib ... -Xlinker -reexport_library PATH_TO_ORIGINAL_LIB does the trick. PATH_TO_ORIGINAL_LIB could for example be /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/Versions/Current/SearchKit.
If PATH_TO_ORIGINAL_LIB is a library that is part of an umbrella framework (as in the example above), then replace PATH_TO_ORIGINAL_LIB by the path of some other lib (I created a lib empty.dylib for that) and as a second step do
install_name_tool -change /usr/local/lib/empty.dylib PATH_TO_ORIGINAL_LIB replacement-lib.dylib
To see if the actual reexporting worked use:
otool -l replacement-lib.dylib | grep -A2 LC_REEXPORT_DYLIB
The output should look like
cmd LC_REEXPORT_DYLIB
cmdsize XX
name empty.dylib (offset YY)
After launching the install_name_tool it could be
cmd LC_REEXPORT_DYLIB
cmdsize XX
name /System/Library/Frameworks/CoreServices.framework/Frameworks/SearchKit.framework/Versions/Current/SearchKit (offset YY)
You could link against both libraries and use the link order to make sure to link against the right symbols. This works on both OS X and Linux:
cc -o executable -lmylib -loriglib
Where origlib is the original library and mylib contains symbols that are supposed to overwrite symbols in origlib. Then the executable will be linked against your symbols from mylib first and all unresolved symbols will be linked against origlib.
This works in the same way when linking against OS X frameworks. Just link against your library that replaces symbols first and against the framework after.
cc -o executable -lmylib -framework SomeFramework
Edit: If you just want to replace symbols at runtime then you can use LD_PRELOAD in the same way:
cc -o executable -framework SomeFramework
LD_PRELOAD=libmylib.dylib ./executable

how do I always include symbols from a static library?

Suppose I have a static library libx.a. How to I make some symbols (not all) from this library to be always present in any binary I link with my library? Reason is that I need these symbols to be available via dlopen+dlsym. I'm aware of --whole-archive linker switch, but it forces all object files from library archive to linked into resulting binary, and that is not what I want...
Observations so far (CentOS 5.4, 32bit) (upd: this paragraph is wrong; I could not reproduce this behaviour)
ld main.o libx.a
will happily strip all non-referenced symbols, while
ld main.o -L. -lx
will link whole library in. I guess this depends on version of binutils used, however, and newer linkers will be able to cherry-pick individual objects from a static library.
Another question is how can I achieve the same effect under Windows?
Thanks in advance. Any hints will be greatly appreciated.
Imagine you have a project which consists of the following three C files in the same folder;
// ---- jam.h
int jam_badger(int);
// ---- jam.c
#include "jam.h"
int jam_badger(int a)
{
return a + 1;
}
// ---- main.c
#include "jam.h"
int main()
{
return jam_badger(2);
}
And you build it with a boost-build bjam file like this;
lib jam : jam.c <link>static ;
lib jam_badger : jam ;
exe demo : jam_badger main.c ;
You will get an error like this.
undefined reference to `jam_badger'
(I have used bjam here because the file is easier to read, but you could use anything you want)
Removing the 'static' produces a working binary, as does adding static to the other library, or just using the one library (rather than the silly wrapping on inside the other)
The reason this happens is because ld is clever enough to only select the parts of the archive which are actually used, which in this case is none of them.
The solution is to surround the static archives with -Wl,--whole-archive and -Wl,--no-whole-archive, like so;
g++ -o "libjam_candle_badger.so" -Wl,--whole-archive libjam_badger.a Wl,--no-whole-archive
Not quite sure how to get boost-build to do this for you, but you get the idea.
First things first: ld main.o libx.a does not build a valid executable. In general, you should never use ld to link anything directly; always use proper compiler driver (gcc in this case) instead.
Also, "ld main.o libx.a" and "ld main.o -L. -lx" should be exactly equivalent. I am very doubtful you actually got different results from these two commands.
Now to answer your question: if you want foo, bar and baz to be exported from your a.out, do this:
gcc -Wl,-u,foo,-u,bar,-u,baz main.o -L. -lx -rdynamic
Update:
your statement: "symbols I want to include are used by library internally only" doesn't make much sense: if the symbols are internal to the library, why do you want to export them? And if something else uses them (via dlsym), then they are not internal to the library -- they are part of the library public API.
You should clarify your question and explain what you really are trying to achieve. Providing sample code will not hurt either.
I would start with splitting off those symbols you always need into a seperate library, retaining only the optional ones in libx.a.
Take an address of the symbol you need to include.
If gcc's optimiser anyway eliminates it, do something with this address - should be enough.

Resources