Scenario:
Executable loads shared object at run time via dlopen.
The shared object references some symbol (a function) that is actually compiled into the main executable.
This works fine if I add -rdynamic to gcc when linking the executable.
-rdynamic exports all non-static symbols of the executable. My shared object only needs a select few.
Question: Is there a way to achieve the effect of -rdynamic, but restricted the the few select symbols that I know are needed by my shared object?
Edit:
At least two people misunderstood the question, so I try to clarify:
This question is about exporting a symbol from the main executable.
This question is not about exporting a symbol from a dynamic library.
Here is a minimal example:
func.h, the common header file
#include <stdio.h>
void func(void);
main.c, the main executable code:
#include <dlfcn.h>
#include "func.h"
// this function is later called by plugin
void func(void) {
printf("func\n");
}
int main() {
void * plugin_lib = dlopen("./plugin.so", RTLD_NOW);
printf("dlopen -> %p, error: %s\n", plugin_lib, dlerror());
// find and call function "plugin" in plugin.so
void (*p)(void); // declares p as pointer to function
p = dlsym(plugin_lib, "plugin");
p();
return 0;
}
plugin.c, code for the plugin that is loaded at runtime:
#include "func.h"
void plugin()
{
printf("plugin\n");
func();
}
If I compile with
$ gcc -o main main.c -ldl
$ gcc -shared -fPIC -o plugin.so plugin.c
Then plugin.so cannot be loaded, because it references the symbol func, which cannot be resolved:
$ ./main
dlopen -> (nil), error: ./plugin.so: undefined symbol: func
Segmentation fault (core dumped)
I can convince the main executable to export all its global symbols by compiling with -rdynamic:
$ gcc -rdynamic -o main main.c -ldl
$ ./main
dlopen -> 0x75e030, error: (null)
plugin
func
But this fills the dynamic symbol table unnecessarily with all symbols.
(This dynamic symbol table can be inspected with nm -D main.)
The question is, how can I add only "func" to the dynamic symbol table of the main executable, and not everything.
Unfortunately it's harder to achieve this for executables. You need to generate a list of symbols that you want to export and then add -Wl,--dynamic-list=symfile.txt to LDFLAGS.
Here's example of how it's done in Clang (and here's the script they use to generate the symbols file).
You could do it with the visibility attribute of GCC.
Declare the function you need to export with __attribute__ ((visibility ("default"))) flag. Then compile your whole library passing -fvisibility=hidden argument to GCC.
For full explanation on this, refer to the following GCC documentation page.
Related
I have downloaded libgcrypt library source code and
added my own customize function inside one particular file.
Although compilation/build process of customized shared library is successful, and both nm and objdump show
the customized function is global, it nonetheless shows an error (undefined reference) at linking time.
Here is what I have done:
inside /src/visibility.c file, I have added my custom function,
void __attribute__((visibility("default"))) MyFunction(void)
{
printf("This is added just for testing purpose");
}
build process
./configure --prefix=/usr/local --disable-ld-version-script
sudo make install
nm and objdump command find this custom function as global inside shared library.
nm /usr/local/lib/libgcrypt.so | grep MyFunction
000000000000fbf0 T MyFunction
objdump -t /usr/local/lib/libgcrypt.so | grep MyFunction
000000000000fbf0 g F .text 0000000000000013 MyFunction
Here is my sample code to access my custom function.
//gcrypt_example_test.c
#include <stdio.h>
#include <gcrypt.h>
#include <assert.h>
int main()
{
MyFunction();
return 0;
}
export LD_RUN_PATH=/usr/local/lib
gcc gcrypt_example_test.c -o test -lgcrypt
/tmp/ccA0qgAB.o: In function `main':
gcrypt_example_test.c:(.text+0x3a2): undefined reference to `MyFunction'
collect2: error: ld returned 1 exit status
Edit 1:
I tried all possible way to include function prototype declaration inside header file (/src/gcrypt.h) as follows:
void __attribute__((visibility("default"))) MyFunction(void);
... or:
extern void __attribute__((visibility("default"))) MyFunction(void);
... or:
extern void MyFunction(void);
... or:
void MyFunction(void);
I am still getting the same error (undefined reference) although no build error results in all above cases.
Why is this happening, and what mistake am I making?
Although other global functions which are part of standard shared library and defined inside visibility.c (nm also shows T for those functions) are accessible, why is my customized global function (MyFunction) of the shared library still inaccessible? Thanks!
Any link or explanation to resolve this error will be highly appreciable.
From the GCC documentation (emphasis mine):
Some linkers allow you to specify the path to the library by setting LD_RUN_PATH in your environment when linking.
But, from the GNU ld man page:
-rpath=dir
Add a directory to the runtime library search path. This is used
when linking an ELF executable with shared objects. All -rpath
arguments are concatenated and passed to the runtime linker,
which uses them to locate shared objects at runtime. The -rpath
option is also used when locating shared objects which are needed
by shared objects explicitly included in the link; see the
description of the -rpath-link option. If -rpath is not used
when linking an ELF executable, the contents of the environment
variable "LD_RUN_PATH" will be used if it is defined.
Note that there is no mention at all of the link time library search path.
You need to compile/link with /usr/local/lib in the link time library search path:
gcc gcrypt_example_test.c -o test -L/usr/local/lib -lgcrypt
most likely cause of the problem:
The header file for the library has not been updated to include the prototype for the new function
I don't understand the reason behind why it is working now, but not before. Anyway, I found the way to make the code working after adding customized function inside standard library. This post may help others in future.
I first locate libgcrypt.so and then remove all versions of libgcrypt.so
locate libgcrypt.so
sudo rm /usr/local/lib/libgcrypt.so
sudo rm /usr/local/lib/libgcrypt.so.20
sudo rm /usr/local/lib/libgcrypt.so.20.2.2
then I delete the libgcrypt folder (which I had extracted for building library) to start fresh.
Again, I follow these steps
Step 0 : extract libgcrypt source code
Step 1 : add my custom function, inside /src/visibility.c file
void __attribute__((visibility("default"))) MyFunction(void)
{
printf("This is added just for testing purpose");
}
Step 2 : build library
export LD_RUN_PATH=/usr/local/lib
./configure --prefix=/usr/local --disable-ld-version-script
sudo make install
Step 3: Open another terminal to compile
export LD_RUN_PATH=/usr/local/lib
gcc gcrypt_example_test.c -o test -lgcrypt
Step 4 : run
./test
This is added just for testing purpose
This is working fine now as expected.
What I noticed that __attribute__((visibility("default"))) in function definition and --disable-ld-version-script during build process is very important to make the customized function global, elimination of any makes the customized function local inside shared library(.so) file.
Below changes are working at my end
visibility.h
#include <cstdio>
void __attribute__((visibility("default"))) MyFunction(void);
visibility.cpp
#include "visibility.h"
void MyFunction(void)
{
printf("This is added just for testing purpose");
}
library build command
gcc -shared -o libtest.so -Wall -Werror -fpic -I. visibility.cpp
test.cpp
#include <stdio.h>
#include <gcrypt.h>
#include <assert.h>
#include "visibility.h"
extern void MyFunction();
int main()
{
MyFunction();
return 0;
}
exe build command
gcc test.cpp -o test -I. -L. -ltest -lstdc++
My gcc version is 4.4.7
And of-course I did not try and install the lib under /usr/local/lib but kept it local for quick testing.
I have files foo.c bar.c and baz.c, plus wrapper code myfn.c defining a function myfn() that uses code and data from those other files.
I would like to create something like an object file or archive, myfn.o or libmyfn.a, so that myfn() can be made available to other projects without also exporting a load of symbols from {foo,bar,baz}.o as well.
What's the right way to do that in Linux/gcc? Thanks.
Update: I've found one way of doing it. I should've emphasised originally that this was about static archives, not DSOs. Anyway, the recipe:
#define PUBLIC __attribute__ ((visibility("default"))) then mark myfn() as PUBLIC in myfn.c. Don't mark anything else PUBLIC.
Compile objects with gcc -c foo.c bar.c baz.c myfn.c -fvisibility=hidden, which marks everything as hidden except for myfn().
Create a convenience archive using ld's partial-linking switch: ld -r foo.o bar.o baz.o myfn.o -o libmyfn.a
Localise everything that wasn't PUBLIC like so: objcopy --localize-hidden libmyfn.a
Now nm says myfn is the only global symbol in libmyfn.a and subsequent linking into other programs works just fine: gcc -o main main.c -L. -lmyfn (here, the program calls myfn(); if it tried to call foo() then compilation would fail).
If I use ar instead of ld -r in step 3 then compilation fails in step 5: I guess ar hasn't linked foo etc to myfn, and no longer can once those functions are localised, whereas ld -r resolves the link before it gets localised-away.
I'd welcome any response that confirms this is the "right" way, or describes a slicker way of achieving the same.
Unfortunately, C linkage for globals is all-or-nothing, in the sense that the globals of all modules would be available in libmyfn.a's final list of external symbols.
gcc tool chain offers an extension that lets you hide symbols from outside users, while making them available to other translation units in your library:
foo.h:
void foo();
foo.c:
void foo() __attribute__ ((visibility ("hidden")));
myfn.h:
void myfn();
myfn.c:
#include <stdio.h>
#include "foo.h"
void myfn() {
printf("calling foo...\n");
foo();
printf("calling foo again...\n");
foo();
}
For portability, you would probably benefit from making a macro for __attribute__ ((visibility ("hidden"))), and placing it in a conditional compilation block conditioned on gcc.
In addition, Linux offers a utility called strip, which lets you remove some of the symbols from compiled object files. Options -N and -K let you identify individual symbols that you want to keep or remove.
Start with this to build a static library
gcc -c -O2 foo.c bar.c baz.c myfn.c
ar av libmyfunctions.a foo.o bar.o baz.o myfn.o
Compile and link with other programs like:
gcc -O2 program.c -lmyfunctions -o myprogram
Now your libmyfunctions.a will ultimately have extra stuff from the source that isn't required by the code in myfn.c But the linker should do a reasonable job of removing this when it creates the final program.
Suppose myfn.c has function myfun() which you want to use in other three files foo.c, bar.c & baz.c
Now create a shared library from code in myfn.c viz libmyf.a
Use this function call myfun() in other three files. Declare function as extern in these files. Now you can create object code of these thee files and link the libmyf.a at linking phase.
Refer to following link for using shared libraries.
http://www.cprogramming.com/tutorial/shared-libraries-linux-gcc.html
Let's say I have two files:
// shared.c (will be compiled to 'shared.so')
#include <stdio.h>
int f() { printf("hello\n"); }
and
// exe.c (will be compiled to 'exe')
#include <stdio.h>
int f();
int main() {
int i;
scanf("%d", &i);
if (i == 5) f();
}
I compile both files as following:
gcc -shared shared.c -o libshared.so
gcc exe.c -o exe -lshared -L.
When I run exe and type 5, it will call f and then exit. However, if I delete f from shared.c and recompile it I will get a runtime symbol lookup error only if I type 5. Is there a way that I can check that exe has all its symbols that will work independent of user input in this case? Preferrably without running it.
You can use ldd -r exe command to list the shared library dependencies.
Here is my output for your example without the f function:
$ LD_LIBRARY_PATH=. ldd -r ./exe
linux-vdso.so.1 (0x00007ffcfa7c3000)
libshared.so => ./libshared.so (0x00007f303a02e000)
libc.so.6 => /lib64/libc.so.6 (0x0000003e26c00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003e26400000)
undefined symbol: f (./exe)
(Don't mind the LD_LIBRARY_PATH=. part. It is used to tell to look for shared libraries in the current directory)
#tohava
When you compile the executable and link it with the shared object, ld (linker) checks if all referenced symbols are available in the list of shared objects your executable is dependent on and will throw an error if any symbol was unresolved.
So, I am not sure how you managed to get a runtime error when you removed f() from the shared library and rebuilt the executable. (I did the exercise myself and got the linker error).
libp2.c
#include <stdio.h>
void pixman()
{
printf("pixman in libp1\n");
}
libc2.c
#include <stdio.h>
void pixman();
void cairo()
{
printf("cairo2\n");
pixman();
}
main.c
#include <stdio.h>
#include <dlfcn.h>
void pixman()
{
printf("pixman in main\n");
}
int main()
{
pixman();
void* handle=NULL;
void (*callfun)();
handle=dlopen("/home/zpeng/test/so_test/libc2.so",RTLD_LAZY);
callfun = (void(*)())dlsym(handle, "cairo");
callfun();
...
}
compile
gcc -c libp2.c -fPIC -olibp2.o
rm libp2.a
ar -rs libp2.a libp2.o
gcc -shared -fPIC libc2.c ./libp2.a -o libc2.so
gcc main.c -ldl -L. -g
the result:
pixman in main
cairo2
pixman: libp2
why the last is not "pixman in main"?
I see the symbols processing(LD_DEBUG=symbols), it begins with :
21180: symbol=pixman; lookup in file=./a.out
21180: symbol=pixman; lookup in file=/lib64/libdl.so.2
21180: symbol=pixman; lookup in file=/lib64/tls/libc.so.6
21180: symbol=pixman; lookup in file=/lib64/ld-linux-x86-64.so.2
21180: symbol=pixman; lookup in file=/home/zpeng/test/so_test/libc2.so
if I add -lc2 or -rdynamic to gcc main cmd , it will generate:
pixman in main
cairo2
pixman in main
My questions:
why lookup symbol in a.out but not get the result and continue to search libc2.so when without -rdynamic and -lc2 ?
Why the last is not "pixman in main"?
That's because shared libraries have their own global offset table or GOT. When you use the cairo function in libc2.so, the pixman function that will be called is the same function that was resolved when compiling the .so file in the first place.
That is:
# creates object file only -- contains first pixman implementation
gcc -c libp2.c -fPIC -olibp2.o
# just turns the object file into an archive
ar -rs libp2.a libp2.o
# creates the .so file -- all symbols in libc2.c are resolved here
# and you passed in the .a file for that purpose. The .a file containing the
# first pixman implementation gets put in libc2.so.
gcc -shared -fPIC libc2.c ./libp2.a -o libc2.so
After this, anyone using libc2.so will get the copy stored in libc2.so. The lookup order you post is for a.out I believe and it's right. It looks for pixman in a.out, then libc2.so, and so on.
Why lookup symbol in a.out but not get the result and continue to search libc2.so when without -rdynamic and -lc2?
The rdynamic option loads ALL symbols to the dynamic symbol table -- not just the ones it thinks are used (lc2 has the same effect). When you load all those symbols you have a conflict -- the pixman function. The main.c implementation is used in this case. As others have pointed out, this will probably generate a warning.
You need to compile the sources that get archived into the .a file with -fvisibility=hidden, to indicate that, although they are global functions, they are not meant to be used outside the resulting library but are instead meant to resolve symbols inside the library. That will cause the symbols in the .a file to appear with the qualifier " t " in nm -a instead of " T ", which is used for symbols available to other libraries.
It just auto binded to LOCAL symbol,
Since there not __attribute__((visibility("default"))) explicit in libp2.c, the compiler auto bind this function calling to LOCAL .symtab, instead of .dynsym
appendix1: more about ELF header: readelf -s xxx.lib
appendix2: keyword of ld argument -Bsymbolic-functions
Let me explain the context first. I have a header with a function declaration, a .c program with the body of the function, and the main program.
foo.h
#ifndef _FOO_H_
#define _FOO_H_
void foo();
#endif
foo.c
#include<stdio.h>
#include "include/foo.h"
void foo()
{
printf("Hello\n");
}
mainer.c
#include <stdio.h>
#include "include/foo.h"
int main()
{ foo();
return 0;
}
For the purpose of this program, both the header and the static library need to be in separate folders, so the header is on /include/foo.h and the static library generated will be on /lib/libfoo.a, and both .c programs on the main directory. The idea is to generate the object program, then the static library, then linking the static library to create the executable, and finally executing the program.
I have no problem in both creating the object program and the static library.
$ gcc -c foo.c -o foo.o
$ ar rcs lib/libfoo.a foo.o
But when I try to link the static library...
$ gcc -static mainer.c -L. -lfoo -o mainfoo
It gaves to me an error, claiming the static library can't be found
/usr/bin/ld: cannot find -lfoo
collect2: ld returned 1 exit status
It's strange, considering I asked before how to work with static libraries and headers on separate folders and in this case the static libraries were found. Any idea what I'm doing wrong?
Change -L. to -Llib as it looks like you create the .a file there.
Basically the linker is telling you that it cannot find the library foo. It normally searches in the default library directories + any you give it with the -L option. You're telling it to look in the current directory, but not in lib where libfoo.a is located, which is why it can't find it. You need to change -L. to -Llib.
I am not completely sure that I understand your directory structure, but maybe what you need is this:
gcc -static mainer.c -L./lib -lfoo -o mainfoo