I have one base library written in C. And other external extensions/definitions files. With a simple search tool the base library (when is used) can search in a directory for extensions libraries. Without telling the compiler which files it are. Then the C program can call a function in that external file.
So i want to run a function in a external file. I can choose which file I want to run the function from at running time. (Ex, I enter that the file is located at ./external_file at running time) (NOT COMPILE TIME!). Is this possible?
I need to read a return from the function, I need to wait until the function is finished, I need to call the function with arguments and it must be possible that the program at run time can choose the file and function.
Is this possible? Yes, program files does contains function names (with names). Only I need to know HOW do I call that function in a external file at run time.
Use dynamic libraries - most OS's provide support for loading a library given a name and getting the address of the function to call.
In windows see LoadLibrary and ProcAddress. In linux see dlopen and dlsym.
You can use the dlopen function to load a shared library (i.e. a .so file) at runtime, then use dlsym to get the functions to call.
For example, suppose you had the following library file:
mylib.c:
#include <stdio.h>
void f1(int a)
{
printf("in f1, a=%d\n", a);
}
int f2(char *s)
{
printf("in f2, s=%s\n", s);
return *s;
}
Then you compile it to a shared library:
gcc -g -Wall -Wextra -fPIC -c mylib.c
gcc -g -Wall -Wextra -fPIC -o libmylib.so mylib.o
You can call these functions as follows:
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
int main()
{
// get a handle to the shared library
void *handle = dlopen("./libmylib.so", RTLD_LAZY);
if (!handle) {
printf("dlopen failed: %s\n", dlerror());
exit(1);
}
// get a pointer to the function to call
void (*myf1)(int) = dlsym(handle, "f1");
if (!myf1) {
printf("dlsym 1 failed: %s\n", dlerror());
exit(1);
}
myf1(4);
// get a pointer to another function to call
int (*myf2)(char *) = dlsym(handle, "f2");
if (!myf2) {
printf("dlsym 2 failed: %s\n", dlerror());
exit(1);
}
printf("rval=%d\n", myf2("hello"));
dlclose(handle);
return 0;
}
Note that you need to know at compile time what the function signatures look like so you can call them correctly.
Related
So everyone probably knows that glibc's /lib/libc.so.6 can be executed in the shell like a normal executable in which cases it prints its version information and exits. This is done via defining an entry point in the .so. For some cases it could be interesting to use this for other projects too. Unfortunately, the low-level entry point you can set by ld's -e option is a bit too low-level: the dynamic loader is not available so you cannot call any proper library functions. glibc for this reason implements the write() system call via a naked system call in this entry point.
My question now is, can anyone think of a nice way how one could bootstrap a full dynamic linker from that entry point so that one could access functions from other .so's?
Update 2: see Andrew G Morgan's slightly more complicated solution which does work for any GLIBC (that solution is also used in libc.so.6 itself (since forever), which is why you can run it as ./libc.so.6 (it prints version info when invoked that way)).
Update 1: this no longer works with newer GLIBC versions:
./a.out: error while loading shared libraries: ./pie.so: cannot dynamically load position-independent executable
Original answer from 2009:
Building your shared library with -pie option appears to give you everything you want:
/* pie.c */
#include <stdio.h>
int foo()
{
printf("in %s %s:%d\n", __func__, __FILE__, __LINE__);
return 42;
}
int main()
{
printf("in %s %s:%d\n", __func__, __FILE__, __LINE__);
return foo();
}
/* main.c */
#include <stdio.h>
extern int foo(void);
int main()
{
printf("in %s %s:%d\n", __func__, __FILE__, __LINE__);
return foo();
}
$ gcc -fPIC -pie -o pie.so pie.c -Wl,-E
$ gcc main.c ./pie.so
$ ./pie.so
in main pie.c:9
in foo pie.c:4
$ ./a.out
in main main.c:6
in foo pie.c:4
$
P.S. glibc implements write(3) via system call because it doesn't have anywhere else to call (it is the lowest level already). This has nothing to do with being able to execute libc.so.6.
I have been looking to add support for this to pam_cap.so, and found this question. As #EmployedRussian notes in a follow-up to their own post, the accepted answer stopped working at some point. It took a while to figure out how to make this work again, so here is a worked example.
This worked example involves 5 files to show how things work with some corresponding tests.
First, consider this trivial program (call it empty.c):
int main(int argc, char **argv) { return 0; }
Compiling it, we can see how it resolves the dynamic symbols on my system as follows:
$ gcc -o empty empty.c
$ objcopy --dump-section .interp=/dev/stdout empty ; echo
/lib64/ld-linux-x86-64.so.2
$ DL_LOADER=/lib64/ld-linux-x86-64.so.2
That last line sets a shell variable for use later.
Here are the two files that build my example shared library:
/* multi.h */
void multi_main(void);
void multi(const char *caller);
and
/* multi.c */
#include <stdio.h>
#include <stdlib.h>
#include "multi.h"
void multi(const char *caller) {
printf("called from %s\n", caller);
}
__attribute__((force_align_arg_pointer))
void multi_main(void) {
multi(__FILE__);
exit(42);
}
const char dl_loader[] __attribute__((section(".interp"))) =
DL_LOADER ;
(Update 2021-11-13: The forced alignment is to help __i386__ code be SSE compatible - without it we get hard to debug glibc SIGSEGV crashes.)
We can compile and run it as follows:
$ gcc -fPIC -shared -o multi.so -DDL_LOADER="\"${DL_LOADER}\"" multi.c -Wl,-e,multi_main
$ ./multi.so
called from multi.c
$ echo $?
42
So, this is a .so that can be executed as a stand alone binary. Next, we validate that it can be loaded as shared object.
/* opener.c */
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
void *handle = dlopen("./multi.so", RTLD_NOW);
if (handle == NULL) {
perror("no multi.so load");
exit(1);
}
void (*multi)(const char *) = dlsym(handle, "multi");
multi(__FILE__);
}
That is we dynamically load the shared-object and run a function from it:
$ gcc -o opener opener.c -ldl
$ ./opener
called from opener.c
Finally, we link against this shared object:
/* main.c */
#include "multi.h"
int main(int argc, char **argv) {
multi(__FILE__);
}
Where we compile and run it as follows:
$ gcc main.c -o main multi.so
$ LD_LIBRARY_PATH=./ ./main
called from main.c
(Note, because multi.so isn't in a standard system library location, we need to override where the runtime looks for the shared object file with the LD_LIBRARY_PATH environment variable.)
I suppose you'd have your ld -e point to an entry point which would then use the dlopen() family of functions to find and bootstrap the rest of the dynamic linker. Of course you'd have to ensure that dlopen() itself was either statically linked or you might have to implement enough of your own linker stub to get at it (using system call interfaces such as mmap() just as libc itself is doing.
None of that sounds "nice" to me. In fact just the thought of reading the glibc sources (and the ld-linux source code, as one example) enough to assess the size of the job sounds pretty hoary to me. It might also be a portability nightmare. There may be major differences between how Linux implements ld-linux and how the linkages are done under OpenSolaris, FreeBSD, and so on. (I don't know).
I want to load a shared library with dlopen and have the symbols in it available without having to individually grab function pointers to them with dlsym. The man page says that the RTLD_DEEPBIND flag will place lookup of symbols in the library ahead of global scope, but evidently this does not mean it overrides existing symbols because this does not work. Consider this example:
main.c:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
int is_loaded(){return 0;}
int main(){
void *h = dlopen("./libimplementation.so", RTLD_NOW | RTLD_DEEPBIND);
if(!h){
printf("Could not load implementation: %s\n", dlerror());
return 1;
}
puts(is_loaded() ? "Implementation loaded" : "Implementation not loaded");
dlclose(h);
}
implementation.c:
int is_loaded(){return 1;}
Makefile:
all: main libimplementation.so
main: main.c
gcc -Wall -std=c99 -o $# $^ -ldl
lib%.so: %.c
gcc -Wall -std=c99 -o $# $^ -shared
clean:
-rm main *.so
When I build and run with make and ./main, I expect the test() function from libimplementation.so to override the test() function from main but it doesn't. I know I could also move all of the code in main() into another shared library run and then have main() dlopen libimplementation.so with RTLD_GLOBAL and then have librun.so refer to the symbols from libimplementation.so without having them defined so it loads them:
modified main.c:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
int main(){
void *impl_h = dlopen("./libimplementation.so", RTLD_LAZY | RTLD_GLOBAL);
if(!impl_h){
printf("Could not load implementation: %s\n", dlerror());
return 1;
}
void *run_h = dlopen("./librun.so", RTLD_LAZY);
if(!run_h){
printf("Could not load run: %s\n", dlerror());
dlclose(impl_h);
return 1;
}
void (*run)(void);
*(void**)&run = dlsym(run_h, "run");
if(!*(void**)&run){
printf("Could not find entry point in run: %s\n", dlerror());
dlclose(impl_h);
dlclose(run_h);
return 1;
}
run();
dlclose(impl_h);
dlclose(run_h);
}
run.c:
#include <stdio.h>
int is_loaded(void);
void run(void){
puts(is_loaded() ? "Implementation loaded" : "Implementation not loaded");
}
and the Makefile gets librun.so added as a prerequisite for all.
Is there a way to get the symbols from the shared library available all at once without dlsym or putting the actual code in another shared library like with librun.so?
There is fundamentally no way to do what you're asking for. Imagine the main program had something like:
static char *myptr = array_in_lib1;
Later, at the time you dlopen, myptr has some other value. Has the program just changed the variable to point to a different object? Or has it been incremented to point to some element later in the array - in which case, would you want it adjusted to account for the redefinition of array_in_lib1 with a new definition from the newly-opened library? Or is it just a random integer cast to char *? Deciding how to treat it is impossible without understanding programmer intent and full process history of how it arrived in the current state.
The above is a particulrly egregious sort of example I've constructed, but the idea of symbols changing definition at runtime is fundamentally inconsistent in all sorts of ways. Even RTLD_DEEPBIND, in what it already does, is arguably inconsitent and buggy. Whatever you're trying to do, you should find another way to do it.
I am using dlsym to look up symbols in my program, but it always returns NULL, which I am not expecting. According to the manpage, dlsym may return NULL if there was an error somehow, or if the symbol indeed is NULL. In my case, I am getting an error. I will show you the MCVE I have made this evening.
Here is the contents of instr.c:
#include <stdio.h>
void * testing(int i) {
printf("You called testing(%d)\n", i);
return 0;
}
A very simple thing containing only an unremarkable example function.
Here is the contents of test.c:
#include <dlfcn.h>
#include <stdlib.h>
#include <stdio.h>
typedef void * (*dltest)(int);
int main(int argc, char ** argv) {
/* Declare and set a pointer to a function in the executable */
void * handle = dlopen(NULL, RTLD_NOW | RTLD_GLOBAL);
dlerror();
dltest fn = dlsym(handle, "testing");
if(fn == NULL) {
printf("%s\n", dlerror());
dlclose(handle);
return 1;
}
dlclose(handle);
return 0;
}
As I step through the code with the debugger, I see the dlopen is returning a handle. According to the manpage, If filename is NULL, then the returned handle is for the main program. So if I link a symbol called testing into the main program, dlsym should find it, right?
Here is the way that I am compiling and linking the program:
all: test
instr.o: instr.c
gcc -ggdb -Wall -c instr.c
test.o: test.c
gcc -ggdb -Wall -c test.c
test: test.o instr.o
gcc -ldl -o test test.o instr.o
clean:
rm -f *.o test
And when I build this program, and then do objdump -t test | grep testing, I see that the symbol testing is indeed there:
08048632 g F .text 00000020 testing
Yet the output of my program is the error:
./test: undefined symbol: testing
I am not sure what I am doing wrong. I would appreciate if someone could shed some light on this problem.
I don't think you can do that, dlsym works on exported symbols. Because you're doing dlsym on NULL (current image), even though the symbols are present in the executable ELF image, they're not exported (since it's not a shared library).
Why not call it directly and let the linker take care of it? There's no point in using dlsym to get symbols from the same image as your dlsym call. If your testing symbol was in a shared library that you either linked against or loaded using dlopen then you would be able to retrieve it.
I believe there's also a way of exporting symbols when building executables (-Wl,--export-dynamic as mentioned in a comment by Brandon) but I'm not sure why you'd want to do that.
I faced the similar issue in my code.
I did the following to export symbols
#ifndef EXPORT_API
#define EXPORT_API __attribute__ ((visibility("default")))
#endif
Now for each of the function definition I used the above attribute.
For example the earlier code was
int func() { printf(" I am a func %s ", __FUNCTION__ ) ;
I changed to
EXPORT_API int func() { printf(" I am a func %s ", __FUNCTION__ ) ;
Now it works.
dlsym gives no issues after this.
Hope this works for you as well.
Language: C
Operating System: Red Hat EL
Starting with a "for instance":
Assume I have two libraries: libJUMP.so and libSIT.so.
JUMP contains the function jump() and similarly SIT contains the function sit()
I have an application that I want to provide to different people; they can either get the jump() feature, the sit() feature, or both. However, I would like to NOT use #ifdef if at all possible.
Header for libJUMP.so:
#ifndef JUMP_H_
#define JUMP_H_
#define JUMP_ENABLED
void jump();
#endif /* JUMP_H_ */
Header for libSIT.so:
#ifndef SIT_H_
#define SIT_H_
#define SIT_ENABLED
void sit();
#endif /* SIT_H_ */
I have an application:
#include "jump.h"
#include "sit.h"
int main()
{
// #ifdef JUMP_ENABLED
jump();
// #endif /* JUMP_ENABLED */
// #ifdef SIT_ENABLED
sit();
// #endif /* SIT_ENABLED */
}
So:
Is there a way to do this without using #ifdef? Is there a better way at all?
I have heard we could do this by compiling with both SO libraries, and if one is missing when I run the application on the target system, it could just exclude the feature automatically (using some combination of dlopen() and dlsym()?) Any easy examples, if this is indeed correct? An example with my code from above, if possible :D?
If this is a stupid question, or just not possible, please feel free to tell me so. If there is a similar question that this would be considered a duplicate of, let me know and I will delete this post.
Consider these three files. First, jump.c:
#include <stdio.h>
int jump(const double height)
{
fflush(stdout);
fprintf(stderr, "Jumping %.3g meters.\n", height);
fflush(stderr);
return 0;
}
Second, sit.c:
#include <stdio.h>
int sit(void)
{
fflush(stdout);
fprintf(stderr, "Sitting down.\n");
fflush(stderr);
return 0;
}
Third, example.c to use one or both of the above, depending on whether they (as libjump.so or libsit.so, respectively) exist in the current working directory:
#include <stdio.h>
#include <dlfcn.h>
static const char *jump_lib_path = "./libjump.so";
static int (*jump)(const double) = NULL;
static const char *sit_lib_path = "./libsit.so";
static int (*sit)(void) = NULL;
static void load_dynamic_libraries(void)
{
void *handle;
handle = dlopen(jump_lib_path, RTLD_NOW | RTLD_LOCAL);
if (handle) {
jump = dlsym(handle, "jump");
/* If no jump symbol, we don't need the library at all. */
if (!jump)
dlclose(handle);
}
handle = dlopen(sit_lib_path, RTLD_NOW | RTLD_LOCAL);
if (handle) {
sit = dlsym(handle, "sit");
/* If no sit symbol, the library is useless. */
if (!sit)
dlclose(handle);
}
}
int main(void)
{
int retval;
load_dynamic_libraries();
if (jump) {
printf("Calling 'jump(2.0)':\n");
retval = jump(2.0);
printf("Returned %d.\n\n", retval);
} else
printf("'jump()' is not available.\n\n");
if (sit) {
printf("Calling 'sit()':\n");
retval = sit();
printf("Returned %d.\n\n", retval);
} else
printf("'sit()' is not available.\n\n");
return 0;
}
Let's first compile and run the example program:
gcc -Wall -O2 example.c -ldl -o example
./example
The program outputs that neither jump() or sit() are available. Let's compile jump.c into a dynamic library, libjump.so, and then run the example again:
gcc -Wall -O2 -fPIC -shared jump.c -Wl,-soname,libjump.so -o libjump.so
./example
Now, the jump() function works. Let's compile sit.c, too, and run the example a final time:
gcc -Wall -O2 -fPIC -shared jump.c -Wl,-soname,libsit.so -o libsit.so
./example
Here, both functions get called, and everything just works.
In example.c, jump and sit are function pointers. We initialize them to NULL, so that we can use if (jump) to check if jump points to a valid function.
The load_dynamic_libraries() function uses dlopen() and dlsym() to obtain the function pointers. Note that if the dynamic library is opened successfully, and the necessary symbol is found, we do not dlclose() it because we want to keep the dynamic library in memory. (We only dlclose() it if it looks like it is not the kind of library we want.)
If you want to avoid the if (jump) and if (sit) clauses, you can use stubs like
int unsupported_jump(const double height)
{
return ENOTSUP;
}
int unsupported_sit(void)
{
return ENOTSUP;
}
and at the end of load_dynamic_libraries(), divert the functions to the stubs instead of NULL pointers, i.e.
if (!jump)
jump = unsupported_jump;
if (!sit)
sit = unsupported_sit;
Note that function-like interfaces are easiest to use, because the function pointer acts as the effective prototype. If you need objects, I recommend using getter functions. Objects do work just fine, as long as you remember that dlsym() returns a pointer to the object; using a getter function, that is explicit in the getter function pointer type.
Plug-in interfaces commonly have a single function (say, int properties(struct plugin *const props, const int version)), which is used to populate a structure of function and object pointers. The application supplies the version of the structure it uses, and the plug-in function returns either success or failure, depending on whether it can populate the structure to accommodate that version.
As plug-ins are typically stored in a single directory (/usr/lib/yourapp/plugins/ is very common), you can trivially load all plugins by using opendir() and readdir() to scan the file names in the plug-in directory one by one, dlopen()ing each one, obtaining the properties() function pointer, and calling it to see what kinds of services the plugin provides; typically creating an array or a linked list of the plugin structures.
All of this is very, very simple and straightforward in Linux, as you can see. If you want a specific plug-in functionality example, I recommend you pose that as a separate question, with more details on what kind of functionality the interface should expose -- the exact data structures and function prototypes do depend very much on what kind of application we have at hand.
Questions? Comments?
I have a shared library that I implemented and want the .so to call a function that's implemented in the main program which loads the library.
Let's say I have main.c (executable) which contains:
void inmain_function(void*);
dlopen("libmy.so");
In the my.c (the code for the libmy.so) I want to call inmain_function:
inmain_function(NULL);
How can the shared library call inmain_function regardless the fact inmain_function is defined in the main program.
Note: I want to call a symbol in main.c from my.c not vice versa which is the common usage.
You have two options, from which you can choose:
Option 1: export all symbols from your executable.
This is simple option, just when building executable, add a flag -Wl,--export-dynamic. This would make all functions available to library calls.
Option 2: create an export symbol file with list of functions, and use -Wl,--dynamic-list=exported.txt. This requires some maintenance, but more accurate.
To demonstrate: simple executable and dynamically loaded library.
#include <stdio.h>
#include <dlfcn.h>
void exported_callback() /*< Function we want to export */
{
printf("Hello from callback!\n");
}
void unexported_callback() /*< Function we don't want to export */
{
printf("Hello from unexported callback!\n");
}
typedef void (*lib_func)();
int call_library()
{
void *handle = NULL;
lib_func func = NULL;
handle = dlopen("./libprog.so", RTLD_NOW | RTLD_GLOBAL);
if (handle == NULL)
{
fprintf(stderr, "Unable to open lib: %s\n", dlerror());
return -1;
}
func = dlsym(handle, "library_function");
if (func == NULL) {
fprintf(stderr, "Unable to get symbol\n");
return -1;
}
func();
return 0;
}
int main(int argc, const char *argv[])
{
printf("Hello from main!\n");
call_library();
return 0;
}
Library code (lib.c):
#include <stdio.h>
int exported_callback();
int library_function()
{
printf("Hello from library!\n");
exported_callback();
/* unexported_callback(); */ /*< This one will not be exported in the second case */
return 0;
}
So, first build the library (this step doesn't differ):
gcc -shared -fPIC lib.c -o libprog.so
Now build executable with all symbols exported:
gcc -Wl,--export-dynamic main.c -o prog.exe -ldl
Run example:
$ ./prog.exe
Hello from main!
Hello from library!
Hello from callback!
Symbols exported:
$ objdump -e prog.exe -T | grep callback
00000000004009f4 g DF .text 0000000000000015 Base exported_callback
0000000000400a09 g DF .text 0000000000000015 Base unexported_callback
Now with the exported list (exported.txt):
{
extern "C"
{
exported_callback;
};
};
Build & check visible symbols:
$ gcc -Wl,--dynamic-list=./exported.txt main.c -o prog.exe -ldl
$ objdump -e prog.exe -T | grep callback
0000000000400774 g DF .text 0000000000000015 Base exported_callback
You'll need make a register function in your .so so that the executable can give a function pointer to your .so for it's later used.
Like this:
void in_main_func () {
// this is the function that need to be called from a .so
}
void (*register_function)(void(*)());
void *handle = dlopen("libmylib.so");
register_function = dlsym(handle, "register_function");
register_function(in_main_func);
the register_function needs to store the function pointer in a variable in the .so where the other function in the .so can find it.
Your mylib.c would the need to look something like this:
void (*callback)() = NULL;
void register_function( void (*in_main_func)())
{
callback = in_main_func;
}
void function_needing_callback()
{
callback();
}
Put your main function's prototype in a .h file and include it in both your main and dynamic library code.
With GCC, simply compile your main program with the -rdynamic flag.
Once loaded, your library will be able to call the function from the main program.
A little further explanation is that once compiled, your dynamic library will have an undefined symbol in it for the function that is in the main code. Upon having your main app load the library, the symbol will be resolved by the main program's symbol table. I've used the above pattern numerous times and it works like a charm.
The following can be used to load a dynamic library and call it from the loading call (in case somebody came here after looking for how to load and call a function in an .so library):
void* func_handle = dlopen ("my.so", RTLD_LAZY); /* open a handle to your library */
void (*ptr)() = dlsym (func_handle, "my_function"); /* get the address of the function you want to call */
ptr(); /* call it */
dlclose (func_handle); /* close the handle */
Don't forget to put #include <dlfcn.h> and link with the –ldl option.
You might also want to add some logic that checks if NULL is returned. If it is the case you can call dlerror and it should give you some meaningful messages describing the problem.
Other posters have however provided more suitable answers for your problem.