I have a lib, called "test.so" that uses functions from two libraries that reference each other. If I call the functions of test.so within a C program, it works just fine, so I assume there's no error in C functions.
However, when I call it from Lua, it throws a Seg Fault error due to an "undefined symbol". The problem is, the symbol is defined, I can see it when I run nm test.so.
Reading the thread below
Lua: C++ modules can't reference eachother, undefined symbol
I tried to creating a new module that would load test.so using dlopen, as described by Rena.
#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>
#include <stdio.h>
#include <dlfcn.h>
int luaopen_drmaa(lua_State *L){
printf("Chamou luaopen_test\n");
if( dlopen("/home/erica/Downloads/test.so", RTLD_NOW | RTLD_GLOBAL) == NULL)
printf("%s\n", dlerror());
else
printf("Chamou dlopen e teve retorno nao null\n");
return 0;
}
I compiled it:
gcc -g drmaa_lib.c -shared -fpic -I /usr/include/lua5.1 -I /usr/local/include/ -L /usr/local/lib/m -llua5.1 -o drmaa.so
But when I run, I get:
$ lua5.1
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio
> require "drmaa"
Chamou luaopen_test
Chamou dlopen e teve retorno nao null
> submitJob()
stdin:1: attempt to call global 'submitJob' (a nil value)
stack traceback:
stdin:1: in main chunk
[C]: ?
Then I tried inserting in main function
luaopen_test(L);
and above the main function
extern int luaopen_test(lua_State *L);
to actually open the lib, but I get this error:
$ lua5.1
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio
> require "drmaa"
error loading module 'drmaa' from file './drmaa.so':
./drmaa.so: undefined symbol: luaopen_test
stack traceback:
[C]: ?
[C]: in function 'require'
stdin:1: in main chunk
[C]: ?
The author of the referenced question doesn't show details of the development of the solution.
Does anyone has a clue of what may make it work?
Thanks in advance.
I was obviously doing it wrong, to get the function that actually opens the library I should have used ldsym function. The new code for drmaa lib (the one that loads test.so) is
#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>
#include <stdio.h>
#include <dlfcn.h>
typedef void Register(lua_State*);
int luaopen_drmaa(lua_State *L){
void* lib = dlopen("/home/erica/Downloads/test.so", RTLD_NOW | RTLD_GLOBAL);
if(!lib)
{
printf("%s\n", dlerror());
return 1;
}
Register* loadFunc = (Register*)dlsym(lib, "luaopen_test");
if(!loadFunc)
{
printf("%s\n", dlerror());
return 1;
}
loadFunc(L);
return 0;
}
This answer was base in some code of this thread:
Lua shared object loading with C++ segfaults
Related
So everyone probably knows that glibc's /lib/libc.so.6 can be executed in the shell like a normal executable in which cases it prints its version information and exits. This is done via defining an entry point in the .so. For some cases it could be interesting to use this for other projects too. Unfortunately, the low-level entry point you can set by ld's -e option is a bit too low-level: the dynamic loader is not available so you cannot call any proper library functions. glibc for this reason implements the write() system call via a naked system call in this entry point.
My question now is, can anyone think of a nice way how one could bootstrap a full dynamic linker from that entry point so that one could access functions from other .so's?
Update 2: see Andrew G Morgan's slightly more complicated solution which does work for any GLIBC (that solution is also used in libc.so.6 itself (since forever), which is why you can run it as ./libc.so.6 (it prints version info when invoked that way)).
Update 1: this no longer works with newer GLIBC versions:
./a.out: error while loading shared libraries: ./pie.so: cannot dynamically load position-independent executable
Original answer from 2009:
Building your shared library with -pie option appears to give you everything you want:
/* pie.c */
#include <stdio.h>
int foo()
{
printf("in %s %s:%d\n", __func__, __FILE__, __LINE__);
return 42;
}
int main()
{
printf("in %s %s:%d\n", __func__, __FILE__, __LINE__);
return foo();
}
/* main.c */
#include <stdio.h>
extern int foo(void);
int main()
{
printf("in %s %s:%d\n", __func__, __FILE__, __LINE__);
return foo();
}
$ gcc -fPIC -pie -o pie.so pie.c -Wl,-E
$ gcc main.c ./pie.so
$ ./pie.so
in main pie.c:9
in foo pie.c:4
$ ./a.out
in main main.c:6
in foo pie.c:4
$
P.S. glibc implements write(3) via system call because it doesn't have anywhere else to call (it is the lowest level already). This has nothing to do with being able to execute libc.so.6.
I have been looking to add support for this to pam_cap.so, and found this question. As #EmployedRussian notes in a follow-up to their own post, the accepted answer stopped working at some point. It took a while to figure out how to make this work again, so here is a worked example.
This worked example involves 5 files to show how things work with some corresponding tests.
First, consider this trivial program (call it empty.c):
int main(int argc, char **argv) { return 0; }
Compiling it, we can see how it resolves the dynamic symbols on my system as follows:
$ gcc -o empty empty.c
$ objcopy --dump-section .interp=/dev/stdout empty ; echo
/lib64/ld-linux-x86-64.so.2
$ DL_LOADER=/lib64/ld-linux-x86-64.so.2
That last line sets a shell variable for use later.
Here are the two files that build my example shared library:
/* multi.h */
void multi_main(void);
void multi(const char *caller);
and
/* multi.c */
#include <stdio.h>
#include <stdlib.h>
#include "multi.h"
void multi(const char *caller) {
printf("called from %s\n", caller);
}
__attribute__((force_align_arg_pointer))
void multi_main(void) {
multi(__FILE__);
exit(42);
}
const char dl_loader[] __attribute__((section(".interp"))) =
DL_LOADER ;
(Update 2021-11-13: The forced alignment is to help __i386__ code be SSE compatible - without it we get hard to debug glibc SIGSEGV crashes.)
We can compile and run it as follows:
$ gcc -fPIC -shared -o multi.so -DDL_LOADER="\"${DL_LOADER}\"" multi.c -Wl,-e,multi_main
$ ./multi.so
called from multi.c
$ echo $?
42
So, this is a .so that can be executed as a stand alone binary. Next, we validate that it can be loaded as shared object.
/* opener.c */
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
void *handle = dlopen("./multi.so", RTLD_NOW);
if (handle == NULL) {
perror("no multi.so load");
exit(1);
}
void (*multi)(const char *) = dlsym(handle, "multi");
multi(__FILE__);
}
That is we dynamically load the shared-object and run a function from it:
$ gcc -o opener opener.c -ldl
$ ./opener
called from opener.c
Finally, we link against this shared object:
/* main.c */
#include "multi.h"
int main(int argc, char **argv) {
multi(__FILE__);
}
Where we compile and run it as follows:
$ gcc main.c -o main multi.so
$ LD_LIBRARY_PATH=./ ./main
called from main.c
(Note, because multi.so isn't in a standard system library location, we need to override where the runtime looks for the shared object file with the LD_LIBRARY_PATH environment variable.)
I suppose you'd have your ld -e point to an entry point which would then use the dlopen() family of functions to find and bootstrap the rest of the dynamic linker. Of course you'd have to ensure that dlopen() itself was either statically linked or you might have to implement enough of your own linker stub to get at it (using system call interfaces such as mmap() just as libc itself is doing.
None of that sounds "nice" to me. In fact just the thought of reading the glibc sources (and the ld-linux source code, as one example) enough to assess the size of the job sounds pretty hoary to me. It might also be a portability nightmare. There may be major differences between how Linux implements ld-linux and how the linkages are done under OpenSolaris, FreeBSD, and so on. (I don't know).
Code:
#include <stdio.h>
#include <sched.h>
#include <stdlib.h>
#include <sys/wait.h>
#define _GNU_SOURCE
void *stack_memory()
{
const int stackSize = 65536;
void* stack = (void*)malloc(stackSize);
if (stack == NULL) {
printf("%s\n", "Cannot allocate memory \n");
exit(EXIT_FAILURE);
}
return stack;
}
int jail(void *args)
{
printf("Hello !! - child \n");
return EXIT_SUCCESS;
}
int main()
{
printf("%s\n", "Hello, world! - parent");
clone(jail, stack_memory(), SIGCHLD, 0);
return EXIT_SUCCESS;
}
Error:
Undefined symbols for architecture x86_64: "_clone", referenced
from:
_main in docker-4f3ae8.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code
1 (use -v to see invocation)
Linux doesn't prefix symbols with a leading _ so you're not using Linux.
But the clone(2) system call is Linux-specific, according to its man page.
clone() is Linux-specific and should not be used in programs intended
to be portable.
Probably you're using OS X or something. And you're compiling as C, so calling an un-declared function isn't a compile-time error (just a big warning). This is why it's a linker error instead of a compile-time error (and you ignored compiler warnings.)
And BTW, #define _GNU_SOURCE after including header files is pointless. You have to define feature-request macros before including headers to get them to define prototypes for GNU-only functions in cases where that's not already the default.
I want to load a shared library with dlopen and have the symbols in it available without having to individually grab function pointers to them with dlsym. The man page says that the RTLD_DEEPBIND flag will place lookup of symbols in the library ahead of global scope, but evidently this does not mean it overrides existing symbols because this does not work. Consider this example:
main.c:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
int is_loaded(){return 0;}
int main(){
void *h = dlopen("./libimplementation.so", RTLD_NOW | RTLD_DEEPBIND);
if(!h){
printf("Could not load implementation: %s\n", dlerror());
return 1;
}
puts(is_loaded() ? "Implementation loaded" : "Implementation not loaded");
dlclose(h);
}
implementation.c:
int is_loaded(){return 1;}
Makefile:
all: main libimplementation.so
main: main.c
gcc -Wall -std=c99 -o $# $^ -ldl
lib%.so: %.c
gcc -Wall -std=c99 -o $# $^ -shared
clean:
-rm main *.so
When I build and run with make and ./main, I expect the test() function from libimplementation.so to override the test() function from main but it doesn't. I know I could also move all of the code in main() into another shared library run and then have main() dlopen libimplementation.so with RTLD_GLOBAL and then have librun.so refer to the symbols from libimplementation.so without having them defined so it loads them:
modified main.c:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
int main(){
void *impl_h = dlopen("./libimplementation.so", RTLD_LAZY | RTLD_GLOBAL);
if(!impl_h){
printf("Could not load implementation: %s\n", dlerror());
return 1;
}
void *run_h = dlopen("./librun.so", RTLD_LAZY);
if(!run_h){
printf("Could not load run: %s\n", dlerror());
dlclose(impl_h);
return 1;
}
void (*run)(void);
*(void**)&run = dlsym(run_h, "run");
if(!*(void**)&run){
printf("Could not find entry point in run: %s\n", dlerror());
dlclose(impl_h);
dlclose(run_h);
return 1;
}
run();
dlclose(impl_h);
dlclose(run_h);
}
run.c:
#include <stdio.h>
int is_loaded(void);
void run(void){
puts(is_loaded() ? "Implementation loaded" : "Implementation not loaded");
}
and the Makefile gets librun.so added as a prerequisite for all.
Is there a way to get the symbols from the shared library available all at once without dlsym or putting the actual code in another shared library like with librun.so?
There is fundamentally no way to do what you're asking for. Imagine the main program had something like:
static char *myptr = array_in_lib1;
Later, at the time you dlopen, myptr has some other value. Has the program just changed the variable to point to a different object? Or has it been incremented to point to some element later in the array - in which case, would you want it adjusted to account for the redefinition of array_in_lib1 with a new definition from the newly-opened library? Or is it just a random integer cast to char *? Deciding how to treat it is impossible without understanding programmer intent and full process history of how it arrived in the current state.
The above is a particulrly egregious sort of example I've constructed, but the idea of symbols changing definition at runtime is fundamentally inconsistent in all sorts of ways. Even RTLD_DEEPBIND, in what it already does, is arguably inconsitent and buggy. Whatever you're trying to do, you should find another way to do it.
SO basically i got the lua5.3.3 source code and i am trying to build it with mingw
what i have done so far is that i did the whole operation in msys and then copied over the lib files and bin files and include files to mingw appropriate folders
however when i try to actually compile an application that uses it, i get these errors
this is the command i used to compile my program that uses lua
gcc syx.cpp -llua
C:\Users\User\AppData\Local\Temp\cckJPF8N.o:syx.cpp:(.text+0xf): undefined reference to `luaL_newstate()'
C:\Users\User\AppData\Local\Temp\cckJPF8N.o:syx.cpp:(.text+0x21): undefined reference to `luaL_openlibs(lua_State*)'
C:\Users\User\AppData\Local\Temp\cckJPF8N.o:syx.cpp:(.text+0x3e): undefined reference to `luaL_loadfilex(lua_State*, char const*, char const*)'
C:\Users\User\AppData\Local\Temp\cckJPF8N.o:syx.cpp:(.text+0x77): undefined reference to `lua_pcallk(lua_State*, int, int, int, int, int (*)(lua_State*, int, int))'
C:\Users\User\AppData\Local\Temp\cckJPF8N.o:syx.cpp:(.text+0x87): undefined reference to `lua_close(lua_State*)'
collect2.exe: error: ld returned 1 exit status
and here is the file(very basic) in case you need to see it
#include <stdio.h>
#include <lua5.3/lua.h>
#include <lua5.3/lualib.h>
#include <lua5.3/lauxlib.h>
/* the Lua interpreter */
lua_State* L;
int main ( int argc, char *argv[] )
{
L = luaL_newstate();
luaL_openlibs(L);
luaL_dofile(L, "test.lua");
/* cleanup Lua */
lua_close(L);
return 0;
}
i know the library file exists since it was created with mingw namely liblua.a which sits in my mingw lib folder , as well as the other files related to lua such as lua.exe luac.exe the include files etc so i am not sure what else is missing
Okay i found the solution
turns out gcc is mangling the symbols from the library header(whatever that means)
what needs to be done is the headers need to be wrapped in a extern "C" {//headers} to make it work
reference is here :
http://www.linuxquestions.org/questions/programming-9/undefined-reference-error-when-using-lua-api-892782/
and just to be clear, here is a working example:
#include <stdio.h>
extern "C"{
#include <lua5.3/lua.h>
#include <lua5.3/lualib.h>
#include <lua5.3/lauxlib.h>
}
/* the Lua interpreter */
lua_State* L;
int main ( int argc, char *argv[] )
{
L = luaL_newstate();
luaL_openlibs(L);
luaL_dofile(L, "test.lua");
/* cleanup Lua */
lua_close(L);
return 0;
}
I am using dlsym to look up symbols in my program, but it always returns NULL, which I am not expecting. According to the manpage, dlsym may return NULL if there was an error somehow, or if the symbol indeed is NULL. In my case, I am getting an error. I will show you the MCVE I have made this evening.
Here is the contents of instr.c:
#include <stdio.h>
void * testing(int i) {
printf("You called testing(%d)\n", i);
return 0;
}
A very simple thing containing only an unremarkable example function.
Here is the contents of test.c:
#include <dlfcn.h>
#include <stdlib.h>
#include <stdio.h>
typedef void * (*dltest)(int);
int main(int argc, char ** argv) {
/* Declare and set a pointer to a function in the executable */
void * handle = dlopen(NULL, RTLD_NOW | RTLD_GLOBAL);
dlerror();
dltest fn = dlsym(handle, "testing");
if(fn == NULL) {
printf("%s\n", dlerror());
dlclose(handle);
return 1;
}
dlclose(handle);
return 0;
}
As I step through the code with the debugger, I see the dlopen is returning a handle. According to the manpage, If filename is NULL, then the returned handle is for the main program. So if I link a symbol called testing into the main program, dlsym should find it, right?
Here is the way that I am compiling and linking the program:
all: test
instr.o: instr.c
gcc -ggdb -Wall -c instr.c
test.o: test.c
gcc -ggdb -Wall -c test.c
test: test.o instr.o
gcc -ldl -o test test.o instr.o
clean:
rm -f *.o test
And when I build this program, and then do objdump -t test | grep testing, I see that the symbol testing is indeed there:
08048632 g F .text 00000020 testing
Yet the output of my program is the error:
./test: undefined symbol: testing
I am not sure what I am doing wrong. I would appreciate if someone could shed some light on this problem.
I don't think you can do that, dlsym works on exported symbols. Because you're doing dlsym on NULL (current image), even though the symbols are present in the executable ELF image, they're not exported (since it's not a shared library).
Why not call it directly and let the linker take care of it? There's no point in using dlsym to get symbols from the same image as your dlsym call. If your testing symbol was in a shared library that you either linked against or loaded using dlopen then you would be able to retrieve it.
I believe there's also a way of exporting symbols when building executables (-Wl,--export-dynamic as mentioned in a comment by Brandon) but I'm not sure why you'd want to do that.
I faced the similar issue in my code.
I did the following to export symbols
#ifndef EXPORT_API
#define EXPORT_API __attribute__ ((visibility("default")))
#endif
Now for each of the function definition I used the above attribute.
For example the earlier code was
int func() { printf(" I am a func %s ", __FUNCTION__ ) ;
I changed to
EXPORT_API int func() { printf(" I am a func %s ", __FUNCTION__ ) ;
Now it works.
dlsym gives no issues after this.
Hope this works for you as well.