I'm using JNI as an interface between a Scala software and a dynamic native library that we will call libnative.so.
Part of said native code uses openmp to perform some parallel operations. Therefore, when building this dynamic library, I specify '-fopenmp' both as a CFLAG and as a LINKFLAG.
There are no errors during the compile process. However, on run time, I get this one:
java: symbol lookup error: /tmp/jni-5458866585640472540/libnative.so:
undefined symbol: GOMP_parallel
When using objdump to explore the symbols contained by the library, the referred one is found, although it seems to be marked as undefined:
$ objdump -TC libnative.so | grep "OMP"
0000000000000000 D *UND* 0000000000000000 GOMP_parallel
After exploring this error for a while, it seems to me that references to external libraries (such as openmp) have to be, in fact, undefined and loaded on run time. I am also thinking that the JRE is not being able to find such library at execution time on the system.
Would these assumptions be true? How could this problem be fixed?
Thank you all in advance.
Fix:
When building the dynamic library from several static libraries, we need to add -fopenmp to this process, even if we have already done so for each of said modules.
Related
We are porting an application from HPUX to Linux using Microfocus COBOL and GNU C on both platforms.One of our shared libraries is failing at runtime with the following error:
AB123: symbol lookup error. libRTS.so: undefined symbol: _mFldhandle
My understanding is that _mFldhandle is internal to Microfocus.
Can anyone point me to why we might be having an issue / what we should be including to make sure _mFldhandle is available at runtime?
Thanks!
Contrary to the comments from above the usual reason for this symbol being missing is not using "cob" to link your exe or shared object.
The other reason is not using the same 'C' compiler that the product was created with.
I'm trying to debug an API in use by a third party app.
The third party app has a configuration I had thought impossible:
The application binary contains exports required by one of its shared library dependencies. That is, they dynamically linked their dependency, but that dependency's dependency is statically linked.
This is occurring on a MIPS based linux with an ancient kernel.
I have confirmed this using Ghidra to disassemble the executable/shared library.
Basically we have the binary and the shared object
file initApp
initApp: ELF 32-bit LSB executable, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, interpreter /lib/ld-uClibc.so.0, stripped
file libhyCoreSdk.so
libhyCoreSdk.so: ELF 32-bit LSB shared object, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, stripped
objdump -T libhyCoreSdk.so | grep -i IMP_ISP_Open
00047900 DF *UND* 00000000 IMP_ISP_Open
initApp calls "coreAvInit" in libhyCoreSdk.so, and then libhyCoreSdk.so calls IMP_ISP_Open() back inside initApp.
You might be thinking "that's not that unusual, you can do that with callbacks, passing pointers, a call to dlsym etc. etc." But this isn't any of those things, this is a direct import/export of symbols exactly what you'd expect to see if libhyCoreSdk depended on another shared library.
All that aside, my real problem is that I am trying to figure out what parameters are being passed by IMP_ISP_Open and LD_PRELOAD does not help this strange/unique circumstance. I have another example application which is calling the same APIs against a shared library version of the API. When I use LD_PRELOAD (loading a little interceptor program I wrote) against that version it works great. But when I use it against this version that links back to the binary, it doesn't work.
I'm hoping someone has ideas on how I can intercept those API calls.
Partial Solution:
Daniel Kleinstein has given me a good start. I renamed the target functions from IMP_... to XMP_... (e.g. IMP_ISP_DisableSensor-> XMP_ISP_DisableSensor). Now the IMP_ISP_DisableSensor from my LD_PRELOAD is correctly hit. Unfortunately, I'm still running into trouble. Calling dlsym(RTLD_DEFAULT, "XMP_ISP_DisableSensor") returns NULL for no obvious reason... This means I can't redirect back to the original function.
Despite objdump showing this:
objdump -T initApp_mod | grep -i XMP
0061fb5c g DF .text 00000338 Base XMP_FrameSource_CreateChn
00613c6c g DF .text 00000204 Base XMP_ISP_DisableSensor
006097f0 g DF .text 00000eb0 Base XMP_Encoder_CreateChn
Your dynamic dependency is not statically linked against your main executable. When the dynamic dependency is loaded, a search is made for its import and is resolved by a symbol exported by the executable - indeed, the executable's symbols will always resolve before any of the other dynamic dependencies.
Unfortunately, this also prevents your usage of LD_PRELOAD.
LD_PRELOAD doesn't work because its injected library does not take precedence over symbols exported by the main executable itself - only over symbols exported by shared libraries.
If you wish to intercept the call using LD_PRELOAD, a crude but effective solution is to patch the main executable's exported symbol to a different name - this way it won't resolve when the dynamic dependency gets loaded, and your injected library can supply the symbol.
I'm compiling my shared library, which is meant to be used in another (main) shared library of mine.
So, the whole set-up is compiled using cmake roughly as follows:
For the "main" shared library:
ADD_LIBRARY(lib_outer SHARED
....
)
TARGET_LINK_LIBRARIES(lib_outer
lib_inner
...
)
For the additional shared library:
ADD_LIBRARY(lib_inner SHARED
....
)
Now, the lib_inner uses some functions defined and implemented in the second lib_outer, Which leads to Undefined symbols for architecture x86_64 linking error.
Can i somehow tell the cmake to ignore those?
I shall i use something like 'externalwhen using those functions insidelib_inner` ?
I guess the solution is simple, but i never came across such a problem.
After some discussion with colleagues, it seems that the only good option is to compile the two libraries simultaneously as a whole. Effectively combining the CMakeLists.txt files which before were used to produce two separate, but dependent libraries.
I'm getting errors in the lua plugin that I'm writing that are symptomatic of linking in two copies of the lua runtime, as per this message:
http://lua-users.org/lists/lua-l/2008-01/msg00671.html
Quote:
Which in turn means the equality test for dummynode is failing.
This is the usual symptom, if you've linked two copies of the Lua
core into your application (causing two instances of dummynode to
appear).
A common error is to link C extension modules (shared libraries)
with the static library. The linker command line for extension
modules must not ever contain -llua or anything similar!
The Lua core symbols (lua_insert() and so on) are only to be
exported from the executable which contains the Lua core itself.
All C extension modules loaded afterwards can then access these
symbols. Under ELF systems this is what -Wl,-E is for on the
linker line. MACH-O systems don't need this since all non-static
symbols are exported.
This is exactly the error I'm seeing... what I don't know is what I should be doing instead.
I've added the lua src directory to the include path of the DLL that is the c component of my lua plugin, but when I link it I get a pile of errors like:
Creating library file: libmo.dll.a
CMakeFiles/moshared.dir/objects.a(LTools.c.obj): In function `moLTools_dump':
d:/projects/mo-pong/deps/mo/src/mo/lua/LTools.c:38: undefined reference to `lua_gettop'
d:/projects/mo-pong/deps/mo/src/mo/lua/LTools.c:47: undefined reference to `lua_type'
d:/projects/mo-pong/deps/mo/src/mo/lua/LTools.c:48: undefined reference to `lua_typename'
d:/projects/mo-pong/deps/mo/src/mo/lua/LTools.c:49: undefined reference to `lua_tolstring'
So, in summary, I have this situation:
A parent binary that is statically linked to the lua runtime.
A lua library that loads a DLL with C code in it.
The C code in the DLL needs to invoke the lua c api (eg. lua_gettop())
How do I link that? Surely the dynamic library can't 'see' the symbols in the parent binary, because the parent binary isn't loading them from a DLL, they're statically linked.
...but if I link the symbols in as part of the plugin, I get the error above.
Help? This seems like a problem that should turn up a lot (dll depends on symbols in parent binary, how do you link it?) but I can't seem to see any useful threads about it.
(before you ask, no, I dont have control over the parent binary and I cant get it to load the lua symbols from the DLL)
It's probably best to use libtool for this to make your linking easier and more portable. The executable needs to be linked with -export-dynamic to export all the symbols in it, including the Lua symbols from the static library. The module needs to then be linked with -module -shared -avoid-version and, if on Windows, additionall -no-undefined; if on MacOS, additionally -no-undefined -flat_namespace -undefined suppress -bundle; Linux and FreeBSD need no other symbols. This will leave the module with undefined symbols that are satisfied in the parent. If there are any missing, the module will fail to be dlopened by the parent.
The semantics are slightly different for each environment, so it might take some fiddling. Sometimes order of the flags matters. Again, libtool is recommended since it hides much of the inconsistency.
I recently ran nm -m -p -g on the System.B.dylib library from the iOS SDK4.3 and was surprised to find a lot of symbols marked (undefined) (external). Why and when would an undefined symbol be marked external? I can understand a undefined external symbol marked lazy or weak but these aren't. Many of the pthread_xxx functions fall in this category. When I link with this library however, all symbols are resolved. The pthread_xxx symbols are defined in one of the libraries in the \usr\lib\system folder so I am assume they are satisfied from there. How does that work during linking?
It's been a while since I was an nm and ld C-coding ninja, but I think this only means that there are other libraries this one links against.
Usually this is how dynamic linking works. If you were to nm a static archive of System.B, you would not have observed this behavior. The System.B.dylib on it's own would not do much; unless you make it as part of an ensemble set of dynamic and static libraries whose functions it makes use of. If you now try to compile your final binary BUT omit the library path '/usr/lib/system' then you linker will cry foul and exit with an error telling you that it cannot find a reference to pthread_XXX() (using your above example). During the final assembling of the binary, it needs to make sure it knows the location of each and every function used.
HTH