Building standalone OCaml program - linker

I'm getting crazy while trying to understand how to produce a binary in OCaml that doesn't need anything too specific (eg. having anything as OCaml build tools installed).
I'm using OMake and everything seems enough cool while working on my machine, also linking libs with
OCAMLPACKS[] = str unix batteries
works quite well, but if I try to distribute the executable it gives me the following error:
Fatal error: exception Sys_error("/usr/godi/share/camomile/database/general_category.mar: No such file or directory")
yes I'm using GODI but I don't see any apparent reason for this error, can anyone enlighten me? In addition, if you have any tricks regarding building stand-alone applications please tell me (I'm already arguing with GODI that compiled everylib just in its x64 version and now I'm unable to produce x86 executables)
Thanks in advance

The camomile library requires runtime configuration. See README.txt for informations about how to set it up.
In particular, it requires runtime files (the missing .mar you see) to correctly work.

I don't use OMake, neither batteries, neither GODI nor Camomile. But from what I see the problem seems to come from the fact that you link against Batteries which I guess must use Camomile internally and Camomile relies on data stored in the file system to provide some of its service. It fails to load if that data is not present on the final host.
Note that shipping purely static excutables is something that is fairly simple to do with OCaml. But for that you need to consider which libraries you use (e.g. if you use camomille you will have to ship the support files with).

Related

Linker not finding dependency of Lua C Module

So I am in a bit of a pickle here and am at my wits end.
Issue:
I have made some lua C module that target another package's C++ shared library. Attempting to require the Lua module fails due to a dependency issue. I can find the module shared library just fine but when the linker attempts to find the library I need to link against, I get an error that looks like:
/usr/bin/lua: error loading module 'mytest' from file '/usr/lib/lua/5.3/mytest.so':
Error loading shared library libmy_test.so: No such file or directory (needed by /usr/lib/lua/5.3/mytest.so)
The directory structure used for linking is
|-usr
|--lib
|---lua
|----5.3
|-----mytest.so
|-mockfs
|--usr
|---lib
|----libmy_test.so
My LUA_CPATH is:
/usr/lib/lua/5.3/?.so;/mockfs/usr/lib/?.so;
My LD_LIBRARY_PATH is:
/mockfs/usr/lib/
What is driving me mad is that when the libmy_test.so is in the same directory as mytest.so, there are no problems. But due to other restrictions I cannot modify the path of libmy_test.so or add a symlink without substantial refactoring and lag time for builds/regressions. I'm unsure of where to even continue here because it is definitely a "searching for the library" problem but I don't know how to get insight into what is going on under the covers to fix the issue.
Adding /mockfs/usr/lib/?.so to LUA_CPATH is not going to help, as it only works for Lua modules loaded with require (which already works in your case for mytest.so) and doesn't work for loading their dependencies (which should be handled using dynamic linking).
It's not clear if LD_LIBRARY_PATH is even used in your case; I'd set LD_DEBUG=all to get LD troubleshooting information, which should tell you what is or is not being loaded and why.
Also, even if you can't put libmy_test.so where mytest.so is located, you can still put mytest.so where libmy_test.so is and configure LUA_CPATH like you have to try making it work that way.
So this will be unrelated to other people's Lua code. This problem description is slightly abstracted from reality as there is a lot of internal tooling from my company in front of it. I followed Paul's answer to try and get more information, LD_DEBUG was working up until I called my Lua script. Then the linker would stop giving me any information even though the require was still failing with the same error.
After a LOT of digging and my personal discovery of strace I managed to identify that the linker being used when calling require was in fact using "LD_PATH" to search for libraries instead of "LD_LIBRARY_PATH". Why this was? I still have no idea other than maybe an intentional design decision to combat some of the limitations of our internal tooling when doing cross compilation.
For anyone who experiences linker issues with Lua, LD_DEBUG and strace are very nice utilities for running down your issues.

How can I inject or dynamically load an c function into another c program

I want to build an interface in a c program which is running on an embedded system. This should accept some bytecode that represents a c function. This code will then be loaded into the memory and executed. This will then be something like remotely inject code into a running app. The only difference here is that i can implement, or change the running code and provide an interface.
The whole thing should be used to inject test code on a target system.
My current problem is that I do not know how to build such a byte code out of an existing c function. Mapping and executing this is no problem if I would knew the start address of the function.
Currently I am working with Ubuntu for testing purposes, this allows me to try some techniques which are not possible in the embedded system (according to missing operating system libs).
I build an shared object and used dlopen() and dlsym() to run this function. This works fine, the problem is just that i do not have such functions in the embedded system. I read something about loading a shared object into memory and run it, but i could not find examples for that. (see http://www.nologin.org/Downloads/Papers/remote-library-injection.pdf)
I also took a simple byte code that just print hello world in stdout. I stored this code in memory using mmap() and execute it. This also worked fine. Here the problem is that I don't know how to create such a byte code, I just used an hello world example form the internet. (see https://www.daniweb.com/programming/software-development/threads/353077/store-binary-code-in-memory-then-execute-it)
I also found something here: https://stackoverflow.com/a/12139145/2479996 which worked very well. But here i need a additional linker script, already for such a simple program.
Further I looked at this post: https://stackoverflow.com/a/9016439/2479996
According to that answer my problem would be solved with the "X11 project".
But I did not really find much about that, maybe some of you can provide me a link.
Is there another solution to do that? Did I miss something? Or can someone provide me another solution to this?
I hope I did not miss something.
Thanks in advance
I see no easy solution. The closest that I am aware of is GCC's JIT backend (libgccjit). Here is a blog post about it.
As an alternative, you could using a scripting language for that code that needs to be injected. For instance, ChaiScript or Lua. In this question, there is a summary of options. As you are on an embedded device, the overhead might be significant, though.
If using an LLVM based backend instead of GCC is possible, you can have a look at Cling. It is a C++ interpreter based on LLVM and Clang. In my personal experience, it was not always stable, but it is used in production in CERN. I would except that the dynamic compilation features are more advanced in LLVM than in GCC.

Trying to solve WindowError 126 - module not found

Though this is my first question on the StackOverflow, I consider myself a long time member of the community.
Considering myself as a pre-intermediate programmer, I'll try to be as specific as possible.
I'm writing a Python package that uses a C dll to load image files with ctypes. I'm using Python 3.3.5 32bits version on Windows 8 x64. I had to build the dll from the C code. The dll is stb_image.h which is available in GitHub. I used Code::Blocks version 13.12 with the GCC compiler. The dll seems to have built fine.
By my definition, the dll is must be in the same folder as the py code file that wraps its functions, but when ctypes attempts to load the dll an exception is raised: the [in]famous WindowsError: [Error 126] Module not found.
I've found several similar questions and attempted to glean from their solutions by
adding the dll path via os.environ['PATH']
adding the dll path manually in the Path environment variable
changing the current working directory with os.chdir()
using the file module attribute to locate the library
none of these solved the issue
The stb_image library itself has no dependencies so I don't understand why windows can't find it since the dll is where it should be and the path addresses it's exact location. And, of course, there's a lot of other ways to get the job done: PyGame, pySFML (which also uses stb_image), PIL, PyPng, PySDL, you name it, but I'm doing this mostly for learning purposes.
At the writing of this question, something popped up in my head (which may or may not have any relation with the problem): whenever I compile and link programs with GCC and try to run them, I got a message box telling me that some dll (libgcc.dll or something like that) was not found. Could that be the reason Windows can't load stb_image.dll?
I tried everything my experience allowed me to do to solve it but it was not enough. Would you guys, please, give me a light on this?
PS.: Sorry for any bad english. I'm natural from Brazil.

unknown type name ‘gpgme_decrypt_result_t’ when trying to compile balsa

I've been trying to compile balsa the email reader on ubuntu (i'm new to the whole process of compiling an open source project directly from sources.
so i did the whole configure and make but then I ran into this problem:
error: unknown type name 'gpgme_decrypt_result_t
after doing a little research on the error code.. it turns out that it belongs to gpgme a scary looking encryption library..
what do I do in this case? i hope i'm not expected to go in the code and debug stuff am i?
update:
this question is related to this other question here on the ubuntu forum
Unfortunately, you may be required to delve into the code to figure out why something doesn't build!
In this case however the answer is probably quite simple. The main reasons I would expect a symbol from an external library to not have been defined are:
The library header files do not exist on your system
You have the wrong version of the library header files on your system
The headers exist, but the compiler is not looking in the correct place
Clearly the solutions are similar: you will need to install the correct version of gpgme's header files for this software in the right place. Note that having the library itself installed is not the same as having the headers. In Ubuntu this is evident due to the presence of -dev packages which are the package's header files, and if you look through the repose you will find libgpgme11-dev. I suggest you start by installing this, which will eliminate point 1 above. If it still doesn't work, you'll need to start investigating the other two possibilities.

Tool to determine symbol origin in C

I'm looking for a tool that, given a bit of C, will tell you what symbols (types, precompiler definitions, functions, etc) are used from a given header file. I'm doing a port of a large driver from Solaris to Windows and figuring out where things are coming from is getting to be difficult, so this would be a huge help. Any ideas?
Edit: Not an absolute requirement, but tools that work on Windows would be a plus.
Edit #2: To clarify what I'm trying to do, I have a codebase I'm trying to port, which brings in a large number of headers. What I'd like is a tool that, given foo.c, will tell me which symbols it uses from bar.h.
I like KScope, which copes with very large projects.
KScope http://img110.imageshack.us/img110/4605/99101zd3.png
I use on both Linux and Windows :
gvim + ctags + cscope.
Same environment will work on solaris as well, but this is of course force you to use vim as editor, i pretty sure that emacs can work with both ctags and cscope as well.
You might want give a try to vim, it's a bit hard at first, but soon you can't work another way. The most efficient editor (IMHO).
Comment replay:
Look into the cscope man:
...
Find functions called by this function:
Find functions calling this function:
...
I think it's exactly what are you looking for ... Please clarify if not.
Comment replay 2:
ok, now i understand you. The tools i suggested can help you understand code flow, and find there certain symbol is defined, but not what are you looking for.
Not what you asking for but since we are talking i have some experience with porting and drivers (feel free to ignore)
It seems like compiler is good enough for your task. You just starting with original file and let compiler find what missing part, it will be a lot of empty stubs and you will get you code compiled.
At least for beginning i suggest you to create a lot of stubs and modifying original code as less as possible, later on once you get it working you can optimize.
It's might be more complex depending on the type of driver your are porting (I'm assuming kernel driver), the Windows and Solaris subsystems are not so alike. We do have a driver working on both solaris and windows, but it was designed to be multi platform from the beginning.
emacs and etags.
And I leverage make to run the tag indexing for me---that way I can index a large project with one command. I've been thinking about building a master index and separate module indecies, but haven't gotten around to implementing this yet...
#Ilya: Would pistols at dawn be acceptable?
Try doxygen, it can produce graphs and/or HTML and highly customizable

Resources