Here's my predicament: I am using PhysFS, which allows me to treat multiple directories and archives as one virtual directory. This is for a video game in the future, which I intend to make modder-friendly. PhysFS is the best way to make it modding-friendly.
At the same time, I also intend to use a script-extender of a sort that dnyamically loads libraries and registers them in Lua. I already created a crude, makeshift, largerly proof-of-concept Lua script extender.
So what is my issue? My issue is that dlopen / LoadLibrary only works with files on a real filesystem. And I want to load via PhysFS. I load the file via PHYSFS_openRead, then use PHYSFS_read to load the file's full content into a memory buffer.
Some people suggested loading the .so / .dll file into the memory from PhysFS, and then writing it out to /tmp in Linux, or C:\temp in Windows, and then dlopen-ing it. But I don't think that's a very elegant way to do it.
So... any other ideas? I did look into mmap and thought maybe I could manually load the ELF file (on Linux) and somehow manually extract the functions, and finally register them for Lua, but so far, all I could produce was a program that gave me information about the elf.
People said that I should look into LibJIT, but I'm not exactly sure how that would help me.
So what should I do? How do I load a library into memory via PhysFS, and use an alternative to dlopen to... dlsym the functions out of it?
Please don't suggest dlopen, unless you genuinely suggest that I write out the file temporarily.
My question is largerly: how do I link? How do I get the functions out of the library which I already have in memory? And why do some people suggest LibJIT to me?
Related
I was wondering if there was a specific set of library that can decompress tar.gz files in an rtems operating system. Most of the answer I seen is for linux. I was looking at gzip and libz but was not sure if these were for linux or can I use them for rtems as well.
There are some integrated functions that can extract tgz files in RTEMS. Take a look at the tar01 test for some examples with different sources (memory or file): https://git.rtems.org/rtems/tree/testsuites/libtests/tar01/init.c?id=8d989c56ff0c65beb7ec3390aebef6ea52840fab
Note that this is a test application intended to test the functionality. So some things might try to trigger expected failures. The test tar file is automatically generated by the Makefile, translated into some object file and linked into the application.
There is also the tarfs that can use a tar (not sure about tgz) as a read only file system. It's used in tar02 test.
Regarding libz: Although I haven't tried I would expect that it is possible to compile libz for RTEMS. If you need specifically that library you might want to ask on the RTEMS users mailing list whether someone already did that. There are a lot more RTEMS specific users and developers than on stackoverflow.
I have a program written in C that uses dlopen for loading plug-in modules. When the library is dynamically loaded, it runs constructor code which register pointer to structure with function implementations with the main application by use of exported function. I want to use absolute path for specifying the file to dlopen.
Then I have other part of the program with takes file, determine if it is ELF, then looks into the ELF header for specific ELF section, read this section and extract from it pertinent information. This way it filters only shared libraries which I have previously tagged as a plug-in module.
However, I am solving a problem how to discover them on the fly (in portable Linux way, i.e. it will run on Debian and on Fedora too and so on) from the main program. I have been thinking about using ldconfig for this. (As the modules will be installed by way of distro packaging system, APT for example.) Is there any way how to programmatically get the string list of known libraries from C program other than directly reading the /etc/ld.co.cache file? I was thinking that maybe there is some header library which will give char** when I ask.
Or, maybe is there any better solution to my problem?
(I am proponent of using standard system components that programming one-off solutions which will need support in the future.)
I have a legacy Linux application written for C that relies upon static external files on the filesystem. I'd like to bundle all of them together into a single executable, so that the single-file executable doesn't rely upon anything in the filesystem. Is there a way to do this, without having to make lots of changes to the existing code?
I can link the program statically to avoid any dependencies on dynamic libraries, but the application also relies upon other static resources (other read-only files on the filesystem), which I'd like to embed into the application. I know how to embed an external file into the final executable using objcopy (e.g., as described here or here), but then I need to arrange for the program to use the embedded blob instead of trying to open a file on the filesystem. I've seen some ways to access these ELF sections at runtime (e.g., using linker symbol names or elfdataembed, as described here and here), but they require me to change every place in the program that accesses one of these external files to instead refer to the embedded resource. That sounds tedious and error-prone.
To reduce my workload and reduce bugs, I'd prefer to minimize the amount of changes needed to the application's code. How can I do this, minimizing changes to the application? I'm thinking maybe something that wraps open() to detect attempts to open one of the external files and redirecting them to read from the blob embedded in the executable, for instance, though I'm not sure about some of the details (e.g., how the open() wrapper can create a fake fd; whether I'll need to wrap all of the other filesystem functions as well) or what the pitfalls might be.
How can I achieve this? Or, is there an existing tool I should know about, to avoid re-inventing the wheel?
If I want to clone a library and change just one function, say memcpy or memmove, and have an already built executable link to it for debugging/exploration purposes, what is the proper way to do this?
I am guessing I need to recompile the entire library with my modifications, but is there another way to do this?
I understand that there are things like malloc hooks but this seems to be a special case for malloc.
I am curious about the specifics for how valgrind and gdb do this from within another program, if someone has a resource on that.
I am interested in mac and linux solutions. On linux I've used LD_LIBRARY_PATH before - is this all that I need to do besides have the library names the same? How would I do this on mac?
For those curious as to why I want to do this, the purpose is for experimental music. I am doing this to sonify memory operations, so memcpy/memmove will work as normal but the data accessed will also be sent to the sound card. I know there are other ways of doing this (I have done a few other methods already,) but currently I am interested in focusing on memcpy/memmove, so I will appreciate it if you can restrict your answers to this focus.
You can use LD_LIBRARY_PATH to cause a program to load a shared object library different from the usual one. But if you want to replace just one function (or a few) rather than a whole library, you can use LD_PRELOAD to cause the linker (ld.so) to load a particular shared object early on, and your program will use the symbols (functions) in there rather than looking for them in the usual places.
I have an application (for which I do not have the source code).
I know that it is designed to dynamically load a shared library, depending on the command line parameters.
I know which library it should be loading and I've set up LD_LIBRARY_PATH to the appropriate directory.
The application works on one server without any problems, but does not work on another.
I'm trying to figure out why and it would be helpful if I could confirm that the script is looking for the library that I think it is and if I could track where it is looking.
Are there any tools that could help me? I've been googling, but most of the information I'm finding is about ldd which really only tells you about statically linked libraries.
This is in a Linux environment and the application and library are both written in C
Thanks
Use strace. You will see the libraries being searched etc., which will help you figure out what is happening.
As every shared library is memory-mapped into the process's address space, you can also inspect the /proc/[PID]/maps file.