Is there a way one can compile a script in which everything except standard Lua libraries is linked statically?
I am embedding a script in my C program, the functionality is split between modules, which are then loaded in main module. I would like to compile the main module into Lua bytecode, convert it to hex code, so it can easily be stored in my program. The problem is that main module still requires source code of other modules to be present either in LUA_PATH or in current directory. Is there a way how to override this behaviour? Sure, I could simply merge all files (automatically, during the compilation) into one lua source file, but since I am a beginning with Lua, I would like to know if there are some other solutions to this problem, and perhaps expand my horizons.
I was looking at luac and luajit but I could not make them to do what I need.
Any hints?
There are tools like srlua that may help. You may also check this presentation on luawrap and this discussion for ideas.
serialise lua_state
In C if you iterate through the loaded lua_state for functions and values you could create an as-if lua representation.
This could be serialised to lua form and then luac compiled.
You would have to ignore C bound functions and userdata.
You would need to walk meta tables.
encapsulated form
By changing the loader, you could have a single resource which has each of lua files by name and treats load requests as seeks and reads in resource.
Related
I have a simple C function, called add:
int add(int a, int b) {
return a + b;
}
I need to get the optimized executable binary code for this given function without any of the other side effects of the binary. This would then be dynamically loaded and modified to include templates via a JIT system. I have experimented with trying to use objdump, but that doesn't really work for getting the actual binary as the object file is not the actual executable.
I need to link together the moving parts of functions to include their binary together at runtime and enabled PROT_EXEC to execute all of these partials together as a single function and call as a function pointer. The reason I need this is because I have a byte instruction encoding type which I would like to turn into JIT functions on the fly (similar to JIT compiling RegExp into validator functions). If there are any other viable solutions I should consider, then please describe them.
The main reason I want to use this approach is because I can see the godbolt shows the binary for functions as self-contained, so hopefully I can do something similar to include the binary in my own project.
Thank you for any help.
Note: the C function is entirely self-contained without referencing other functions
The easiest way is using IDA. You can compile your code (preferably with symbols) into ELF file/ EXE file and open the generated file in IDA.
There you can see your function in the function window and then move into the hex window to copy the hex dump of your function.
Note: There are a lot of things that can go wrong here, for example, you must compile your code with the flag -PIC so it wouldn't use absolute addresses.
It is also recommended to know at least basic assembly.
This link might be interesting: https://github.com/Neetx/Shellcode-Extractor
I'm building a plugin (extension module) system for a language interpreter I'm writing in C.
During runtime, the program uses LoadLibrary in order to load a specified DLL file. This seems to work, with basic DLLs which don't depend on functions defined in the main program.
However, I'm trying to build a plugin DLL which does depend on functions which are defined in the main program binary.
To do so, I defined a interface.h header in the main code base, for these plugins to include and use. It defines the headers for the functions they might require. The plugin does #include <interface.h> in its header.
I compile the plugin DLL like so:
gcc myplugin.c -shared -Wl,--subsystem,windows -D MYPLUGIN_EXPORTS -o myplugin.dll -I..\main_program_headers
I then get the following kind of errors:
undefined reference to 'some function name'
Does this mean I have to compile the plugins with dynamic linking to the actual binaries they depend on in the main program?
If so, does this mean I need to keep the individual .o files of the main program around, even after linking them to the result .exe? Can GCC link directly against the .o files?
Anyway, I really hoped LoadLibrary takes care of fixing the function references on load during runtime. Doesn't it?
Update:
As #tenfour kindly answered, a DLL must follow normal linking rules and function references need to be resolved at build time. He / she suggested a system where the host program passes into the plugin pointers for the needed function.
This approach will indeed work, but I would still like to know:
What kind of build process is necessary to allow the plugin to call functions from the main app directly, without any special system at runtime (except for LoadLibrary).
I would like to say that my main source of influence here is the extension system for the CPython interpreter. It seems to me judging by its documentation that a CPython extension doesn't receive function pointers from the host interpreter, and is still able to directly call Py_* functions from it.
Any idea how such a thing can be done? How can the plugin DLL be built to support this? What and how do I need to link against?
Since you didn't post interface.h, I can only guess that you are forward-declaring functions there, for example:
int some_func();
If the plugin code tries to invoke this method, it will compile, but the linker has no reference to it. The body of that function only exists in your host application.
Instead, if you want to dynamically link, using LoadLibrary, you need to use function pointers, e.g.:
typedef int (*some_func_ptr)(); // declare the fn pointer type
...
some_func_ptr some_func = x; // assign it after the host passes you x
...
some_func(); // now you can call it without linker issues.
And viola you have used dynamic linking to create a plugin system. Of course the design of your interface will force this to be more complex but that's mostly just labor once you understand the concept.
I am working on a program in C, and I am having trouble with libconfig.h. Because of this, I think if I could have my program download an external function from the Internet (using libcurl.h) and have my program treat it as a compiled and linked shared object, that would be perfect. It would need to work on all desktop platforms (Windows, Mac, and Linux), so no .dll's, and would have to be downloaded by the program, treated as a function, and then get deleted by the program. So, my question is: is that possible in C?
The reason that I need to download it separately is because the function would need to be updated regularly, and requiring the user to download a new version of the program regularly would defeat the purpose of the program.
Well the closest to what you ask for would be this
Download .so/.dll using curl
Dynamically load .so/.dll into your process
set up function pointer in your process to point to a function in .so/.dll
On Windows:
HMODULE handle = LoadLibrary("mylib.dll");
if (handle)
myfunc = GetProcAddress(handle, "myfunc");
To unload call
FreeLibrary(handle)
It decreases ref count, and the DLL is actually unloaded when ref count hits 0.
On Linux, check this post:
How do I load a shared object in C++?
You can't just treat it as compiled; you would have to do one of two things:
Actually compile it on the fly, then load it as a dynamic library, which requires ensuring that there is a compiler on the system and will probably cause an unholy mess of errors on the user end.
Build your own C parser to interpret the external function, which is no small feat.
Far simpler solution: just write a function that works and compile platform-specific versions of it into your binary (or a library, if you prefer) before shipping the product.
You could link a Python interpreter into your program and have it execute a Python version of your function.
This approach would actually work with different languages, such as Java, Ruby, etc.
I have a bunch of code I need to analyse that I don't know how to do. I have a pile of code that here and there are using math functions from a header file I have included called math.h that came with my IDE. I am being asked to see how much space is used to include this. Specifically is the compiler including all of the library functions or just the ones we use. There is no object file being created. So I think the library code is being compiled into the individual files. Any ideas of a slick way to figure this out? I can't just comment out the includes because then the code wont complie and I won't know a size and if I comment out all the lines that use math functions it is not really representative.
You can use the objdump command to see the individual symbols inside your object files and the space they require.
Note that unless you're doing static compilation, library methods aren't generally copied into your produced binary, but only referenced (and brought in via the dynamic linker when your program is loaded).
As math.h is part of the standard C library, a copy of that library is basically guaranteed to always be in memory, so the additional memory and space requirements on dynamic linking are minimal. (During static linking, all symbols which aren't directly required by your program are discarded, and math functions don't tend to be very big, so usage should be fairly minimal there too).
The code in the header file is being complied into the object file of the .c you are using if your header has the definitions of the functions and just being referenced to if they are simply the declarations. The linker will then find a definition for each symbol and place it in your executable if you are using dynamic linking the OS will pull in the definition at run time.
Using TinyCC in my C program lets me use C as a sort of scripting language, reload C files on the fly, and do a lot of fairly neat things... But, one thing is really bothering me. Linking.
I do my normal tcc_new, and tcc_set_output_type with TCC_OUTPUT_MEMORY, but if I don't include a lot of these:
tcc_add_symbol(tcc_ctx, "printf", &printf);
tcc_add_symbol(tcc_ctx, "powf", &powf);
tcc_add_symbol(tcc_ctx, "sinf", &sinf);
everything is very limited.
I want a way to automatically bring in all symbols in the host program. I don't want to have to manually link in every last function in libc, and libm. What mechanisms exist to facilitate auto linking, or adding of symbols. How can I use libm in my code without manually dropping in every last component.
I'm currently using GCC, but on another platform use Visual Studio to compile my program. I could switch entirely to TCC.
TCC comes with a rudimentary runtime library libtcc1. It includes basic functions like those you mention. Therefore, in most cases you can replace all your calls with a single tcc_add_library(tcc_ctx, "libtcc1.a").
libtcc1 is not complete, so you might have to add manually some functions.