I have a program and bunch of "plug-ins" (shared libraries) that the main program loads on request during the runtime.
The plug-ins can access all the internal global data-structures/functions of the program, so there is no option to keep version for each time the internal data-structures changed.
I'm seeking for a way, that the main program can check if the plug-in it tries to load is supported (uses the appropriate data-structures).
Is there a creative way you can think of, doing this?
Have a function in the plugin returns information about the version of the protocol its support (The protocol of a plugin isn't restricted to what it provides, it is also what is required from the calling program.)
AProgrammer's answer (or simply exporting a global variable with the version number) will work, but bear in mind that no solution is foolproof or safe against malicious plugin files. Loaded modules run in the same memory space as your program, with the same privileges, and unfortunately the dynamic loader will happily run global constructors in the plugin before you are able to query the version or perform any checking yourself. (Grumble anyone have a link to Global Constructors Considered Harmful?)
In any case, if the plugin architecture is your design, I would highly recommend you ban any use of global constructors in specification for plugins. Of course you can't enforce this at runtime, but at least then you can blame any plugin author who breaks things for violating the contract.
Related
I'm writing some new functionality for a graphics program (written mostly in C, with small parts in C++). The new functionality will make use of libgmic. Some users of the program will have libgmic installed, quite a lot will not. The program is monolithic: I'm not writing a plugin, this will be part of the main program. Compiling the program with the right headers is easy, but I need to be able to check at runtime whether the library is installed on the user's system or not in order to enable / disable the particular menu item so that users without the library installed can't invoke this piece of functionality and crash the program. What's the best way of going about this?
You need to load the library at runtime with dlopen (or LoadLibrary on Windows) instead of linking to it, get function pointers with dlsym (GetProcAddress on Windows) and use them instead of function prototypes from the headers. Otherwise your program will simply fail to startup without the library (or crash, in some cases).
Some libraries support such usage well, such as providing types for all the functions you need. With others you’re on your own but that’s still possible.
Suppose that I have a C library that requires initialization and cleanup functions that aren’t thread-safe. Specifically, these functions may invoke other thread-unsafe functions in other libraries. I don’t know (in a default build) which libraries these will be.
Now consider the case of writing Java bindings to this library. Java spawns multiple threads before running any Java code. Worse, in the case of (say) an Eclipse plugin, there could be multiple threads running Java code by the time my code receives control. Some of the other threads could be using the aforementioned unsafe functions.
My current plan is to statically link the C library (in my case, libcurl) and all transitive dependencies – in my case, a TLS library (probably mbedTLS) and (on Windows platforms) the CRT. Fortunately, libcurl cleans up everything it has allocated, so problems related to allocating from one heap and freeing it on another should not arise. Because everything is statically linked, and won’t try to load any other shared libraries, I can then initialize libcurl from a static initializer.
Will this even work? Is there a better way?
Edit: The reason that serializing library calls won’t work, and that I believe that my solution might work, is that the global state is stored not only in libcurl itself, but also in libraries libcurl depends on. Some of these libraries (ex. OpenSSL) might be in use by other code when my code is loaded. So I would need to lock against the entire process.
The reason I believe that isolating the global state would work is that libcurl (and every library it depends on) is thread safe after initialization. I need to make sure that the initialization of libcurl doesn’t create race conditions. Afterwards I am fine.
[Updated and revised]
Your concern seems to be that you will have both direct and indirect bindings to some native library -- say mbedTLS --, that that native library requires one-time initialization that is not thread-safe, and that, beyond your ability to detect or control, different threads of the process may concurrently attempt to initialize that library, or perhaps may (unsafely) attempt to initialize it more than once. That certainly seems to be a worst-case scenario.
On the other hand, you postulate that you can successfully build a monolithic, dynamically-loadable library containing the native library you want along with the transitive closure of all its dependencies (outside the kernel), so as to ensure that this library does not share state with any other library loaded by the process. You assert that after a non-thread-safe initialization, the combined stack will be thread safe, at least as you intend to use it. You want to know about how to initialize the library.
Java promises that each class will be initialized by exactly one thread, and that afterward its initialized state will be visible to all threads. Although that does not explicitly address the question, it certainly implies that if the initialization of your native libraries is performed entirely as part of the initialization of a class -- e.g. via a static initializer, as you propose -- then the correct initialized state will be visible to all Java threads. That adequately addresses the problem as I understand it.
I remain dubious that building the monolithic library is necessary, but if you truly have to deal with the worst-case scenario you seem to anticipate then perhaps it is. Inasmuch as you cannot isolate the library from conflicting demands on the kernel, however, it is conceivable that the strategy will not be sufficient. That would be one of the few conceivable good reasons for a library to rely on the kind of shared state you postulate, and your strategy would thwart that particular purpose. I cannot judge how probable such an eventuality might be, but I doubt it's very likely.
I'm writing a Linux/Unix program that has a lot of implementation in plugins that are dlopened by the program on-demand.
I'd like to prevent these plugin libraries from using some libc functions that mess with global state of the host process (such as manipulating signal handlers and suchlike).
What would be the best way to do this?
As far as I know I can't employ the classical LD_PRELOAD trick here since the libs are dlopened.
In practical terms, you can't. Code running from a library runs with the full privileges of the host application. Don't load libraries that you don't trust to not do stupid things.
You could conceivably examine the library before loading it and (for instance) reject libraries which have unexpected dependencies, or which have relocations for functions which they shouldn't be using. (This could be accomplished using ldd or readelf, for instance.) However, this will never be entirely reliable; there are numerous ways that a malicious library could hide its use of various functions.
I am curently using the GLib g_module functions to load some shared objects during runtime.
The basic way I use is the following:
Call g_module_open to get the module
After that, call g_module_make_resident
Load Symbols by using g_module_symbol
As I am using this as a basic way to add plugin compatability, I am interested if there is a good way to make sure that even if the loaded module has a bug (like memory corruption (malloc/free)) the main application can just 'catch' this error without crashing everything?
I realy do not want you to write any code, I am just interested if there is a good way to achive this...
As Severin mentioned, there isn't really anything you can do easily. That said, you do have a few options:
The first thing you might want to consider is using something like libpeas, which allows you to load plugins in languages with non-C linkage (JavaScript, Python, etc.). Many of these languages provide much more safety than C, so if you're trying to protect against programmer error (as opposed to malicious modules) this could be a good way to go.
The other relatively straightforward way to accomplish this would be to run each plugin in a separate process. You can communicate over D-Bus, pipes, etc. One advantage of this approach is that some modules can have less permissions; if you have a program which interacts with hardware that may need root permissions, but your UI could still run as an unprivileged user. Telepathy is an example of this sort of architecture.
I'm writing an application in C that can be extended at runtime by means of modules / shared objects / DLLs. Those modules may use the API of the existing program but may also provide new functions for use in later loaded modules, so there is the possibility for modules to have dependencies on each other.
My current approach under Linux is to have every module define a depends() function that returns a list of other module names it depends on. That way, I can compile and link every module for itself, load a module with dlopen() and RTLD_LAZY, resolve its dependencies first and then fully load it with RTLD_GLOBAL. This works just fine and does exactly what I want. It also allows me to replace a module with a different version without recompiling all other modules depending on it.
The actual problem arises when porting this to Windows. First, I haven't found any way to link a DLL without already providing it with the export symbol tables of all its dependencies. Is there one I've overlooked?
Second, LoadLibraryEx from the Windows API doesn't seem to be able to perform any lazy loading because instead of letting me handle dependencies, it goes ahead and loads all referenced DLLs itself before it even returns. Since I'd like to perform version checking as well before actually loading modules in the future, this it not at all what I want. Is there any way to circumvent this behaviour?
The third odd thing is that I cannot replace a DLL without recompiling all other modules depending on it. It actually does work sometimes, but usually wild things start to happen or the program segfaults.
Is it even possible to write a modular application like that on Windows? Any suggestions or different approaches are highly appreciated!
Update: Just to provide some clarification on how my modules use each others functions on Linux (which I would like to have on Windows as well): Every module just returns the name of another module it would like to call functions from in the described depends() function and includes its header, then calls the used functions directly in the code without any wrapping. This works because Linux does not require you to have all symbols resolved at link time for shared objects.
You can export all functions manually (using __declspec(dllexport)) and load them using GetProcAddress. In this case you need to know the signature of each function, and you're limited to C functions only, but this is going to work. If you compile both modules, your C functions can also return C++ classes, more on this later. Using GetProcAddress & LoadLibrary makes modules totally independent. Basically, you do the linking manually, but as I understand, this is what you do on Linux, right?
LoadLibary only loads the libraries a library is dependent on, so make sure they don't load depend on each other. Either they are really independent, or they aren't, if done properly, changing one library will not force a recompile of the other (as you don't link them together).
A good idea is to use something like COM, so make each of your libraries return an interface, instead of individual functions. That way, you can simply load a whole DLL and also link them easily together (passing a DLL -> passing an object). Look up XPCOM and COM, it's actually very easy to do.