Suppose that I have a C library that requires initialization and cleanup functions that aren’t thread-safe. Specifically, these functions may invoke other thread-unsafe functions in other libraries. I don’t know (in a default build) which libraries these will be.
Now consider the case of writing Java bindings to this library. Java spawns multiple threads before running any Java code. Worse, in the case of (say) an Eclipse plugin, there could be multiple threads running Java code by the time my code receives control. Some of the other threads could be using the aforementioned unsafe functions.
My current plan is to statically link the C library (in my case, libcurl) and all transitive dependencies – in my case, a TLS library (probably mbedTLS) and (on Windows platforms) the CRT. Fortunately, libcurl cleans up everything it has allocated, so problems related to allocating from one heap and freeing it on another should not arise. Because everything is statically linked, and won’t try to load any other shared libraries, I can then initialize libcurl from a static initializer.
Will this even work? Is there a better way?
Edit: The reason that serializing library calls won’t work, and that I believe that my solution might work, is that the global state is stored not only in libcurl itself, but also in libraries libcurl depends on. Some of these libraries (ex. OpenSSL) might be in use by other code when my code is loaded. So I would need to lock against the entire process.
The reason I believe that isolating the global state would work is that libcurl (and every library it depends on) is thread safe after initialization. I need to make sure that the initialization of libcurl doesn’t create race conditions. Afterwards I am fine.
[Updated and revised]
Your concern seems to be that you will have both direct and indirect bindings to some native library -- say mbedTLS --, that that native library requires one-time initialization that is not thread-safe, and that, beyond your ability to detect or control, different threads of the process may concurrently attempt to initialize that library, or perhaps may (unsafely) attempt to initialize it more than once. That certainly seems to be a worst-case scenario.
On the other hand, you postulate that you can successfully build a monolithic, dynamically-loadable library containing the native library you want along with the transitive closure of all its dependencies (outside the kernel), so as to ensure that this library does not share state with any other library loaded by the process. You assert that after a non-thread-safe initialization, the combined stack will be thread safe, at least as you intend to use it. You want to know about how to initialize the library.
Java promises that each class will be initialized by exactly one thread, and that afterward its initialized state will be visible to all threads. Although that does not explicitly address the question, it certainly implies that if the initialization of your native libraries is performed entirely as part of the initialization of a class -- e.g. via a static initializer, as you propose -- then the correct initialized state will be visible to all Java threads. That adequately addresses the problem as I understand it.
I remain dubious that building the monolithic library is necessary, but if you truly have to deal with the worst-case scenario you seem to anticipate then perhaps it is. Inasmuch as you cannot isolate the library from conflicting demands on the kernel, however, it is conceivable that the strategy will not be sufficient. That would be one of the few conceivable good reasons for a library to rely on the kind of shared state you postulate, and your strategy would thwart that particular purpose. I cannot judge how probable such an eventuality might be, but I doubt it's very likely.
Related
I am a beginner C/C++ programmer first of all, but I am curious about it.
My question is more theoretical.
I heard that C does not have explicit multithreading (MT) support, however there are libraries which implement this. I found "process.h" header which has to be included for building MT programs, but the thing I don't understand is how the MT itself works.
I know there are threads in CPU (assume it's single core for simplicity) running and there is only one thread per moment. The CPU is switching between threads really fast so that user sees it as a simultaneous work (correct me if not).
But - what really happens when I write the following
beginthread( Thread, 0, NULL ) //or whatever function/class method we use
keeping in mind that C does not have MT support. I mean, how does code tell the PC to run two functions multithreaded while it is not possible by the language explicit methods? I guess there is some "cheat" inside library related to "process.h", but what is that cheat, I can't just find on the web.
To be more specific - I am not asking about how to use MT, but how is it build?
Sorry if was answered earlier, or question is too complicated :)
UPD:
Imagine we have C language. It has functions, variables, pointers etc. I dont know any "special" function type that can run concurrently with other. Unless there are calls to some other functions from it. But then the caller function stops and waits?
Is it so that when I run MT applications, there is a special "global" function that calls my f1() and f2() repeatedly which looks like they were simultaneously working?
First of all, C11 does actually add multithreading support to the standard, so the premise that C does not support multithreading is no longer entirely correct.
However, I'm assuming your question is more to do with how can multithreading be implemented by a C library when standard C does(/did) not provide the necessary tools. The answer lies in the word “standard” – compilers and platforms can provide additional functionality beyond that required by the standard. Using such extra features makes the program/library less portable (i.e., more is required than is specified in the C standard), but the language and function call semantics can still be C.
Perhaps it is helpful to consider a standard library function such as fopen – somewhere inside that function code must eventually be called which could not be written in standard C, i.e., the implementation of the standard library itself must also rely on platform-specific code to access operating system functionality such as the file system. Every implementation of the standard library must thus implement the non-portable parts in a way specific to that platform (this is kind of the point of having a standard library instead of all code being platform-specific). But likewise a multithreading library can be implemented with non-standard features provided by that platform, but using such a library makes the code portable only to the platforms for which the same (or compatible) multithreading library is available.
As for how multithreading itself works, it is certainly outside the scope of what can be answered here, but as a simplified conceptual model on a single processor core, you can imagine the operating system managing “concurrent” processes by running one process for a short time, interrupting it, saving its state (current instruction, registers, etc), loading the saved state of another process, and repeating this. This gives the illusion of concurrent execution though in actual fact it is switching rapidly between different processes. On multi-core systems the execution on different cores can actually be concurrent, but there are typically more processes than there are cores, so this kind of switching will still happen on individual cores. Things are further complicated by processes waiting for something (I/O, another process, a timer, etc). Perhaps it suffice to say that the scheduler is a piece of software managing all of this inside the operating system and the multithreading library communicates with it.
(Note that there are many different ways to implement multithreading and multitasking, and statements in the above paragraph do not apply to all of them.)
It's platform specific. On Windows it eventually goes down to NtCreateThread which uses the assembly instruction syscall to call the operating system. So you can qualify it as a cheat.
On Linux it's the same, just the function with the syscall is called clone instead.
So here my problem. I have a Linux program running on a VM which uses OpenCL via dlopen to execute some commands. About half way through the program's execution it will sleep, and upon resume can make no assumptions about any state on the GPU (in fact, the drivers may have been reloaded and the physical GPU may have changed). Before sleeping dlclose is called and this does unload the OpenCL memory regions (which is a good thing), but the libraries OpenCL uses (cuda and nvidia libraries in this case) are NOT unloaded. Thus when the program resumes and tries to reinitalize everything again, things fail.
So what I'm looking for is a method to effectively unlink/unload the shared libraries that OpenCL was using so it can properly "restart" itself. As a note, the process itself can be paused (stopped) during this transition for as long as needed, but it may not be killed.
Also, assume I'm working with root access and have more or less no restrictions on what files I can modify/touch.
For a less short-term practical solution, you could probably try forcing re-initialization sequence on a loaded library, and/or force unloading.
libdl is just an ordinary library you can copy, modify, and link against instead of using the system one. One option is disabling RTLD_NODELETE handling. Another is running DT_FINI/DT_INIT sequence for a given library.
dl_iterate_phdr, if available, can probably be used to do the same without the need to modify libdl.
Poking around in libdl and/or using LD_DEBUG (if supported) may shed some light on why it is not being unloaded in the first place.
If they don't unload, this is either because something is keeping a reference to them, or they have the nodelete flag set, or because the implementation of dlclose does not support unloading them for some other reason. Note that there is no requirement that dlclose ever unload anything:
An application writer may use dlclose() to make a statement of intent on the part of the process, but this statement does not create any requirement upon the implementation. When the symbol table handle is closed, the implementation may unload the executable object files that were loaded by dlopen() when the symbol table handle was opened and those that were loaded by dlsym() when using the symbol table handle identified by handle.
Source: http://pubs.opengroup.org/onlinepubs/9699919799/functions/dlclose.html
The problem you're experiencing is a symptom of why it's an utterly bad idea to have graphics drivers loaded in the userspace process, but fixing this is both a technical and political challenge (lots of people are opposed to fixing it).
The only short-term practical solution is probably either factoring out the code that uses these libraries as a separate process that you can kill and restart, and talking to it via some IPC mechanism, or simply having your whole program serialize its state and re-exec itself on resume.
I am developing a photo booth application that uses 3 modules to provide printing, capturing, and triggering functionality. The idea is that people can develop modules for it that extend this functionality. These modules are implemented as shared libraries that are loaded at runtime when the user clicks "start".
I am trying to implement a printer module that "prints" to a facebook image gallery. I want to use libcurl for this. My problem is with the initialization function: curl_global_init() The libcurl API documentation states that this function is absolutely not thread safe. From the docs:
This function is not thread safe. You must not call it when any other thread in the program (i.e. a thread sharing the same memory) is running. This doesn't just mean no other thread that is using libcurl. Because curl_global_init() calls functions of other libraries that are similarly thread unsafe, it could conflict with any other thread that uses these other libraries.
Elsewhere in the documentation it says:
The global constant situation merits special consideration when the code you are writing to use libcurl is not the main program, but rather a modular piece of a program, e.g. another library. As a module, your code doesn't know about other parts of the program -- it doesn't know whether they use libcurl or not. And its code doesn't necessarily run at the start and end of the whole program.
A module like this must have global constant functions of its own, just like curl_global_init() and curl_global_cleanup(). The module thus has control at the beginning and end of the program and has a place to call the libcurl functions.
...which seems to address the issue. However, this seems to imply that my module's init() and finalize() functions would be called at the program's beginning and end. Since the modules are designed to be swappable at runtime, there is no way I can do this. Even if I could, my application uses GLib, which per their documentation, it is never safe to assume there are no threads running:
...Since version 2.32, the GLib threading system is automatically initialized at the start of your program, and all thread-creation functions and synchronization primitives are available right away.
Note that it is not safe to assume that your program has no threads even if you don't call g_thread_new() yourself. GLib and GIO can and will create threads for their own purposes...
My question is: is there any way to safely call curl_global_init() in my application? Can I put the calls to curl_global_init() and curl_global_cleanup() in my module's init() and finalize() functions? Do I need to find another HTTP library?
First, you won't really find any other library without these restrictions since they are inherited by libcurl from 3rd party (SSL mostly) libraries with those restrictions. For example OpenSSL.
This said, the thread safe situation for global_init is very unfortunate and something we (in the curl project) really strongly dislike but cannot do much about as long as we use those other libraries. This also means that the exact situation for you depends on exactly which dependency libraries your libcurl is built to use.
You will in most situations be perfectly fine with calling curl_global_init() from your modules init() function the way you suggest. I can't guarantee this to be safe with 100% certainty of course since there are a few unknowns here that I cannot speak to.
I'm implementing a pthread replacement library which is extremely lightweight. There are several reasons why I want to disable __thread completely.
It's a waste of memory. If I'm creating a thousand threads which has nothing to do with the context that declares a variable with __thread they will still allocate the program will still have allocated 1000*the size of that data bytes and never use it. It's simply not memory compatible with the mass-concurrency model. If we need extremely lightweight fibers with only 8K of stack, a TLS block of just 4K would be an overhead of 50% of the memory used by each thread. In some scenarios the TLS overhead would be enormous.
TLS is a complex standard and I simply don't have the time/resources to support it. It's to expensive. Personally I think the standard is poorly designed. It should have defined standard functions that had to be provided by the linker so thread libraries can take control over where TLS allocation takes place and insert relevant offsets and addresses it requires. Also the standard ELF implementation has been infected with pthread, expecting pthread sized structs to calculate offsets making it really hard to adapt to something else.
It's just a bad pattern. It encourages using globals and creating functions with static functions/side effects. This is not a territory we want to be in if we're creating correct programs that are easy to analyze.
If we really need "thread context" for some magic that tracks thread state behind the scenes (like for allocation or cancellation tracking) why not just expose the magic that TLS uses to understand that context in the first place? Personally I'm just going to use the %fs register directly. This would not be possible in libraries for obvious reasons but why should they be thread aware to begin with? Why not just design them correctly so they get the context related data they need in the first place right in the argument list?
My question is simply: What is the simplest way to disable __thread support and make clang emit errors if you accidentally used it? How can I get errors if I load a dynamic library which happens to require TLS?
I believe the simplest way is adding something like this unconditionally to your CFLAGS (perhaps from clang's equivalent of the gcc specfile if you want it to be system-global):
-D__thread='^-^'
where the righthand side can be anything that's syntactically invalid (a constraint violation) at any point in a C program.
As for preventing loading of libraries with TLS, you'd have to patch the linker and/or dynamic linker to reject them. If you're just talking about dlopen, your program could first read the file and parse the ELF headers for TLS relocations, then reject the library (without passing it to dlopen) if it has any. This might even be possible with an LD_PRELOAD wrapper.
I agree with you that, especially in its current implementation, TLS is something whose use should generally be avoided, but may I ask if you've measured the costs? I think stamping it out completely will be fairly difficult on a system that's designed to use it, and there's much lower-hanging fruit for cutting bloat. Which libc are you using? If it's glibc, I'm pretty sure glibc has lots of TLS it uses internally itself these days... Of course if you're writing your own threads implementation, that's going to entail a lot of interaction with the rest of the standard library, so perhaps you're already patching it out...?
By the way (shameless plug follows), we have an extremely light-weight threads implementation in musl libc which presently does not have TLS. I don't think it would be easy to integrate with another libc (and I'm sure if you're writing your own you will find it difficult to integrate with glibc, especially glibc's dynamic linker, which expects TLS to be supported) but if you can use the whole library as-is, it may meet your needs for specific projects, or have useful code you can borrow (license is MIT).
I have a program and bunch of "plug-ins" (shared libraries) that the main program loads on request during the runtime.
The plug-ins can access all the internal global data-structures/functions of the program, so there is no option to keep version for each time the internal data-structures changed.
I'm seeking for a way, that the main program can check if the plug-in it tries to load is supported (uses the appropriate data-structures).
Is there a creative way you can think of, doing this?
Have a function in the plugin returns information about the version of the protocol its support (The protocol of a plugin isn't restricted to what it provides, it is also what is required from the calling program.)
AProgrammer's answer (or simply exporting a global variable with the version number) will work, but bear in mind that no solution is foolproof or safe against malicious plugin files. Loaded modules run in the same memory space as your program, with the same privileges, and unfortunately the dynamic loader will happily run global constructors in the plugin before you are able to query the version or perform any checking yourself. (Grumble anyone have a link to Global Constructors Considered Harmful?)
In any case, if the plugin architecture is your design, I would highly recommend you ban any use of global constructors in specification for plugins. Of course you can't enforce this at runtime, but at least then you can blame any plugin author who breaks things for violating the contract.