I am using google cloud PubSub and was wondering whether google.cloud.pubsub_v1.PublisherClient was thread safe. Do I need to pass a new instance of this object to each threading.Thread or is it safe to share the same instance across threads?
It depends on the client library you are using.
This Python client library is not thread safe due to being built on top of the httplib2 library, which is not thread-safe. But, as the first link says, this is an old library. The newer Python library for Pub/Sub is thread safe.
Other Google client libraries are "thread safe" too. The .NET one, for example, was bult with the convention in mind that all static members should be thread-safe.
Related
I am looking at IPC between .NET and C on Linux.
I have managed to get some shared memory working using:
C shm_open and mmap
C# MemoryMappedFile
I need a mechanism though to be able to synchronize the changes between my .NET application and the C application and i cannot find a way to have a shared lock mechanism.
I have looked at named semaphores (but this is not supported on linux in .NET).
I have looked at named Mutex (but this is not supported in C on linux).
I have looked at creating a shared memory area for a phtread_mutex_t object in C on linux which works but i have no way to map that to anything in the .NET environment.
Does anyone know of a way I can do this or maybe get the pthread_mutex_t method working?
Goal
I'd like to write C/C++ applications and provide an API for them to communicate directly with systemd-managed services. Effectively, I'd like my applications to be able to do the equivalent of systemctl start service_name#unit_number.service without using system(), popen(), or exec() calls (hard requirement).
Question
Is there a simple C/C++ API for communicating with systemd, assuming systemd version 219 (i.e. CentOS v7.4, also a hard requirement)?
Work So Far
I've scoured online for examples of controlling systemd-managed services in C/C++ and found the odd discussion on the topic, but haven't found any good examples.
I could always reverse-engineer systemctl version 219 from source, but then I potentially run afoul of GPL licensing.
Since I'm stuck with CentOS v7.4, which means I can't get at version 221 or later of systemd, I can't use the newer "SD Bus" API. Nobody would allow manually upgrading libsystemd just for an application-specific benefit, and I'm apparently not the first person to raise this concern.
I'm hesitant to use the low-level C API for DBUS, since the maintainers themselves state "If you use this low-level API directly, you're signing up for some pain.".
Hard Requirements
systemd v219.
CentOS v7.4
C/C++.
No system()/popen()/exec() calls to the systemctl CLI utility.
Non-propagating license (i.e. LGPLv2 is OK, though BSD/MIT is preferred).
Question (Redux)
Is there a simpler API that I could use on CentOS v7.4 (even if I have to supply the shared library myself) that can be used to communicate with systemd in a simple, reliable manner directly via C/C++ code? If the existing libdbus-1.so API is complicated but still reliable, and I can create a reliable/stable wrapper library around it, I'm open to exploring that option. Also, if it's possible to manually build and deploy the SD-BUS components independent of systemd and use them without having to modify/upgrade the systemd library/daemon already on the system, I could potentially go that route too.
Edits & Comments
I would be open to using a mature C++ library, so long as it doesn't require total release of all source.
Thank you.
As you already figured out, you should only interact with systemd using a dbus library, there is no other supported way to do so.
Even if you lifted the requirement of no execution of binaries, it will remain frowned upon, as the *ctl tools are command line user interfaces not intended or designed to be called from other programs.
Suppose that I have a C library that requires initialization and cleanup functions that aren’t thread-safe. Specifically, these functions may invoke other thread-unsafe functions in other libraries. I don’t know (in a default build) which libraries these will be.
Now consider the case of writing Java bindings to this library. Java spawns multiple threads before running any Java code. Worse, in the case of (say) an Eclipse plugin, there could be multiple threads running Java code by the time my code receives control. Some of the other threads could be using the aforementioned unsafe functions.
My current plan is to statically link the C library (in my case, libcurl) and all transitive dependencies – in my case, a TLS library (probably mbedTLS) and (on Windows platforms) the CRT. Fortunately, libcurl cleans up everything it has allocated, so problems related to allocating from one heap and freeing it on another should not arise. Because everything is statically linked, and won’t try to load any other shared libraries, I can then initialize libcurl from a static initializer.
Will this even work? Is there a better way?
Edit: The reason that serializing library calls won’t work, and that I believe that my solution might work, is that the global state is stored not only in libcurl itself, but also in libraries libcurl depends on. Some of these libraries (ex. OpenSSL) might be in use by other code when my code is loaded. So I would need to lock against the entire process.
The reason I believe that isolating the global state would work is that libcurl (and every library it depends on) is thread safe after initialization. I need to make sure that the initialization of libcurl doesn’t create race conditions. Afterwards I am fine.
[Updated and revised]
Your concern seems to be that you will have both direct and indirect bindings to some native library -- say mbedTLS --, that that native library requires one-time initialization that is not thread-safe, and that, beyond your ability to detect or control, different threads of the process may concurrently attempt to initialize that library, or perhaps may (unsafely) attempt to initialize it more than once. That certainly seems to be a worst-case scenario.
On the other hand, you postulate that you can successfully build a monolithic, dynamically-loadable library containing the native library you want along with the transitive closure of all its dependencies (outside the kernel), so as to ensure that this library does not share state with any other library loaded by the process. You assert that after a non-thread-safe initialization, the combined stack will be thread safe, at least as you intend to use it. You want to know about how to initialize the library.
Java promises that each class will be initialized by exactly one thread, and that afterward its initialized state will be visible to all threads. Although that does not explicitly address the question, it certainly implies that if the initialization of your native libraries is performed entirely as part of the initialization of a class -- e.g. via a static initializer, as you propose -- then the correct initialized state will be visible to all Java threads. That adequately addresses the problem as I understand it.
I remain dubious that building the monolithic library is necessary, but if you truly have to deal with the worst-case scenario you seem to anticipate then perhaps it is. Inasmuch as you cannot isolate the library from conflicting demands on the kernel, however, it is conceivable that the strategy will not be sufficient. That would be one of the few conceivable good reasons for a library to rely on the kind of shared state you postulate, and your strategy would thwart that particular purpose. I cannot judge how probable such an eventuality might be, but I doubt it's very likely.
I am developing a photo booth application that uses 3 modules to provide printing, capturing, and triggering functionality. The idea is that people can develop modules for it that extend this functionality. These modules are implemented as shared libraries that are loaded at runtime when the user clicks "start".
I am trying to implement a printer module that "prints" to a facebook image gallery. I want to use libcurl for this. My problem is with the initialization function: curl_global_init() The libcurl API documentation states that this function is absolutely not thread safe. From the docs:
This function is not thread safe. You must not call it when any other thread in the program (i.e. a thread sharing the same memory) is running. This doesn't just mean no other thread that is using libcurl. Because curl_global_init() calls functions of other libraries that are similarly thread unsafe, it could conflict with any other thread that uses these other libraries.
Elsewhere in the documentation it says:
The global constant situation merits special consideration when the code you are writing to use libcurl is not the main program, but rather a modular piece of a program, e.g. another library. As a module, your code doesn't know about other parts of the program -- it doesn't know whether they use libcurl or not. And its code doesn't necessarily run at the start and end of the whole program.
A module like this must have global constant functions of its own, just like curl_global_init() and curl_global_cleanup(). The module thus has control at the beginning and end of the program and has a place to call the libcurl functions.
...which seems to address the issue. However, this seems to imply that my module's init() and finalize() functions would be called at the program's beginning and end. Since the modules are designed to be swappable at runtime, there is no way I can do this. Even if I could, my application uses GLib, which per their documentation, it is never safe to assume there are no threads running:
...Since version 2.32, the GLib threading system is automatically initialized at the start of your program, and all thread-creation functions and synchronization primitives are available right away.
Note that it is not safe to assume that your program has no threads even if you don't call g_thread_new() yourself. GLib and GIO can and will create threads for their own purposes...
My question is: is there any way to safely call curl_global_init() in my application? Can I put the calls to curl_global_init() and curl_global_cleanup() in my module's init() and finalize() functions? Do I need to find another HTTP library?
First, you won't really find any other library without these restrictions since they are inherited by libcurl from 3rd party (SSL mostly) libraries with those restrictions. For example OpenSSL.
This said, the thread safe situation for global_init is very unfortunate and something we (in the curl project) really strongly dislike but cannot do much about as long as we use those other libraries. This also means that the exact situation for you depends on exactly which dependency libraries your libcurl is built to use.
You will in most situations be perfectly fine with calling curl_global_init() from your modules init() function the way you suggest. I can't guarantee this to be safe with 100% certainty of course since there are a few unknowns here that I cannot speak to.
In case of Linux, for time functions we have a _r versions Ex: localtime has localtime_r, but in Windows I am unable to find some such functions. Are the Windows time functions inherently thread-safe?
With Microsoft Visual Studio you have a choice of c-runtimes to use: typically they were:
static single threaded library (libc)
Static multithreaded library (libcmt)
dynamic multithreaded library (msvcrt)
The multithreaded libraries are thread safe. The single threaded library was last seen in MSVC 2005 and has been dropped from MSVC 2008.
The dll runtime (msvcrt.dll) just has to be thread safe - As the implementation is in a dll and therefore shared between multiple other modules in the process, all of which could be using worker threads, It has to be threadsafe as there would be no sane way to design an application to use it otherwise.
On windows the non-_r functions are thread safe because they use thread-local storage for the buffer. See e.g. http://msdn.microsoft.com/en-us/library/bf12f0hc(VS.80).aspx
I think that Windows localtime_s is thread safe: http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx - at least when using MS CRT
I think that they aren't ThreadSafe and there is no _r version. But correct me if I'm wrong.
Maybe it's better if you use the own Windows functions, like
GetSystemTime, GetSystemTimeAsFileTime or GetLocalTime