I'm writing an application running on FreeRTOS where I have different threads that all have to access (read and some write) the same data structures.
So I thought I could implement a global data store holding all the data in some grouped structs. I also thought about using something like SQLite but I think that's an overkill for my application.
Nevertheless there are some open questions
To guarantee thread safety I assume I need to add a semaphore for my read and write access but...
If a thread only has to update certain elements of a struct I'd need to hand it a pointer to that struct, but as soon as I start using pointer references I can no longer protect my access with a semaphore. So how can I allow a thread to modify single struct elements without violating thread safety?
Is there a better way to have something like a global store than the one I planned to use? Google didn't give much hints.
Let me share with you the most common C-way to handle this type of scenarios.
In order to avoid exposing a lot of global data, use opaque pointers and have a list of APIs that take the opaque pointer as a parameter. In this way you can have a clean set of APIs to access all of your structures leaving the implementation specifics just in the .c file. The good thing about using opaque pointers is that no one knows how to deal with the type pointed-to-data except for the APIs that takes in opaque pointer as a parameter. This provides protection & encapsulation to your structures, so that they are ever accessed or mutated with the APIs that you provide.
In each of the structure definitions identified by the opaque pointers, have a mutex (FreeRTOS has mutex implemented in-terms of semaphores) to access the structure members in a thread-safe manner. You have to use xSemaphoreCreateMutex when creating the mutex. In all your APIs use xSemaphoreGive & xSemaphoreTake when you want to access the member data that are potentially accessed by multiple tasks.
Related
Is CryptGenRandom() thread-safe with a single global program-wide HCRYPTPROV instance?
MSDN appears to lack any info on this: https://msdn.microsoft.com/en-us/library/windows/desktop/aa379942(v=vs.85).aspx
Creating a separate HCRYPTPROV per thread and destroying it again would significantly complicate matters (and also risk more security-relevant bugs on my side), so this would be really useful to know. Sharing one global HCRYPTPROV would be a lot easier for sure.
So does anyone here know about the thread-safety of CryptGenRandom(), particularly with a single HCRYPTPROV instance?
Creating a separate HCRYPTPROV per thread doesn't make much sense. This is pointer to memory block from heap in all current implementations, primarily saved pointers to CSP entry points which used to call actual provider implementation (CPGenRandom in our case). The references themselves do not contain state of the CSP, unlike for example HCRYPTKEY which containing actual key state. So even if you create a separate HCRYPTPROV for every thread - this changes nothing.
There may be some global variables / data used by CSP internally during this call; this is however unknown as these would be implementation details. Of course we can serialize calls to CryptGenRandom in the code. However we cannot control that some other dll in our process also call CryptGenRandom concurrently. So serializing all calls to CryptGenRandom also impossible.
As result I think the CPGenRandom must be design to be thread-safe. and it my tests with a well known Microsoft CSP this is true. Internal synchronization is used in function, when need access global data and if multiple threads call CPGenRandom concurrently; every thread receives unique random data.
So my conclusion - CryptGenRandom is thread-safe, at least for all Microsoft CSP
I have two types of threads, one student the other librarian. Also I have a list of struct which holds the basic info like book name, ISBN, publishing year regarding to each books.(which is a shared resource between threads) I want to pass the pointer of a certain book in a student thread/routine to a librarian thread using condition variables. (so that a librarian could reserve the book for the student by means of signaling) How can I accomplish this is or is this even the right way to go about it?
The easiest way is to use pipes man 2 pipe.
Performance wise faster, but far more complicated ways are to use a virtual ring buffer man 3 vrb (userland pipe) or any other message passing middleware.
If these are threads (using pthread library) in the same process, you can share data since the address space is common to them. However, be aware of synchronization issues.
A common way to do that is to use a mutex for every (read or write) access to that common data. Perhaps also use condition variables for synchronization (i.e. thread A needing to tell thread B that something significant changed).
Read a good pthread tutorial (and this perhaps also).
is this even the right way to go about it?
Your example is very artificial... the only reason why you would use threads and some strange local variable list for this, is because some teacher tells you to do so. So no, this is not the right way to implement a program to be used in the real world.
In the real world, things like these would almost certainly be implemented through a database, where the DBMS handles the accessing of individual posts. Most likely in some kind of client/server system, where there is a client used by the librarian. I don't see why the student would even be part of the system, except as a data post over who borrowed the book.
I am developing a rather complex microcontroller application in C, and I have some doubts about how to "link" my shared data between the different tasks/threads without coupling them.
Until now I have used a time-sliced scheduler for running my application, and therefore there has been no need for data protection. But I want to make the application right, and I want to make it ready for an multi-threaded OS later on.
I have tried to simplify my question by using a completely different system than the actual system i am working on. I couldn't add a picture because i am a new user, but ill try and explain instead:
We got 4 tasks/threads: 3 input threads which reads some sensor data from different sensors through Hardware Abstraction Layers (HAL). The collected sensor data is stored within the task domain (ie: They wont be global!!).
Now we also got 1 output task, lets call it "Regulator". Regulator has to use (read) sensor data collected from all 3 sensors in order to generate a proper output.
Question: How will Regulator read the collected data stored in the different input tasks without coupling with other tasks?
Regulator must only know of the inputs tasks and their data by reference (ie: no #includes, no coupling).
Until now Regulator have had a pointer to each of the needed sensor data, and this pointer is set up at initialization time. This wont work in a multi-threaded application due to data protection.
I could make some getSensorValue() functions, which make use of semaphores, for each sensor value and then link these to Regulator with function pointers. But this would take up a lot of memory!! Is there a more elegant way of doing this? I am just searching for inputs.
I hope all this is understandable :)
From what you described in the question and comments it seems like you're most worried about the interfacing between Sensors and Regulators being low-memory with minimal implementation details and without knowing the explicit details of each Sensor implementation.
Since you're in C and don't have some of the C++ class features that would make encapsulation easier via inheritance, I'd suggest you make a common datapackage from each Sensor thread which is passed to Regulators rather than pass a function pointer. A struct of the form
struct SensorDataWrap {
DataType *data;
LockType *lock;
... other attributes such as newData or sensorName ...
};
would allow you to pass data to Regulators, where you could lock before reading. Similarly the Sensors would need to lock before writing. If you changed data to be a double pointer DataType **data you could make the write command only need to lock for the time it takes to swap the underlying pointer. The Regulator then just needs a single SensorDataWrap struct from each thread to process that thread's information regardless of the Sensor implementation details.
The LockType could be a semaphore, or any higher level lock object which enables single-access acquisition. The memory footprint for any such lock should only be a couple bytes. Furthermore you're not duplicating data here, so you shouldn't have any multiplicative effects on your memory size relative to sensor read-outs. The hardware you're using should have more than enough space for holding a single copy of the data from the sensors you described as well as enough flash space to accommodate the semaphore or lock objects.
The implementation details for communication are now restricted to lock, do operation, unlock and doesn't need complicated function pointers or SensorN specific header includes. It should take close to the minimal logic needed for any threaded shared data program. The program should also be transferable to other microcontrollers without major changes -- the communication only really restricted by the pressence/absence of threading and locks.
Another option is to pass a triple buffer object and do buffer flipping in order to avoid semaphores and locks. This approach needs atomic integer/bool support to be created (which you most likely have exposed by the compiler if you have semaphores). A guide to using triple buffers for concurrency can be found on this blog. This approach will use a little more active memory, but is a very slick way of avoiding most concurrency problems.
Im writing an Erlang C NIF that will only be used by one Erlang process. I want to create a struct that will hold an array of pointers. I need this to exist in between the process' calls to the NIF.
What I need insight to is the proper way to do this approach from the Erlang NIF side of things. Im thinking of writing a struct outside of all the functions so its accessible to all. When I create it in one call to the NIF, and then come back and use it with another call to the NIF, it seems to work just fine.
Im worried that this could be because the process is staying local to the scheduling thread and therefore does not have to move the struct and underlying array in memory.
Should I be using erlang:memalloc from within a function and avoiding globals all together or, staying as is with global structs?
Possibly return a pointer to a single array containing all my data?
You could certainly return a pointer to a single array containing your data; to do that, look at ErlNifResourceType. You would pass this back to the calling erlang process, and it in turn would pass it back to you on subsequent NIF calls. This would ensure that only one thread was operating on your data at a time (assuming only one process had a copy of the resource; it's not something you want to share, especially if it contains pointers).
You could also encode it as an erlang list, but that would probably be very inefficient.
That being said, you can use shared memory from a NIF. For example, here's an ets-like database implemented as a NIF using shared data.
You just have to keep in mind that you're accessing shared resources. The NIF API provides thread creation, thread specific data, mutexes, conditions, and read/write locks. You can even send a message to an erlang process from a NIF-created thread (in the event of a long-running NIF call, this is actually how you'd want to implement it to prevent scheduling problems).
Given your requirements, you're probably better off using the ErlNifResource type rather than messing with multithreading and shared resource controls. Technically if you're only using one erlang process you could leave it as a global variable (read: shared resource) without any harmful side effects. That being said, things change, and you don't want to be the cause of someone's headache down the road when they try to use your code from multiple processes. Whichever method you wind up using, make sure it's thread safe.
We can pass argument to a thread when using pthread_create /(not key_), if it is over using value per thread. If about using static storage, we use mutexes, so, what is it saying we use pthread_key_create? With example & emphasize on why it is born (application & need of it)?
I heard it is useful for migrating single threded applications, but they still use mutexes. So, what is the point?
Thanks!
pthread_key_create is used to create a unique identifier for a piece of thread-local storage.
Each thread can then store a different value for each key. This can be useful when migrating code that initially uses globals, but where you want each thread of execution to have its own independent version of the global value. You can use thread-local storage in place of the global.
You don't need to use mutexes when accessing thread-local data belonging to the thread you're in.