I have something like a list of things to calculate, and they are somewhat dependent on eachother (some of those calculations, may show that other calculations on the list are not needed).
Also I want to calculate always two of them at a time (two child threads and one main thread (which is a child thread of another one, but that's another story)).
So I want the main thread to wait for ANY of the two treads -the one that finishes first makes the main thread continue-. After it continues, it will run some code to see if the other running thread can be killed (if the one that finished shows that the other is not needed), or not, and also to run a new thread.
The idea is to do something like this:
while (/*list not empty*/) {
/*detect which two entries need to be calculated*/
/*detect if running thread can be killed*/
if (/*running thread can be killed*/) {
pthread_join(/*threadnum*/, NULL)
}
switch (/*how many threads already running?*/) {
case 0:
pthread_create(/*&threadnum*/, NULL, /*calculate*/, /*foo*/);
case 1:
pthread_create(/*&threadnum*/, NULL, /*calculate*/, /*foo*/);
break;
}
/* !!!!!!!!! Question code is next line: !!!!!!!!!*/
pthread_join(/*What goes here?*/, NULL);
// If it is impossible to do with pthread_join, what else can be used?
}
My first approach (if this was impossible) would be to store in an array the status of both threads, and check every second (with a while and sleep(1)) if any of them finished, but that would make me lose time (between 0 and 1 seconds) every time a thread finishes. So I want to avoid that if possible.
EDIT: pthread_cond_wait(/* something */) seems the way to go. However I want it to be easy: the main thread and both child threads share a global variable (parent) that is set to 0 if child threads are running, and is set to 1 when one of them stops. Ideally I want to control from the main thread everything in this way:
while (/*list not empty*/) {
/*detect which two entries need to be calculated*/
/*detect if running thread can be killed*/
if (/*running thread can be killed*/) {
pthread_join(/*threadnum*/, NULL)
}
switch (/*how many threads already running?*/) {
case 0:
pthread_create(/*&threadnum*/, NULL, /*calculate*/, /*foo*/);
case 1:
pthread_create(/*&threadnum*/, NULL, /*calculate*/, /*foo*/);
break;
}
/* !!!!!!!!! Question code is next line: !!!!!!!!!*/
pthread_cond_wait(parent, /*wtf?*/)
}
Now I have an idea to stop the parent until that condition is met, which I can set to 1 inside the child threads.
Instead of making the main thread monitor and try to kill other threads, make the other threads communicate directly amongst themselves.
For example, if thread A finishes and it becomes clear that the computation in thread B is no longer needed, simply set a boolean flag. Make thread B check that flag between steps of its computation, and give up if the flag is set.
Trying to interrupt threads is bad practice--you're better off just setting a flag that the threads will check.
Related
I am encountering an issue where I have a hard time telling which synchronization primitive I should use.
I am creating n parallel threads that work on a region of memory, each is assigned to a specific part of this region and can accomplish its task independently from the other ones. At some point tho I need to collect the result of the work of all the threads, which is a good case for using barriers, this is what I'm doing.
I must use one of the n worker threads to collect the result of all their work, for this I have the following code that follows the computation code in my thread function:
if (pthread_barrier_wait(thread_args->barrier)) {
// Only gets called on the last thread that goes through the barrier
// This is where I want to collect the results of the worker threads
}
So far so good, but now is where I get stuck: the code above is in a loop as I want the threads to accomplish work again for a certain number of loop spins. The idea is that each time pthread_barrier_wait unblocks it means all threads have finished their work and the next iteration of the loop / parallel work can start again.
The problem with this is that the result collector block statements are not guaranteed to execute before other threads start working on this region again, so there is a race condition. I am thinking of using a UNIX condition variable like this:
// This code is placed in the thread entry point function, inside
// a loop that also contains the code doing the parallel
// processing code.
if (pthread_barrier_wait(thread_args->barrier)) {
// We lock the mutex
pthread_mutex_lock(thread_args->mutex);
collectAllWork(); // We process the work from all threads
// Set ready to 1
thread_args->ready = 1;
// We broadcast the condition variable and check it was successful
if (pthread_cond_broadcast(thread_args->cond)) {
printf("Error while broadcasting\n");
exit(1);
}
// We unlock the mutex
pthread_mutex_unlock(thread_args->mutex);
} else {
// Wait until the other thread has finished its work so
// we can start working again
pthread_mutex_lock(thread_args->mutex);
while (thread_args->ready == 0) {
pthread_cond_wait(thread_args->cond, thread_args->mutex);
}
pthread_mutex_unlock(thread_args->mutex);
}
There is multiple issues with this:
For some reason pthread_cond_broadcast never unlocks any other thread waiting on pthread_cond_wait, I have no idea why.
What happens if a thread pthread_cond_waits after the collector thread has broadcasted? I believe while (thread_args->ready == 0) and thread_args->ready = 1 prevents this, but then see next point...
On the next loop spin, ready will still be set to 1 hence no thread will call pthread_cond_wait again. I don't see any place where to properly set ready back to 0: if I do it in the else block after pthread_cond_wait, there is the possibility that another thread that wasn't cond waiting yet reads 1 and starts waiting even if I already broadcasted from the if block.
Note I am required to use barriers for this.
How can I solve this issue?
You could use two barriers (work and collector):
while (true) {
//do work
//every thread waits until the last thread has finished its work
if (pthread_barrier_wait(thread_args->work_barrier)) {
//only one gets through, then does the collecting
collectAllWork();
}
//every thread will wait until the collector has reached this point
pthread_barrier_wait(thread_args->collect_barrier);
}
You could use a kind of double buffering.
Each worker would have two storage slots for results.
Between the barriers the workers would store their results to one slot while the collector would read results from the other slot.
This approach has a few advantages:
no extra barriers
no condition queues
no locking
slot identifier does not even have to be atomic because each thread could have it's own copy of it and toggle it whenever reaching a barrier
much more performant as workers can work when collector is processing the other slot
Exemplary workflow:
Iteration 1.
workers write to slot 0
collector does nothing because no data is ready
all wait for barrier
Iteration 2.
worker write to slot 1
collector reads from slot 0
all wait for barrier
Iteration 3.
workers write to slot 0
collector reads from slot 1
all wait for barrier
Iteration 4.
go to iteration 2
Is there a way to close all created threads if I don't have a list of their identifiers?
It is assumed that I only need the main thread, and the rest can be closed.
It's usually a good idea to have threads in charge of their own lifetime, periodically checking for some event indicating they should shut down. This usually make the architecture of your code much easier to understand.
What I'm talking about is along the lines of (pseudo-code):
def main():
# Start up all threads.
synchronised runFlag = true
for count = 1 to 10:
start thread threadFn, receiving id[count]
sleep for a bit
# Tell them all to exit, then wait.
synchronised runFlag = false
for count = 1 to 10:
wait for thread id[count] to exit
exit program
def threadFn():
initialise
# Thread will do its stuff until told to stop.
while synchronised runFlag:
do something relatively quick
exit thread
The periodic checking is a balance between efficiency of the thread loop and the amount of time you may have to wait for the thread to exit.
And, yes, I'm aware that pseudo-code uses identifiers (that you specifically stated you didn't have), but that's just one example of how to effect shutdown. You could equally, for example:
maintain a (synchronised) thread count incremented as a thread starts and decremented when it stops, then wait for it to reach zero;
have threads continue to run while a synchronised counter hasn't changed from the value it was when the thread started (you could just increment the counter in main then freely create a new batch of threads, knowing that the old ones would eventually disappear since the counter is different).
do one of a half dozen other things, depending on your needs :-)
This "lifetime handled by thread" approach is often the simplest way to achieve things since the thread is fully in control of when things happen to it. The one thing you don't want is a thread being violently killed from outside while it holds a resource lock of some sort.
Some threading implementations have ways to handle that with, for example, cancellability points, so you can cancel a thread from outside and it will die at such time it allows itself to. But, in my experience, that just complicates things.
In any case, pthread_cancel requires a thread ID so is unsuitable based on your requirements.
Is there a way to close all created threads if I don't have a list of their identifiers?
No, with POSIX threads there is not.
It is assumed that I only need the main thread, and the rest can be closed.
What you could do is have main() call fork() and let the calling main() (the parent) return, which will end the parent process along with all its thread.
The fork()ed off child process would live on as a copy of the original parent process' main() but without any other threads.
If going this route be aware, that the threads of the process going down might very well run into undefined behaviour, so that strange things might happen including messy left-overs.
All in all a bad approach.
Is there a way to close all created threads if I don't have a list of their identifiers? It is assumed that I only need the main thread, and the rest can be closed.
Technically, you can fork your process and terminate the parent. Only the thread calling fork exists in the new child process. However, the mutexes locked by other threads remain locked and this is why forking a multi-threaded process without immediately calling exec may be unwise.
Is it possible to build a sort of combined semaphore/spin lock in C?
That is, I want a thread control structure which supports:
Periodically waking up the thread to check the state of some variable. (like a spin lock)
Automatically waking the thread early if the state of the structure is altered by another thread (like sem_wait/sem_post).
For example in a program like this:
Parent:
while(something){
//do some stuff here.
sem_post(child_sem);
sem_wait(parent_sem);
}
Child:
while(something_else){
sem_wait(child_sem);
//do some other stuff here.
sem_post(parent_sem);
}
I would like the parent to unblock if the child fails to set parent_sem within 5 seconds, but also to unblock before 5 seconds have passed if the child has set parent_sem early, while minimizing the number of CPU cycles expended checking and re-checking the state of parent_sem over those 5 seconds. I know I can do this with a spin lock, but setting the waiting period to be high (i.e. 1 second) means wasting almost 1 second most of the time. Setting it to be low (e.g. 100ms) means doing 50 checks in the event the child times out. Neither of these is a nice solution.
This is exactly what timed locks are for. Depending on your library, they may or may not be available.
Your example:
Parent:
while(something){
//do some stuff here.
sem_post(child_sem);
while (sem_timed_wait(parent_sem, MAX_WAIT_TIME) == TIMEOUT)
// check whether you should still continue waiting
}
Child:
while(something_else){
while (sem_timed_wait(child_sem, MAX_WAIT_TIME) == TIMEOUT)
// check whether you should still continue waiting
//do some other stuff here.
sem_post(parent_sem);
}
I have used this method to increase robustness of my threads. That is, you don't want your threads to be blocked indefinitely, because there may be an error and you want to terminate them, or you may simply want to ask them to exit. On the other hand you would want to wake up as soon as possible.
This solution satisfies both conditions.
I have three threads, one thread is the main and the other two are worker threads. The first thread, when there is work to be done wakes up one of the two threads. Each thread when awakened perform some computation and while doing this if it finds more work to do can wake up the other working thread or simply decide to do the job by itself (By adding work to a local queue, for example).
While the worker threads have work to do, the main thread must wait for the work to be done. I have implemented this with condition variables as follows (the code reported here hides a lot of details, please ask if there's something non understandable):
MAIN THREAD (pseudocode):
//this function can be called from the main several time. It blocks the main thread till the work is done.
void new_work(){
//signaling to worker threads if work is available
//Now, the threads have been awakened, it's time to sleep till they have finished.
pthread_mutex_lock(&main_lock);
while (work > 0) //work is a shared atomic integer, incremented each time there's work to do and decremented when finished executing some work unit
pthread_cond_wait(&main_cond);
pthread_mutex_unlock(&main_lock);
}
WORKER THREADS:
while (1){
pthread_mutex_lock(&main_lock);
if (work == 0)
pthread_cond_signal(&main_cond);
pthread_mutex_unlock(&main_lock);
//code to let the worker thread wait again -- PROBLEM!
while (I have work to do, in my queue){
do_work()
}
}
Here is the problem: when a worker thread wakes up the main thread I'm not sure that the worker thread calls a wait to put itself in a waiting state for new work. Even if I implement this wait with another condition variable, it can happen that the main thread is awake, does some work until reaches a point in which he has to wake up the thread that has not called a wait yet... and this can lead to bad results. I've tried several ways to solve this issue but I couldn't find a solution, maybe there is an obvious way to solve it but I'm missing it.
Can you provide a scheme to solve this kind of problem? I'm using the C language and I can use whatever synchronization mechanism you think can be suited, like pthreads or posix semaphores.
Thanks
The usual way to handle this is to have a single work queue and protect it from overflow and underflow. Something like this (where I have left off the "pthread_" prefixes):
mutex queue_mutex;
cond_t queue_not_full, queue_not_empty;
void enqueue_work(Work w) {
mutex_lock(&queue_mutex);
while (queue_full())
cond_wait(&queue_not_full, &queue_mutex);
add_work_to_queue(w);
cond_signal(&queue_not_empty);
mutex_unlock(&queue_mutex);
}
Work dequeue_work() {
mutex_lock(&queue_mutex);
while (queue_empty())
cond_wait(&queue_not_empty, &queue_mutex);
Work w = remove_work_from_queue();
cond_signal(&queue_not_full);
mutex_unlock(&queue_mutex);
}
Note the symmetry between these functions: enqueue <-> dequeue, empty <-> full, not_empty <-> not full.
This provides a thread-safe bounded-size queue for any number of threads producing work and any number of threads consuming work. (Actually, it is sort of the canonical example for the use of condition variables.) If your solution does not look exactly like this, it should probably be pretty close...
If you want the main thread to distribute work to the other two, then wait until both threads have completed their work before moving on, you might be able to accomplish this with a barrier.
A barrier is a synchronization construct that you can use to make threads wait at a certain point in your code until a set number of threads are all ready to move on. Essentially, you initialize a pthread barrier, saying that x number of threads must wait on it before any are allowed to continue. As each thread finishes its work and is ready to go on, it will wait on the barrier, and once x number of threads have reached the barrier, they are all allowed to continue.
In your case, you might be able to do something like:
pthread_barrier_t barrier;
pthread_barrier_init(&barrier, 3);
master()
{
while (work_to_do) {
put_work_on_worker_queues();
pthread_barrier_wait(&barrier);
}
}
worker()
{
while(1) {
while (work_on_my_queue()) {
do_work();
}
pthread_barrier_wait(&barrier);
}
}
This should make your main thread give out work, then wait both worker threads to complete the work they were given (if any) before moving on.
Could you have "new job" queue, which is managed by the main thread? The main thread could dish out 1 job at a time to each worker thread. The main thread would also listen for completed jobs by the workers. If a worker thread finds a new job that needs doing just add it to the "new job" queue and the main thread will distribute it.
Pseudocode:
JobQueue NewJobs;
Job JobForWorker[NUM_WORKERS];
workerthread()
{
while(wait for new job)
{
do job (this may include adding new jobs to NewJobs queue)
signal job complete to main thread
}
}
main thread()
{
while(whatever)
{
wait for job completion on any worker thread
now a worker thread is free put a new job on it
}
}
I believe that what you have here is a variation on the producer-consumer problem. What you are doing is writing up an ad-hoc implementation of a counting semaphore (one that is used to provide more than just mutual exclusion).
If I've read your question right, what you are trying to do is have the worker threads block until there is a unit of work available and then perform a unit of work once it becomes available. Your issue is with the case where there is too much work available and the main thread tries to unblock a worker that is already working. I would structure your code as follows.
sem_t main_sem;
sem_init(&main_sem, 0, 0);
void new_work() {
sem_post(&main_sem);
pthread_cond_wait(&main_cond);
}
void do_work() {
while (1) {
sem_wait(&main_sem);
// do stuff
// do more stuff
pthread_cond_signal(&main_sem);
}
}
Now, if the worker threads generate more work then they can simply sem_post to the semaphore and simply defer the pthread_cond_signal till all the work is done.
Note however, if you actually need the main thread to always block when the worker is working, it's not useful to push the work to another thread when you could just call a function that does the work.
I'm creating n threads & then starting then execution after a barrier breakdown.
In global data space:
int bkdown = 0;
In main():
pthread_barrier_init(&bar,NULL,n);
for(i=0;i<n;i++)
{
pthread_create(&threadIdArray[i],NULL,runner,NULL);
if(i==n-2)printf("breakdown imminent!\n");
if(i==n-1)printf("breakdown already occurred!\n");
}
In thread runner function:
void *runner(void *param)
{
pthread_barrier_wait(&bar);
if(bkdown==0){bkdown=1;printf("barrier broken down!\n");}
...
pthread_exit(NULL);
}
Expected order:
breakdown imminent!
barrier broken down!
breakdown already occurred!
Actual order: (tested repeatedly)
breakdown imminent!
breakdown already occurred!
barrier broken down!!
Could someone explain why the I am not getting the "broken down" message before the "already occurred" message?
The order in which threads are run is dependent on the operating system. Just because you start a thread doesn't mean the OS is going to run it immediately.
If you really want to control the order in which threads are executed, you have to put some kind of synchronization in there (with mutexes or condition variables.)
for(i=0;i<n;i++)
{
pthread_create(&threadIdArray[i],NULL,runner,NULL);
if(i==n-2)printf("breakdown imminent!\n");
if(i==n-1)printf("breakdown already occurred!\n");
}
Nothing stops this loop from executing until i == n-1 . pthread_create() just fires off a thread to be run. It doesn't wait for it to start or end. Thus you're at the mercy of the scheduler, which might decide to continue executing your loop, or switch to one of the newly created threads (or do both, on a SMP system).
You're also initalizing the barrier to n, so in any case none of the threads will get past the barrier until you've created all of them.
In addition to the answers of nos and Starkey you have to take into account that you have another serialization in your code that is often neglected: you are doing IO on the same FILE variable, namely stdin.
The access to that variable is mutexed internally and the order in which your n+1 threads (including your calling thread) get access to that mutex is implementation defined, take it basically as random in your case.
So the order in which you get your printf output is the order in which your threads pass through these wormholes.
You can get the expected order in one of two ways
Create each thread with a higher priority than the main thread. This will ensure that new thread will run immediately after creation and wait on the barrier.
Move the "breakdown imminent!\n" print before the pthread_create() and call use a sched_yield() call after every pthread_create(). This will schedule the newly created thread for execution.