I'm currently learning concurrent programming. I have two threads and I want them to act like this:
the first thread runs N times while the second one waits
when the first one is done, the second does its job and the first waits
when the second thread is done, repeat.
I'm trying to do it in C using pthread library. Here's some pseudocode (hopefully understandable)
int cnt = 0;
void* thread1(){
while(1){
// thread 1 code
cnt ++;
if(cnt == N){
let_thread2_work();
wait_thread2();
}
}
}
void* thread2(){
while(1){
wait_thread1();
// thread2 code
cnt = 0;
let_thread1_work();
}
}
Can anyone please help me ?
Like David Schwartz commented, this doesn't make a ton of sense with just thread1 and thread2 waiting on each other, not actually doing any work in parallel. But maybe eventually you want multiple "thread1"s, all processing jobs at the same time until they finish a batch of N jobs, then they all stop and wait for "thread2" to do some kind of post-processing before the pool of worker threads start back up.
In this situation I would consider using a couple condition variables, one for your worker threads to communicate to your post-processing thread that they're waiting, and one for your post-processing thread to tell the workers to start working again. You can declare a condition variable and a helper mutex in the global scope right next to your "cnt" variable, which I'm calling "jobs_done" for clarity:
#include<pthread.h>
#DEFINE NUM_WORKERS 1 // although it doesn't make sense to just have 1
#DEFINE BATCH_SIZE 50 // this is "N"
// we're going to keep track of how many jobs we have done in
// this variable
int jobs_done = 0;
// when a worker thread checks jobs_done and it's N or greater,
// that means we have to wait for the post-processing thread to set
// jobs_done back to 0. so the worker threads "wait" on the
// condition variable, and the post-processing thread "broadcasts"
// to the condition variable to wake them all up again once it's
// done its work
pthread_cond_t jobs_ready_cv;
// we're going to use this helper mutex. whenever any thread
// reads or writes to the jobs_done variable, we have to lock this
// mutex. that includes the worker threads when they check to see if
// they're ready to wake up again.
pthread_mutex_t jobs_mx;
// here's how the worker threads will communicate to the post-process
// thread that a batch is done. to make sure that all N jobs are fully
// complete before postprocessing happens, we'll use this variable to
// keep track of how many threads are waiting for postprocessing to finish.
int workers_done = 0;
// we'll also use a separate condition variable and separate mutex to
// communicate to the postprocess thread.
pthread_cond_t workers_done_cv;
pthread_mutex_t workers_done_mx;
Then in your setup code, initialize the condition variables and helper mutexes:
int main() { // or something
pthread_cond_init(&jobs_ready_cv, NULL);
pthread_mutex_init(&jobs_mx, NULL);
pthread_cond_init(&workers_done_cv, NULL);
pthread_mutex_init(&workers_done_mx, NULL);
...
}
So, your worker threads (or "thread1"), before taking a job, will check to see how many jobs have been taken. If N (here, BATCH_SIZE) have been taken, then it updates a variable to indicate that it has no work left to do. If it finds that all of the worker threads are done, then it signals the postprocess thread ("thread2") through workers_done_cv. Then, the thread waits for a signal from the postprocess thread through `
void* worker_thread(){
while(1){
/* first, we check if the batch is complete. we do this first
* so we don't accidentally take an extra job.
*/
pthread_mutex_lock(&jobs_mx);
if (jobs_done == BATCH_SIZE) {
/* if BATCH_SIZE jobs have been done, first let's increment workers_done,
* and if all workers are done, let's notify the postprocess thread.
* after that, we release the workers_done mutex so the postprocess
* thread can wake up from the workers_done condition variable.
*/
pthread_mutex_lock(&workers_done_mx);
++workers_done;
if (workers_done == NUM_WORKERS) {
pthread_cond_broadcast(&workers_done_cv);
}
pthread_mutex_unlock(&workers_done_mx);
/* now we wait for the postprocess thread to do its work. this
* unlocks the mutex. when we get the signal to start doing jobs
* again, the mutex will relock automatically when we wake up.
*
* note that we use a while loop here to check the jobs_done
* variable after we wake up. That's because sometimes threads
* can wake up on accident even if no signal or broadcast happened,
* so we need to make sure that the postprocess thread actually
* reset the variable. google "spurious wakeups"
*/
while (jobs_done == BATCH_SIZE) {
pthread_cond_wait(&jobs_ready_cv, &jobs_mx);
}
}
/* okay, now we're ready to take a job.
*/
++jobs_done;
pthread_mutex_unlock(&jobs_mx);
// thread 1 code
}
}
Meanwhile, your postprocess thread waits on the workers_done_cv immediately, and doesn't wake up until the last worker thread is done and calls pthread_cond_broadcast(&workers_done_cv). Then, it does whatever it needs to, resets the counts, and broadcasts to the worker threads to wake them back up.
void* postprocess_thread(){
while(1){
/* first, we wait for our worker threads to be done
*/
pthread_mutex_lock(&workers_done_mx);
while (workers_done != NUM_WORKERS) {
pthread_cond_wait(&workers_done_cv, &workers_done_mx);
}
// thread2 code
/* reset count of stalled worker threads, release mutex */
workers_done = 0;
pthread_mutex_unlock(&workers_done_mx);
/* reset number of jobs done and wake up worker threads */
pthread_mutex_lock(&jobs_mx);
jobs_done = 0;
pthread_mutex_unlock(&jobs_mx);
pthread_cond_broadcast(&jobs_ready_cv);
}
}
Also take heed of David Schwartz's advice that you probably don't actually need the postprocessing thread to wait on the worker threads. If you don't need this, then you can get rid of the condition variable that makes the worker threads wait for the postprocess thread, and this implementation becomes a lot simpler.
edit: mutex protected the assignment to jobs_done in postprocess_thread(), added a forgotten ampersand
One solution is to use mutex:
https://www.geeksforgeeks.org/mutex-lock-for-linux-thread-synchronization/
Another solution, but using semaphore:
https://www.geeksforgeeks.org/difference-between-binary-semaphore-and-mutex/?ref=rp
Or creating own lightthread schedule.
for exemple:
http://www.dunkels.com/adam/pt/ [edited]
Related
In my program the main thread starts multiple child threads, locks a mutex for a counter and then sleeps until the counter reaches N.
In an infinite for loop, the other threads do some work and they increase the counter in each iteration alternately. Once the counter == N one of them has to signal the main thread and all of the child threads have to end.
My problem is that I'm not sure when I should lock the counter mutex and make a signal in the thread function, because once the counter becomes N I managed to wake up the main thread and exit one of the child threads, but the other threads will keep on trying to work when they should all be terminating.
My question is, how can I check if counter == N but send a signal by only one of the threads, and the others just return without any signalling?
Can this be done by only locking the mutex once in each iteration, for the time of checking its value (== N)? If yes, how?
void *thread_function(void *arg) {
int *id = (int *) arg;
for (;;) {
pthread_mutex_lock(&mutex_counter);
counter++;
if (counter == N) {
pthread_cond_signal(&cond);
pthread_mutex_unlock(&mutex_counter);
return NULL;
}
if (counter > N) {
pthread_mutex_unlock(&mutex_counter);
return NULL;
}
pthread_mutex_unlock(&mutex_counter);
sleep(random());
// do some work here
// ...
// finish work
}
The problem with above code is that, despite all threads terminating, they execute the loop one less iteration than needed because the counter is increased before the if clause (so counter will be N, but there were no N loops of work done).
However, if I put the counter to the end of the function, the threads will do more work than needed which messes up my program.
why you lock counter-mutex in main thread? Why you need to send signal from other treads to main? If you have only one global counter and all threads increasing it, you can just start in main thread all threads, remember their thread ids and then wait using while(counter<N); instruction. After all, just kill all threads with pthread_cancel.
Multiple threads will be “busy waiting” for the next_action variable to be set. Ideally one thread would call perform_action whenever the main sets it to a nonzero.
// choose a time-consuming activity based on action ...
void perform_action(int action);
int next_action = 0;
void* threadfunc(void*)
{
while (1)
{
while (next_action == 0);
int my_action = next_action;
next_action = 0;
perform_action(my_action);
}
}
int main()
{
// assume we've spawned threads executing threadfunc ...
while (1)
{
// determine what action should be dispatched to a thread next
next_action = read_action_from_user();
// exit the program
if (next_action == -1) { break; }
}
}
For this issue I would use a semaphore.
Using a while loop will waste processor time by constantly checking for changes in variable.
Here we can use synchronization methods that can alert when is available.
A semaphore has two options: acquire and release.
Main thread initially acquires the semaphore, and the thread is enqueued to acquire it. Thread will wait until semaphore becomes available.
When main thread sets it releases the semaphore to signal that a nonzero value has been set to it.
The thread will now wake up, acquire the semaphore, perform requested operation and release the semaphore.
When main thread needs to change the semaphore, it must acquire the semaphore again, set the and release the semaphore.
To acquire the semaphore, main thread must necessarily wait until the thread has finished.
I would not use a mutex because you also need a signaling mechanism to wake up your thread, and not only a protection for a shared variable.
See also Conditional Variable vs Semaphore
I would like to know what's the best way to wake up a number of threads at once with a loop
I thinked about using a pthread_cond but I don't need a mutex since I'm not waiting for a ressources
Theres's n threads with :
start of thread
loop
action
wait
end of loop
And then the main thread wwill notice them all to start working.
Is it better to use a pthread cond with a mutex per thread or is it better to recreate the threads each time the task is finished ?
You can indeed use a condition variable for this. The way it works is that your condition is over some shared variable that the main thread sets to indicate that the worker threads should continue. You do require a mutex because you do have a shared resource - the shared variable that the main thread sets and the worker threads read.
Something like:
/* shared variables */
int run_iteration = -1;
pthread_mutex_t iteration_lock = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t iteration_cond = PTHREAD_COND_INITIALIZER;
Worker threads:
int current_iteration = 0;
while (... )
{
/* Wait for main to signal start of current_iteration */
pthread_mutex_lock(&iteration_lock);
while (run_iteration < current_iteration)
pthread_cond_wait(&iteration_cond, &iteration_lock);
pthread_mutex_unlock(&iteration_lock);
/* ... Execute current_iteration ... */
current_iteration++;
}
Main thread:
/* signal start of next iteration */
pthread_mutex_lock(&iteration_lock);
run_iteration++;
pthread_cond_broadcast(&iteration_cond);
pthread_mutex_unlock(&iteration_lock);
Alternatively, if you need your threads to run in lockstep then a pthread barrier (pthread_barrier_t) could be used instead.
I have a function: createrWorkerPool which will spawn "n" worker threads and each of them will take input as file specified in the args for pthread_create, read the file modify a shared variable by using a mutex around it and wait at a barrier until all threads have done modifying their shared variable. This operation happens in a loop a large number of times.
The problem I am facing is that lets consider there are two files file1 and file2 and file2 is much bigger in size than file1. The barrier synchronization works till completion of file1- but since it finishes execution it no longer reaches the barrier and file2 is stuck in the barrier forever.
My question is that is there a way to dynamically change the count of the number of threads the barrier is waiting on when a thread exits. so in the above case if file1 finishes early it decrements the barrier count from 2 to 1 so file1 can proceed with its execution. I tried looking at the man page but don't see any function.Example code
pthread_mutex_t m1;
pthread_barrier_t b1;
//common function executed by each worker thread
void* doWork(void* arg) {
const char* file = arg;
while(1) {
pthread_mutex_lock(&m1);
// Modify shared variable
pthread_mutex_unlock(&m1);
// Wait for all threads to finish modifying shared variable
pthread_barrier_wait(&b1);
// Once all threads reach barrier check state of shared variable and do some task based on state
// check for condition if true break out of loop
}
return 0;
}
So basically thread1 manipulating file1 finishes before and thread2 is stuck at the barrier forever
You can't really change the barrier count while the barrier is in use like that.
Presumably the problem is that the condition that is tested to break out of the loop is not true for all files at the same time - ie, each thread might execute a different number of loops.
If this is the case, one solution is to have each thread that finishes early continue to loop around, but do nothing but wait on the barrier in each loop. Then arrange for the threads to all exit together - something like this:
void* doWork(void* arg)
{
const char* file = arg;
int work_done = 0;
while(1) {
if (work_done)
{
if (all_threads_done)
break;
pthread_barrier_wait(&b1);
continue;
}
pthread_mutex_lock(&m1);
// Modify shared variable
pthread_mutex_unlock(&m1);
// Wait for all threads to finish modifying shared variable
pthread_barrier_wait(&b1);
// Once all threads reach barrier check state of shared variable and do some task based on state
if (finish_condition)
work_done = 1;
}
return 0;
}
This code plays a sound clip by creating a thread to do it. When bleep() runs, it sets the global variable bleep_playing to TRUE. In its main loop, if it notices that bleep_playing has been set to FALSE, it terminates that loop, cleans up (closing files, freeing buffers), and exits. I don't know the correct way to wait for a detached thread to finish. pthread_join() doesn't do the job. The while loop here continually checks bleep_id to see if it's valid. When it isn't, execution continues. Is this the correct and portable way to tell a thread to clean up and terminate before the next thread is allowed to be created?
if (bleep_playing) {
bleep_playing = FALSE;
while (pthread_kill(bleep_id, 0) == 0) {
/* nothing */
}
}
err = pthread_create(&bleep_id, &attr, (void *) &bleep, &effect);
I
Hmm... pthread_join should do the job. As far as I remember the thread has to call pthread_exit...?
/* bleep thread */
void *bleep(void *)
{
/* do bleeping */
pthread_exit(NULL);
}
/* main thread */
if (pthread_create(&thread, ..., bleep, ...) == 0)
{
/*
** Try sleeping for some ms !!!
** I've had some issues on multi core CPUs requiring a sleep
** in order for the created thread to really "exist"...
*/
pthread_join(&thread, NULL);
}
Anyway if it isn't doing its thing you shouldn't poll a global variable since it will eat up your CPU. Instead create a mutex (pthread_mutex_*-functions) which is initially locked and freed by the "bleep thread". In your main thread you can wait for that mutex which makes your thread sleep until the "bleep thread" frees the mutex.
(or quick & and dirty: sleep for a small amount of time while waiting for bleep_playing becoming FALSE)