I would like to know what's the best way to wake up a number of threads at once with a loop
I thinked about using a pthread_cond but I don't need a mutex since I'm not waiting for a ressources
Theres's n threads with :
start of thread
loop
action
wait
end of loop
And then the main thread wwill notice them all to start working.
Is it better to use a pthread cond with a mutex per thread or is it better to recreate the threads each time the task is finished ?
You can indeed use a condition variable for this. The way it works is that your condition is over some shared variable that the main thread sets to indicate that the worker threads should continue. You do require a mutex because you do have a shared resource - the shared variable that the main thread sets and the worker threads read.
Something like:
/* shared variables */
int run_iteration = -1;
pthread_mutex_t iteration_lock = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t iteration_cond = PTHREAD_COND_INITIALIZER;
Worker threads:
int current_iteration = 0;
while (... )
{
/* Wait for main to signal start of current_iteration */
pthread_mutex_lock(&iteration_lock);
while (run_iteration < current_iteration)
pthread_cond_wait(&iteration_cond, &iteration_lock);
pthread_mutex_unlock(&iteration_lock);
/* ... Execute current_iteration ... */
current_iteration++;
}
Main thread:
/* signal start of next iteration */
pthread_mutex_lock(&iteration_lock);
run_iteration++;
pthread_cond_broadcast(&iteration_cond);
pthread_mutex_unlock(&iteration_lock);
Alternatively, if you need your threads to run in lockstep then a pthread barrier (pthread_barrier_t) could be used instead.
Related
I'm currently learning concurrent programming. I have two threads and I want them to act like this:
the first thread runs N times while the second one waits
when the first one is done, the second does its job and the first waits
when the second thread is done, repeat.
I'm trying to do it in C using pthread library. Here's some pseudocode (hopefully understandable)
int cnt = 0;
void* thread1(){
while(1){
// thread 1 code
cnt ++;
if(cnt == N){
let_thread2_work();
wait_thread2();
}
}
}
void* thread2(){
while(1){
wait_thread1();
// thread2 code
cnt = 0;
let_thread1_work();
}
}
Can anyone please help me ?
Like David Schwartz commented, this doesn't make a ton of sense with just thread1 and thread2 waiting on each other, not actually doing any work in parallel. But maybe eventually you want multiple "thread1"s, all processing jobs at the same time until they finish a batch of N jobs, then they all stop and wait for "thread2" to do some kind of post-processing before the pool of worker threads start back up.
In this situation I would consider using a couple condition variables, one for your worker threads to communicate to your post-processing thread that they're waiting, and one for your post-processing thread to tell the workers to start working again. You can declare a condition variable and a helper mutex in the global scope right next to your "cnt" variable, which I'm calling "jobs_done" for clarity:
#include<pthread.h>
#DEFINE NUM_WORKERS 1 // although it doesn't make sense to just have 1
#DEFINE BATCH_SIZE 50 // this is "N"
// we're going to keep track of how many jobs we have done in
// this variable
int jobs_done = 0;
// when a worker thread checks jobs_done and it's N or greater,
// that means we have to wait for the post-processing thread to set
// jobs_done back to 0. so the worker threads "wait" on the
// condition variable, and the post-processing thread "broadcasts"
// to the condition variable to wake them all up again once it's
// done its work
pthread_cond_t jobs_ready_cv;
// we're going to use this helper mutex. whenever any thread
// reads or writes to the jobs_done variable, we have to lock this
// mutex. that includes the worker threads when they check to see if
// they're ready to wake up again.
pthread_mutex_t jobs_mx;
// here's how the worker threads will communicate to the post-process
// thread that a batch is done. to make sure that all N jobs are fully
// complete before postprocessing happens, we'll use this variable to
// keep track of how many threads are waiting for postprocessing to finish.
int workers_done = 0;
// we'll also use a separate condition variable and separate mutex to
// communicate to the postprocess thread.
pthread_cond_t workers_done_cv;
pthread_mutex_t workers_done_mx;
Then in your setup code, initialize the condition variables and helper mutexes:
int main() { // or something
pthread_cond_init(&jobs_ready_cv, NULL);
pthread_mutex_init(&jobs_mx, NULL);
pthread_cond_init(&workers_done_cv, NULL);
pthread_mutex_init(&workers_done_mx, NULL);
...
}
So, your worker threads (or "thread1"), before taking a job, will check to see how many jobs have been taken. If N (here, BATCH_SIZE) have been taken, then it updates a variable to indicate that it has no work left to do. If it finds that all of the worker threads are done, then it signals the postprocess thread ("thread2") through workers_done_cv. Then, the thread waits for a signal from the postprocess thread through `
void* worker_thread(){
while(1){
/* first, we check if the batch is complete. we do this first
* so we don't accidentally take an extra job.
*/
pthread_mutex_lock(&jobs_mx);
if (jobs_done == BATCH_SIZE) {
/* if BATCH_SIZE jobs have been done, first let's increment workers_done,
* and if all workers are done, let's notify the postprocess thread.
* after that, we release the workers_done mutex so the postprocess
* thread can wake up from the workers_done condition variable.
*/
pthread_mutex_lock(&workers_done_mx);
++workers_done;
if (workers_done == NUM_WORKERS) {
pthread_cond_broadcast(&workers_done_cv);
}
pthread_mutex_unlock(&workers_done_mx);
/* now we wait for the postprocess thread to do its work. this
* unlocks the mutex. when we get the signal to start doing jobs
* again, the mutex will relock automatically when we wake up.
*
* note that we use a while loop here to check the jobs_done
* variable after we wake up. That's because sometimes threads
* can wake up on accident even if no signal or broadcast happened,
* so we need to make sure that the postprocess thread actually
* reset the variable. google "spurious wakeups"
*/
while (jobs_done == BATCH_SIZE) {
pthread_cond_wait(&jobs_ready_cv, &jobs_mx);
}
}
/* okay, now we're ready to take a job.
*/
++jobs_done;
pthread_mutex_unlock(&jobs_mx);
// thread 1 code
}
}
Meanwhile, your postprocess thread waits on the workers_done_cv immediately, and doesn't wake up until the last worker thread is done and calls pthread_cond_broadcast(&workers_done_cv). Then, it does whatever it needs to, resets the counts, and broadcasts to the worker threads to wake them back up.
void* postprocess_thread(){
while(1){
/* first, we wait for our worker threads to be done
*/
pthread_mutex_lock(&workers_done_mx);
while (workers_done != NUM_WORKERS) {
pthread_cond_wait(&workers_done_cv, &workers_done_mx);
}
// thread2 code
/* reset count of stalled worker threads, release mutex */
workers_done = 0;
pthread_mutex_unlock(&workers_done_mx);
/* reset number of jobs done and wake up worker threads */
pthread_mutex_lock(&jobs_mx);
jobs_done = 0;
pthread_mutex_unlock(&jobs_mx);
pthread_cond_broadcast(&jobs_ready_cv);
}
}
Also take heed of David Schwartz's advice that you probably don't actually need the postprocessing thread to wait on the worker threads. If you don't need this, then you can get rid of the condition variable that makes the worker threads wait for the postprocess thread, and this implementation becomes a lot simpler.
edit: mutex protected the assignment to jobs_done in postprocess_thread(), added a forgotten ampersand
One solution is to use mutex:
https://www.geeksforgeeks.org/mutex-lock-for-linux-thread-synchronization/
Another solution, but using semaphore:
https://www.geeksforgeeks.org/difference-between-binary-semaphore-and-mutex/?ref=rp
Or creating own lightthread schedule.
for exemple:
http://www.dunkels.com/adam/pt/ [edited]
Multiple threads will be “busy waiting” for the next_action variable to be set. Ideally one thread would call perform_action whenever the main sets it to a nonzero.
// choose a time-consuming activity based on action ...
void perform_action(int action);
int next_action = 0;
void* threadfunc(void*)
{
while (1)
{
while (next_action == 0);
int my_action = next_action;
next_action = 0;
perform_action(my_action);
}
}
int main()
{
// assume we've spawned threads executing threadfunc ...
while (1)
{
// determine what action should be dispatched to a thread next
next_action = read_action_from_user();
// exit the program
if (next_action == -1) { break; }
}
}
For this issue I would use a semaphore.
Using a while loop will waste processor time by constantly checking for changes in variable.
Here we can use synchronization methods that can alert when is available.
A semaphore has two options: acquire and release.
Main thread initially acquires the semaphore, and the thread is enqueued to acquire it. Thread will wait until semaphore becomes available.
When main thread sets it releases the semaphore to signal that a nonzero value has been set to it.
The thread will now wake up, acquire the semaphore, perform requested operation and release the semaphore.
When main thread needs to change the semaphore, it must acquire the semaphore again, set the and release the semaphore.
To acquire the semaphore, main thread must necessarily wait until the thread has finished.
I would not use a mutex because you also need a signaling mechanism to wake up your thread, and not only a protection for a shared variable.
See also Conditional Variable vs Semaphore
We have studied that if we dealing with multi-threaded problems then we use one of thread synchronizing method called mutex which allows to lock critical section so that other threads do not interfere and goes into block state until mutex unlocks the critical section.
But I am doing this thing in my program but the output of this program is not matches with the concept of mutex. If I am wrong so correct me.
Code
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <signal.h>
#include <pthread.h>
#define MAX 10
pthread_mutex_t the_mutex;
pthread_cond_t condc, condp;
int toConsume=0;
int i;
void* consumer(void *ptr) {
pthread_mutex_lock(&the_mutex);
while(i<MAX)
{
/* protect buffer */
while (toConsume <= 0) /* If there is nothing in the buffer then wait */
{
printf("Waiting Thread id:%lu \n",pthread_self());
pthread_cond_wait(&condc, &the_mutex);
}
pthread_mutex_unlock(&the_mutex); /* release the buffer */
sleep(2);
pthread_mutex_lock(&the_mutex); /* protect buffer */
toConsume--;
i++;
}
pthread_mutex_unlock(&the_mutex); /* release the buffer */
pthread_exit(0);
}
int main(int argc, char **argv) {
pthread_t pro, con[3];
pthread_mutex_init(&the_mutex, NULL);
pthread_cond_init(&condc, NULL); /* Initialize consumer condition variable */
pthread_cond_init(&condp, NULL); /* Initialize producer condition variable */
// Create the threads
for(int i=0 ;i<3;i++)
pthread_create(&con[i], NULL, consumer, NULL);
for(int i=0 ;i<3;i++)
pthread_join(con[i], NULL);
return 0;
}
OUTPUT
$ ./ex
Waiting Thread id:140580582618880
Waiting Thread id:140580574226176
Waiting Thread id:140580565833472
All thread enters into critical section even the mutex remain its lock.
The function pthread_cond_wait will unlock the held mutex while the thread is waiting. This allows another thread to enter the critical section.
The purpose of using pthread_cond_wait is that the thread needs to wait for some condition to become true before really performing the work inside the critical section. To test the condition in the first place requires locking the mutex. But, if the condition is false, it must wait for some event to make the condition true. If it waits with the lock held, no other thread will be able to update the state for the condition to become true, since updating the condition will also require locking the same mutex.
Thus, when waiting on a condition variable, the mutex is unlocked to allow another thread to grab the lock to perform something that would make the condition true.
As an example, consider a job queue. A thread will lock a mutex to get a job from the queue. However, if the queue is empty, it must wait for a job to appear in the queue. This is the condition that is must wait on, and a condition variable can be used for that purpose. When it calls pthread_cond_wait on the condition variable, the associated mutex is unlocked.
Another thread wishes to place a job into the queue. That thread can lock the mutex, add the job to the queue, signal the condition variable, then unlock the mutex.
When the condition variable is signaled, the waiting thread is woken up and pthread_cond_wait returns with the lock on the mutex held again. It detects that the queue is non-empty, and can enter the critical section of dequeueing a job from the queue.
I am trying to better understand how to use pthread_cond_wait() and how it works.
I am just looking for a bit of clarification to an answer I saw on this site.
The answer is the last reply on this page
understanding of pthread_cond_wait() and pthread_cond_signal()
I am wondering how this would look with three threads. Imagine Thread 1 wants to tell Thread 2 and Thread 3 to wake up
For example
pthread_mutex_t mutex;
pthread_cond_t condition;
Thread 1:
pthread_mutex_lock(&mutex);
/*Initialize things*/
pthread_mutex_unlock(&mutex);
pthread_cond_signal(&condition); //wake up thread 2 & 3
/*Do other things*/
Thread 2:
pthread_mutex_lock(&mutex); //mutex lock
while(!condition){
pthread_cond_wait(&condition, &mutex); //wait for the condition
}
pthread_mutex_unlock(&mutex);
/*Do work*/
Thread 3:
pthread_mutex_lock(&mutex); //mutex lock
while(!condition){
pthread_cond_wait(&condition, &mutex); //wait for the condition
}
pthread_mutex_unlock(&mutex);
/*Do work*/
I am wondering if such a setup is valid. Say that Thread 2 and 3 relied on some intialization options that thread 1 needs to process.
First: If you wish thread #1 to wake up thread #2 and #3, it should use pthread_cond_broadcast.
Second: The setup is valid (with broadcast). Thread #2 and #3 are scheduled for wakeup and they will try to reacquire the mutex as part of waking up. One of them will, the other will have to wait for the mutex to be unlocked again. So thread #2 and #3 access the critical section sequentically (to re-evaluate the condition).
If I understand correctly, you want thr#2 and thr#3 ("workers") to block until thr#1 ("boss") has performed some initialization.
Your approach is almost workable, but you need to broadcast rather than signal, and are missing a predicate variable separate from your condition variable. (In the question you reference, the predicate and condition variables were very similarly named.) For example:
pthread_mutex_t mtx;
pthread_cond_t cv;
int initialized = 0; // our predicate of interest, signaled via cv
...
// boss thread
initialize_things();
pthread_mutex_lock(&mtx);
initialized = 1;
pthread_cond_broadcast(&cv);
pthread_mutex_unlock(&mtx);
...
// worker threads
pthread_mutex_lock(&mtx);
while (! initialized) {
pthread_cond_wait(&cv, &mtx);
}
pthread_mutex_unlock(&mtx);
do_things();
That's common enough that you might want to combine the mutex/cv/flag into a single abstraction. (For inspiration, see Python's Event object.) POSIX barriers which are another way of synchronizing threads: every thread waits until all threads have "arrived." pthread_once is another way, as it runs a function once and only once no matter how many threads call it.
pthread_cond_signal wakes up one (random) thread waiting on the cond variable. If you want to wake up all threads waiting on this cond variable use pthread_cond_broadcast.
Depending what you are doing on the critical session there might be also another solution apart from the ones in the previous answers.
Suppose thread1 is executing first (i.e. it is the creator thread) and suppose thread2 and thread3 do not perform any write in to shared resource in the critical session. In this case with pthread_cond_wait you are forcing one thread to wait the other when actually there is no need.
You can use read-write mutex of type pthread_rwlock_t. Basically the thread1 performs a write-lock so the other threads will be blocked when trying to acquire a read-lock.
The functions for this lock are quite self-explanatory:
//They return: 0 if OK, error number on failure
int pthread_rwlock_init(pthread_rwlock_t *restrict rwlock,
const pthread_rwlockattr_t *restrict attr);
int pthread_rwlock_destroy(pthread_rwlock_t *rwlock);
int pthread_rwlock_rdlock(pthread_rwlock_t *rwlock);
int pthread_rwlock_wrlock(pthread_rwlock_t *rwlock);
int pthread_rwlock_unlock(pthread_rwlock_t *rwlock);
int pthread_rwlock_tryrdlock(pthread_rwlock_t *rwlock);
int pthread_rwlock_trywrlock(pthread_rwlock_t *rwlock);
When thread1 has finished its initialization it unlocks. The other threads will perform a read-lock and since more read-locks can co-exist they can execute simultaneously. Again: this is valid if you do not perform any write in thread2&3 on the shared resources.
I'm trying to implement a blocking queue using POSIX threads. Important code segments are shown below. I tried to run this program. The thread trying to remove an element from the queue goes to sleep when there are no elements in the queue and wakes up again without any signal from the thread that adds an element into the queue (This I conclude because I did not start any thread that adds an element into the queue). The thread woke up again goes to sleep and this process repeats. What am I doing wrong? Please some one tell me what I am missing here?
struct rqueue
{
int qsize;
int capacity;
pthread_mutex_t lock;
pthread_cond_t not_empty;
pthread_cond_t not_full;
};
remove_element_method:
pthread_mutex_lock(&rq->lock);
while(rq->qsize == 0){
perror("Q size is zero going to sleep");
pthread_cond_wait(&rq->not_empty);
perror("woke up");
}
// some code
pthread_cond_signal(&rq->not_full);
pthread_mutex_unlock(&rq->lock);
add_element_method:
pthread_mutex_lock(&rq->lock);
if(rq->capacity != -1 ){
while(rq->qsize == rq->capacity){
pthread_cond_wait(&rq->not_full);
}
}
//some code
pthread_cond_signal(&rq->not_empty);
pthread_mutex_unlock(&rq->lock);
pthread_cond_wait() takes two arguments -- the second is the mutex you're holding. You're only passing it one argument.
Also, did you initialize the mutex and condition variables using pthread_mutex_init() and pthread_cond_init()?