Unique semaphore for each thread - c

I have been assigned a modified version of the "Santa Claus" semaphore problem.
Santa Claus is a thread that randomly wakes up to check how many and what kind of threads are waiting to report to him (Work elves and Collecting elves)
What I have made are work elf and collecting elf threads:
void *collectingElf(void *arg, int semaphoreIndex) {
while (1) {
sem_wait(&elveCountMutex); //semaphore for critical section, the number of elves
printf("\nCollecting! %d\n", (int) pthread_self()); // thread is collecting stuff
sleep((unsigned int) (rand() % 4)); // thread sleeps for a random amount of time
printf("Done collecting! %d\n", (int) pthread_self()); // Print process ID, just for easier tracking
sem_post(&elveCountMutex); // Release the elve count semaphore
sem_wait(&collectingElveSem);
}
}
void *workingElf(void *arg) //same as collecting elf
{
while (1) {
sem_wait(&elveCountMutex);
printf("\nWorking! %d\n", pthread_self());
sleep(1);
workingElveCount++;
printf("Done working! %d\n", pthread_self());
sem_wait(&workElfSem);
sem_post(&elveCountMutex);
}
}
So here the elve counts are protected, since the threads can only access the counters when elveCountMutex is locked. This I can understand and this seems logical. Afterwards the thread should block and wait for Santa to unblock it.
So, from what I read, once semaphore reaches value 0, thread will block. Anything above 0 will not block it, and negative value indicates how many threads are waiting for the semaphore to be unlocked.
So, once the threads running are finished, they decrement the semaphore, and block.
However, what I cannot seem to grasp is this part of the assignment:
For a collecting meeting to start, at least one work elf and three collecting elves are necessary. • If enough elves are present that both meetings could start, the collecting-meeting
always has priority, and all work-elves not needed anymore resume their work.
Say if I have 3 work elves, and I need only 1, how do I release the remaining 2 threads? Do I need a separate semaphore for each thread? Or am I missing something?
Edit: My bad, I completely forgot to tell about the Santa implementation.
Santa wakes up and releases the semaphores, in this way:
void* Santa(void *arg)
{
while (1) {
sleep((unsigned)rand() % 4); //Santa sleeps randomly between 0 and 3 seconds;
sem_wait(&elveCountMutex); //access both elf counters
if(workingElveCount>=2 && collectingElveCount >= 3) //decide which meeting should commence
{
int releaseWorkElveCount = workingElveCount-1;
for(int i = 0;i<releaseWorkElveCount;i++)
{
sem_post(&workElfSem);
}
sleep(5);
collectingMeeting(&collectingMeetingThread); //This just prints that we are in a collecting meeting
pthread_join(collectingMeetingThread,0);
sem_wait(&elveCountMutex);
for(int i=0;i<workingElveCount;i++)
{
sem_post(&workElfSem);
}
for(int i=0;i<collectingElveCount;i++)
{
sem_post(&collectingElveSem);
}
workingElveCount=0;
collectingElveCount=0;
}
}

I do not understand your management of semaphore
In
void *collectingElf(void *arg, int semaphoreIndex) {
while (1) {
...
sem_wait(&collectingElveSem);
}
}
that gets but never releases collectingElveSem, and that in an infinite loop ?
In
void *workingElf(void *arg) //same as collecting elf
{
while (1) {
sem_wait(&elveCountMutex);
...
sem_wait(&workElfSem);
}
}
that get but never release elveCountMutex and workElfSem, and that in an infinite loop. collectingElf also (try to) get elveCountMutex but I will not able to do after one turn in workingElf
If your semaphore are not recursive workingElf will also be blocked after one turn because not able to get again the semaphores. If the semaphore are recursive this cannot be for an infinite deep and workingElf will block itself after enough loops

Related

Synchronising threads in a loop

I want each thread to synchronize at the end of every loop. I have a condition variable at the end, which sends the thread to sleep if the other threads have not reached the pseudo-barrier at the end of the thread. I keep getting a deadlock. Can you help me spot my mistake?
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <pthread.h>
pthread_cond_t continue_cond;
pthread_mutex_t continue_mut;
pthread_mutex_t waiting_threads_mut;
int num_waiting_threads = 0;
pthread_mutex_t working_threads_mut;
int num_working_threads = 0;
int AllThreadsHere() {
pthread_mutex_lock(&waiting_threads_mut);
pthread_mutex_lock(&working_threads_mut);
//printf("%d: %d\n", num_waiting_threads, num_working_threads);
int res = (num_waiting_threads == num_working_threads) ? 1 : 0;
pthread_mutex_unlock(&working_threads_mut);
pthread_mutex_unlock(&waiting_threads_mut);
return res;
}
// used to
void WorkerProcess(int* thread_id) {
int to_process_indices = 1000;
while (to_process_indices > 0) {
// do computation here
to_process_indices -= *(thread_id + 1);
//
pthread_cond_broadcast(&continue_cond);
// increment number of waiting threads
pthread_mutex_lock(&waiting_threads_mut);
++num_waiting_threads;
pthread_mutex_unlock(&waiting_threads_mut);
// this mutex is necessary so as to make the process of sleeping, and decrementing the number of waiting threads atomic.
// note that if a thread wakes up from sleeping, then this mutex is locked again, meaning, the process of decrementing the number of working threads cannot occur without it unlocking.
// this is very important, as an incoming thread may otherwise just finish its chunk
pthread_mutex_lock(&continue_mut);
while (AllThreadsHere() == 0) {
//printf("Thread %d sleeping\n", args->thread_id);
// waits for signal from incoming threads.
pthread_cond_wait(&continue_cond, &continue_mut);
//printf("Thread %d woken\n", args->thread_id);
}
pthread_mutex_unlock(&continue_mut);
// need to decrease the number of waiting threads.
pthread_mutex_lock(&waiting_threads_mut);
--num_waiting_threads;
pthread_mutex_unlock(&waiting_threads_mut);
}
pthread_mutex_lock(&continue_mut);
pthread_mutex_lock(&working_threads_mut);
--num_working_threads;
pthread_cond_broadcast(&continue_cond);
pthread_mutex_unlock(&working_threads_mut);
pthread_mutex_unlock(&continue_mut);
}
and here is my int main which simply initialises the mutexes, pthreads, and joins the launched threads at the end.
int main() {
const unsigned int NUM_THREADS = 3;
const double PRECISION = 0.1;
// make the space for worker threads.
pthread_t* worker_threads = malloc(NUM_THREADS * sizeof(pthread_t));
int* worker_ids = malloc(sizeof(int) * NUM_THREADS);
pthread_cond_init(&continue_cond, NULL);
pthread_mutex_init(&waiting_threads_mut, NULL);
pthread_mutex_init(&working_threads_mut, NULL);
pthread_mutex_init(&continue_mut, NULL);
for (unsigned int k = 0; k < NUM_THREADS; ++k) {
worker_ids[k] = k;
pthread_create(worker_threads + k, NULL, WorkerProcess, (void*)(worker_ids + k));
}
for (unsigned int k = 0; k < NUM_THREADS; ++k) {
pthread_join(worker_threads[k], NULL);
}
}
Throwing code together and moving it around until it seems to work is not software development, it is gambling.
Instead, consider the behaviour you want from your barrier.
In my opinion, the barrier needs a condition variable that threads can wait on, until the number of threads reaches the set number of threads, at which point all waiting threads are woken up and released to proceed.
If the iteration is fast enough, then it is possible that the first-released thread arrives again at the barrier before all threads waiting in the barrier have released. So, we need a second condition variable for such incoming threads, plus at least one flag to indicate when the barrier is still being released.
Let's create a structure to describe such a barrier. Because linux lets us initialize mutexes and condition variables statically, we'll also define a static initializer.
#include <stdlib.h>
#include <pthread.h>
#include <stdio.h>
enum {
BARRIER_RELEASED = (1 << 0),
};
typedef struct {
pthread_mutex_t lock;
// Threads that arrive at the barrier before it has released
// all threads blocked in it, will wait on 'incoming'.
pthread_cond_t incoming;
// Threads blocked on the barrier wait on 'waiting'.
pthread_cond_t waiting;
// Number of threads in the barrier, or release trigger limit.
int limit;
// Number of threads blocked on the barrier (waiting on 'waiting').
int count;
// Barrier state flags, one bit per flag. See BARRIER_ flag enums.
int state;
} barrier;
#define BARRIER_INITIALIZER(limit_) \
{ .lock = PTHREAD_MUTEX_INITIALIZER, \
.incoming = PTHREAD_COND_INITIALIZER, \
.waiting = PTHREAD_COND_INITIALIZER, \
.limit = limit_, \
.count = 0, \
.state = 0 }
Since we do not necessarily know the number of threads participating in the barrier at compile time, let's also define an init function:
void barrier_init(barrier *b, int limit)
{
if (!b) {
fprintf(stderr, "barrier_init(): No barrier (NULL) specified.\n");
exit(EXIT_FAILURE);
}
if (limit < 0) {
fprintf(stderr, "barrier_init(): Negative limit (%d) specified.\n", limit);
exit(EXIT_FAILURE);
}
pthread_mutex_init(&(b->lock), NULL); // Cannot fail in Linux
pthread_cond_init(&(b->incoming), NULL); // Cannot fail in Linux
pthread_cond_init(&(b->waiting), NULL); // Cannot fail in Linux
b->limit = limit;
b->count = 0;
b->state = 0;
}
In both the initializer and the init function, the limit is the (expected) number of threads participating in the barrier.
(If we have each thread register itself in the barrier, then the first thread might iterate several times by itself while the barrier thread count is 1, before any other threads have a chance of registering themselves also.)
Whenever a thread no longer wants to participate in a barrier, it needs to "leave", so that the other threads waiting in the barrier won't wait there forever for the missing thread.
// When a thread no longer participates in a barrier, it needs to leave,
// so that the rest of the threads can keep gathering at the barrier without hanging.
void barrier_leave(barrier *b)
{
if (!b) {
fprintf(stderr, "barrier_leave(): No barrier (NULL) specified.\n");
exit(EXIT_FAILURE);
}
pthread_mutex_lock(&(b->lock));
b->limit--;
// Decreasing the limit may release the threads waiting on the barrier!
if (!(b->state & BARRIER_RELEASED) && b->count >= b->limit) {
b->state |= BARRIER_RELEASED;
pthread_cond_broadcast(&(b->waiting));
}
pthread_mutex_unlock(&(b->lock));
}
For completeness, we can define a function so that if one does not know the limit beforehand, one can init the barrier to an impossibly large number of threads, say SIZE_MAX/2, and then re-set the limit to the actual number of threads. This way, the created threads will start normally but wait at the barrier:
void barrier_set_limit(barrier *b, int limit)
{
if (!b) {
fprintf(stderr, "barrier_set_limit(): No barrier (NULL) specified.\n");
exit(EXIT_FAILURE);
}
if (limit < 0) {
fprintf(stderr, "barrier_set_limit(): Invalid, negative limit (%d) set.\n", limit);
exit(EXIT_FAILURE);
}
pthread_mutex_lock(&(b->lock));
b->limit = limit;
// Decreasing the limit may release the threads waiting on the barrier!
if (!(b->state & BARRIER_RELEASED) && b->count >= b->limit) {
b->state |= BARRIER_RELEASED;
pthread_cond_broadcast(&(b->waiting));
}
pthread_mutex_unlock(&(b->lock));
}
The final function left is the waiting at the barrier.
// Wait/gather at a barrier.
void barrier_wait(barrier *b)
{
if (!b) {
fprintf(stderr, "barrier_wait(): No barrier (NULL) specified.\n");
exit(EXIT_FAILURE);
}
pthread_mutex_lock(&(b->lock));
// Wait, if the barrier is being released.
if (b->state & BARRIER_RELEASED)
pthread_cond_wait(&(b->incoming), &(b->lock));
b->count++;
if (b->count >= b->limit) {
// We filled the barrier: release.
b->state |= BARRIER_RELEASED;
pthread_cond_broadcast(&(b->waiting));
} else {
// Barrier wasn't full, so we wait.
pthread_cond_wait(&(b->waiting), &(b->lock));
}
// If we are the last thread out of the barrier, we need to update
// the barrier state, and let the incoming threads advance.
b->count--;
if (b->count <= 0) {
b->state &= ~(BARRIER_RELEASED);
pthread_cond_broadcast(&(b->incoming));
}
pthread_mutex_unlock(&(b->lock));
}
Quick testing indicates the above indeed works, as long as the barrier is first initialized to the number of threads, and each thread participating in the barrier calls barrier_leave() before it exits.
In other words, you have a global, say static barrier turnstile;, and before creating say n threads, you call barrier_init(&turnstile, n);. Within the thread worker function, inside your loop, you call barrier_wait(&turnstile); to synchronize those threads. If one of those threads wants or needs to exit, it should call barrier_leave(&turnstile); first.
As you can see, the code looks nothing like yours. Everything related to a barrier is contained within the barrier structure, and all functions needed to manipulate barriers start with a barrier_ prefix, and should be straightforward to understand.
The entire process here started at thinking about "What do I need?", then implementing each need step by step. While I was writing this answer, I did need to "go back one step", because I didn't initially remember that an incoming condition queue is also needed, in case the first released thread(s) reach the same barrier again before all waiting threads in it have been released. But, because I went about it constructively, one step at a time (instead of throwing a lot of code together and then trying to see if it compiles and works), it wasn't a huge change –– even though for simplicity I rewrote the functions from scratch, only keeping the error checks.

Conditional Variables with multithreading

I am working on the dining philosophers problem, where n philosophers take turns thinking and eating. I would like to have a version of this where the philosophers will eat in the order of their id: 0,1,2,3,4...,but my threads keep getting blocked. My threads start by calling PhilosopherThread().
void putDownChopsticks(int threadIndex){
//finished eating
pindex++;
pthread_cond_signal(&cond);
pthread_mutex_unlock(&lock);
}
void pickUpChopsticks(int threadIndex){
pthread_mutex_lock(&lock);
while(pindex != threadIndex){
pthread_cond_wait(&cond, &lock);
}
//lets go eat
}
void eating(){
//put thread to sleep
}
void thinking(){
//put thread to sleep
}
void* PhilosopherThread(void *x){
int *index = x;
thinking(); //just puts thread to sleep to simulate thinking
pickUpChopsticks(*index);
eating(); //just puts thread to sleep to simulate eating
putDownChopsticks(*index);
return NULL;
}
I'm having a bit of trouble trying to get the philosophers in order. I can only get the first 2 threads to eat before the threads get blocked.
Edit: as far as i know im doing this right. I first lock the mutex, then I check if pindex is the current thread id, if its not the thread will wait until pindex does equal the id. Then the thread can go eat and once where done, we incread pindex, signal that the thread is done, and unlock the mutex.
This code sometimes works and sometimes does not. First, since you did not provide a complete program, here are the missing bits I used for testing purposes:
#include <stdlib.h>
#include <pthread.h>
static pthread_cond_t cond;
static pthread_mutex_t lock;
static pindex;
/* ... your code ... */
int main () {
int id[5], i;
pthread_t tid[5];
for (i = 0; i < 5; ++i) {
id[i] = i;
pthread_create(tid+i, 0, PhilosopherThread, id+i);
}
for (i = 0; i < 5; ++i) pthread_join(tid[i], 0);
exit(0);
}
The critical piece to notice is how you wake up the next philosopher:
pthread_cond_signal(&cond);
This call will only wake up one thread. But, which thread is at the discretion of the OS. Therefore, if it does not happen to wake up the philosopher that is supposed to wake up, no other philosopher is woken up.
A simple fix would be to wake up all waiting threads instead of just one. The philosophers that don't match will go back to waiting, and the one that is supposed to go next will go.
pthread_cond_broadcast(&cond);
However, since each thread knows which philosopher should wake up, you could change your solution to allow that to happen. One way could be to implement a separate condition variable per philosopher, and use pthread_cond_signal() on the next philosopher's condition variable.

While loop synchronization

I am working on a project with a user defined number of threads I am using 7 at the moment. I have a while loop that runs in each thread but I need all of the threads to wait for each other at the end of the while loop. The tricky thing is that some of the threads do not all end on the same number of times through the loop.
void *entryFunc(void *param)
{
int *i = (int *)param;
int nextPrime;
int p = latestPrime;
while(latestPrime < testLim)
{
sem_wait(&sem);
nextPrime = findNextPrime(latestPrime);
if(nextPrime != -1)
{
latestPrime = nextPrime;
p = latestPrime;
}
else
{
sem_post(&sem);
break;
}
sem_post(&sem);
if(p < 46341)
{
incrementNotPrimes(p);
}
/*
sem_wait(&sem2);
doneCount++;
sem_post(&sem2);
while(go != 1);
sem_wait(&sem2);
doneCount--;
//sem_post(&sem3);
sem_post(&sem2);
*/
}
return NULL;
}
where the chunk of code is commented out is part of my last attempt at solving this problem. That is where the functions all need to wait for each other. I have a feeling I am missing something simple.
If your problem is that on each thread, the while loop has a different numbers of iterations and some threads never reach the synchronization point after exiting the loop, you could use a barrier. Check here for an example.
However you need to decrease the number of threads at the barrier after you exit each thread. Waiting at the barrier will end after count number of threads reached that point.
So you need to update the barrier object each time a thread finishes. And make sure you do this atomically.
As I mentioned in the comments, you should use a barrier instead of a semaphore for this kind of situation, as it should be simpler to implement (barriers have been designed exactly to solve that problem). However, you may still use a semaphore with a little bit of arithmetic
arithmetic: your goal is to have all thread execute the same code path, but somehow the last thread to finish its task should wake all the other threads up. One way to achieve that is to have at the end of the function an atomic counter which each thread would decrement, and if the counter reaches 0, the thread simply calls as many time sem_post as necessary to release all the waiting threads, instead of issuing a sem_wait as the others.
A second method, this time using only a semaphore, is also possible. Since we cannot differentiate the last thread for the others, all the threads must do the same operations with the semaphore, ie try to release everyone, but also wait for the last one. So the idea is to initialize the semaphore to (1-n)*(n+1), so that each of the n-1 first threads would fail at waking up their friends with n+1 calls to sem_post, but still work toward getting the semaphore at exactly 0. Then the last thread would do the same, pushing the semaphore value to n+1, thus releasing the locked threads, and leaving room for it to also perform its sem_wait and be released immediately.
void *entryFunc(void *param)
{
int *i = (int *)param;
int nextPrime;
int p = latestPrime, j;
while(latestPrime < testLim){
nextPrime = findNextPrime(latestPrime);
if(nextPrime != -1)
{
latestPrime = nextPrime;
p = latestPrime;
}
if(p < 46341)
{
incrementNotPrimes(p);
}
}
for (j=0;j<=THREAD_COUNT;j++)
sem_post(&sem);
sem_wait(&sem);
return NULL;
}
The problem with this approach is that it doesn't deal with how the semaphore should be reset in between uses (if your program needs to repeat this mechanism, it will need to reset the semaphore value, since it will end up being 1 after this code has been executed successfully).

The version of pthread_join() that does not block main(): POSIX

I am trying to write a code that does not block main() when pthread_join() is called:
i.e. basically trying to implement my previous question mentioned below:
https://stackoverflow.com/questions/24509500/pthread-join-and-main-blocking-multithreading
And the corresponding explanation at:
pthreads - Join on group of threads, wait for one to exit
As per suggested answer:
You'd need to create your own version of it - e.g. an array of flags (one flag per thread) protected by a mutex and a condition variable; where just before "pthread_exit()" each thread acquires the mutex, sets its flag, then does "pthread_cond_signal()". The main thread waits for the signal, then checks the array of flags to determine which thread/s to join (there may be more than one thread to join by then).
I have tried as below:
My status array which keeps a track of which threads have finished:
typedef struct {
int Finish_Status[THREAD_NUM];
int signalled;
pthread_mutex_t mutex;
pthread_cond_t FINISHED;
}THREAD_FINISH_STATE;
The thread routine, it sets the corresponding array element when the thread finishes and also signals the condition variable:
void* THREAD_ROUTINE(void* arg)
{
THREAD_ARGUMENT* temp=(THREAD_ARGUMENT*) arg;
printf("Thread created with id %d\n",temp->id);
waitFor(5);
pthread_mutex_lock(&(ThreadFinishStatus.mutex));
ThreadFinishStatus.Finish_Status[temp->id]=TRUE;
ThreadFinishStatus.signalled=TRUE;
if(ThreadFinishStatus.signalled==TRUE)
{
pthread_cond_signal(&(ThreadFinishStatus.FINISHED));
printf("Signal that thread %d finished\n",temp->id);
}
pthread_mutex_unlock(&(ThreadFinishStatus.mutex));
pthread_exit((void*)(temp->id));
}
I am not able to write the corresponding parts pthread_join() and pthread_cond_wait() functions. There are a few things which I am not able to implement.
1) How to write corresponding part pthread_cond_wait() in my main()?
2) I am trying to write it as:
pthread_mutex_lock(&(ThreadFinishStatus.mutex));
while((ThreadFinishStatus.signalled != TRUE){
pthread_cond_wait(&(ThreadFinishStatus.FINISHED), &(ThreadFinishStatus.mutex));
printf("Main Thread signalled\n");
ThreadFinishStatus.signalled==FALSE; //Reset signalled
//check which thread to join
}
pthread_mutex_unlock(&(ThreadFinishStatus.mutex));
But it does not enter the while loop.
3) How to use pthread_join() so that I can get the return value stored in my arg[i].returnStatus
i.e. where to put below statement in my main:
`pthread_join(T[i],&(arg[i].returnStatus));`
COMPLETE CODE
#include <stdio.h>
#include <pthread.h>
#include <time.h>
#define THREAD_NUM 5
#define FALSE 0
#define TRUE 1
void waitFor (unsigned int secs) {
time_t retTime;
retTime = time(0) + secs; // Get finishing time.
while (time(0) < retTime); // Loop until it arrives.
}
typedef struct {
int Finish_Status[THREAD_NUM];
int signalled;
pthread_mutex_t mutex;
pthread_cond_t FINISHED;
}THREAD_FINISH_STATE;
typedef struct {
int id;
void* returnStatus;
}THREAD_ARGUMENT;
THREAD_FINISH_STATE ThreadFinishStatus;
void initializeState(THREAD_FINISH_STATE* state)
{
int i=0;
state->signalled=FALSE;
for(i=0;i<THREAD_NUM;i++)
{
state->Finish_Status[i]=FALSE;
}
pthread_mutex_init(&(state->mutex),NULL);
pthread_cond_init(&(state->FINISHED),NULL);
}
void destroyState(THREAD_FINISH_STATE* state)
{
int i=0;
for(i=0;i<THREAD_NUM;i++)
{
state->Finish_Status[i]=FALSE;
}
pthread_mutex_destroy(&(state->mutex));
pthread_cond_destroy(&(state->FINISHED));
}
void* THREAD_ROUTINE(void* arg)
{
THREAD_ARGUMENT* temp=(THREAD_ARGUMENT*) arg;
printf("Thread created with id %d\n",temp->id);
waitFor(5);
pthread_mutex_lock(&(ThreadFinishStatus.mutex));
ThreadFinishStatus.Finish_Status[temp->id]=TRUE;
ThreadFinishStatus.signalled=TRUE;
if(ThreadFinishStatus.signalled==TRUE)
{
pthread_cond_signal(&(ThreadFinishStatus.FINISHED));
printf("Signal that thread %d finished\n",temp->id);
}
pthread_mutex_unlock(&(ThreadFinishStatus.mutex));
pthread_exit((void*)(temp->id));
}
int main()
{
THREAD_ARGUMENT arg[THREAD_NUM];
pthread_t T[THREAD_NUM];
int i=0;
initializeState(&ThreadFinishStatus);
for(i=0;i<THREAD_NUM;i++)
{
arg[i].id=i;
}
for(i=0;i<THREAD_NUM;i++)
{
pthread_create(&T[i],NULL,THREAD_ROUTINE,(void*)&arg[i]);
}
/*
Join only if signal received
*/
pthread_mutex_lock(&(ThreadFinishStatus.mutex));
//Wait
while((ThreadFinishStatus.signalled != TRUE){
pthread_cond_wait(&(ThreadFinishStatus.FINISHED), &(ThreadFinishStatus.mutex));
printf("Main Thread signalled\n");
ThreadFinishStatus.signalled==FALSE; //Reset signalled
//check which thread to join
}
pthread_mutex_unlock(&(ThreadFinishStatus.mutex));
destroyState(&ThreadFinishStatus);
return 0;
}
Here is an example of a program that uses a counting semaphore to watch as threads finish, find out which thread it was, and review some result data from that thread. This program is efficient with locks - waiters are not spuriously woken up (notice how the threads only post to the semaphore after they've released the mutex protecting shared state).
This design allows the main program to process the result from some thread's computation immediately after the thread completes, and does not require the main wait for all threads to complete. This would be especially helpful if the running time of each thread varied by a significant amount.
Most importantly, this program does not deadlock nor race.
#include <pthread.h>
#include <semaphore.h>
#include <stdlib.h>
#include <stdio.h>
#include <queue>
void* ThreadEntry(void* args );
typedef struct {
int threadId;
pthread_t thread;
int threadResult;
} ThreadState;
sem_t completionSema;
pthread_mutex_t resultMutex;
std::queue<int> threadCompletions;
ThreadState* threadInfos;
int main() {
int numThreads = 10;
int* threadResults;
void* threadResult;
int doneThreadId;
sem_init( &completionSema, 0, 0 );
pthread_mutex_init( &resultMutex, 0 );
threadInfos = new ThreadState[numThreads];
for ( int i = 0; i < numThreads; i++ ) {
threadInfos[i].threadId = i;
pthread_create( &threadInfos[i].thread, NULL, &ThreadEntry, &threadInfos[i].threadId );
}
for ( int i = 0; i < numThreads; i++ ) {
// Wait for any one thread to complete; ie, wait for someone
// to queue to the threadCompletions queue.
sem_wait( &completionSema );
// Find out what was queued; queue is accessed from multiple threads,
// so protect with a vanilla mutex.
pthread_mutex_lock(&resultMutex);
doneThreadId = threadCompletions.front();
threadCompletions.pop();
pthread_mutex_unlock(&resultMutex);
// Announce which thread ID we saw finish
printf(
"Main saw TID %d finish\n\tThe thread's result was %d\n",
doneThreadId,
threadInfos[doneThreadId].threadResult
);
// pthread_join to clean up the thread.
pthread_join( threadInfos[doneThreadId].thread, &threadResult );
}
delete threadInfos;
pthread_mutex_destroy( &resultMutex );
sem_destroy( &completionSema );
}
void* ThreadEntry(void* args ) {
int threadId = *((int*)args);
printf("hello from thread %d\n", threadId );
// This can safely be accessed since each thread has its own space
// and array derefs are thread safe.
threadInfos[threadId].threadResult = rand() % 1000;
pthread_mutex_lock( &resultMutex );
threadCompletions.push( threadId );
pthread_mutex_unlock( &resultMutex );
sem_post( &completionSema );
return 0;
}
Pthread conditions don't have "memory"; pthread_cond_wait doesn't return if pthread_cond_signal is called before pthread_cond_wait, which is why it's important to check the predicate before calling pthread_cond_wait, and not call it if it's true. But that means the action, in this case "check which thread to join" should only depend on the predicate, not on whether pthread_cond_wait is called.
Also, you might want to make the while loop actually wait for all the threads to terminate, which you aren't doing now.
(Also, I think the other answer about "signalled==FALSE" being harmless is wrong, it's not harmless, because there's a pthread_cond_wait, and when that returns, signalled would have changed to true.)
So if I wanted to write a program that waited for all threads to terminate this way, it would look more like
pthread_mutex_lock(&(ThreadFinishStatus.mutex));
// AllThreadsFinished would check that all of Finish_Status[] is true
// or something, or simpler, count the number of joins completed
while (!AllThreadsFinished()) {
// Wait, keeping in mind that the condition might already have been
// signalled, in which case it's too late to call pthread_cond_wait,
// but also keeping in mind that pthread_cond_wait can return spuriously,
// thus using a while loop
while (!ThreadFinishStatus.signalled) {
pthread_cond_wait(&(ThreadFinishStatus.FINISHED), &(ThreadFinishStatus.mutex));
}
printf("Main Thread signalled\n");
ThreadFinishStatus.signalled=FALSE; //Reset signalled
//check which thread to join
}
pthread_mutex_unlock(&(ThreadFinishStatus.mutex));
Your code is racy.
Suppose you start a thread and it finishes before you grab the mutex in main(). Your while loop will never run because signalled was already set to TRUE by the exiting thread.
I will echo #antiduh's suggestion to use a semaphore that counts the number of dead-but-not-joined threads. You then loop up to the number of threads spawned waiting on the semaphore. I'd point out that the POSIX sem_t is not like a pthread_mutex in that sem_wait can return EINTR.
Your code appears fine. You have one minor buglet:
ThreadFinishStatus.signalled==FALSE; //Reset signalled
This does nothing. It tests whether signalled is FALSE and throws away the result. That's harmless though since there's nothing you need to do. (You never want to set signalled to FALSE because that loses the fact that it was signalled. There is never any reason to unsignal it -- if a thread finished, then it's finished forever.)
Not entering the while loop means signalled is TRUE. That means the thread already set it, in which case there is no need to enter the loop because there's nothing to wait for. So that's fine.
Also:
ThreadFinishStatus.signalled=TRUE;
if(ThreadFinishStatus.signalled==TRUE)
There's no need to test the thing you just set. It's not like the set can fail.
FWIW, I would suggest re-architecting. If the existing functions like pthread_join don't do exactly what you want, just don't use them. If you're going to have structures that track what work is done, then totally separate that from thread termination. Since you will already know what work is done, what different does it make when and how threads terminate? Don't think of this as "I need a special way to know when a thread terminates" and instead think of this "I need to know what work is done so I can do other things".

How can I wait for any/all pthreads to complete?

I just want my main thread to wait for any and all my (p)threads to complete before exiting.
The threads come and go a lot for different reasons, and I really don't want to keep track of all of them - I just want to know when they're all gone.
wait() does this for child processes, returning ECHILD when there are no children left, however wait does not (appear to work with) (p)threads.
I really don't want to go through the trouble of keeping a list of every single outstanding thread (as they come and go), then having to call pthread_join on each.
As there a quick-and-dirty way to do this?
Do you want your main thread to do anything in particular after all the threads have completed?
If not, you can have your main thread simply call pthread_exit() instead of returning (or calling exit()).
If main() returns it implicitly calls (or behaves as if it called) exit(), which will terminate the process. However, if main() calls pthread_exit() instead of returning, that implicit call to exit() doesn't occur and the process won't immediately end - it'll end when all threads have terminated.
http://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_exit.html
Can't get too much quick-n-dirtier.
Here's a small example program that will let you see the difference. Pass -DUSE_PTHREAD_EXIT to the compiler to see the process wait for all threads to finish. Compile without that macro defined to see the process stop threads in their tracks.
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <time.h>
static
void sleep(int ms)
{
struct timespec waittime;
waittime.tv_sec = (ms / 1000);
ms = ms % 1000;
waittime.tv_nsec = ms * 1000 * 1000;
nanosleep( &waittime, NULL);
}
void* threadfunc( void* c)
{
int id = (int) c;
int i = 0;
for (i = 0 ; i < 12; ++i) {
printf( "thread %d, iteration %d\n", id, i);
sleep(10);
}
return 0;
}
int main()
{
int i = 4;
for (; i; --i) {
pthread_t* tcb = malloc( sizeof(*tcb));
pthread_create( tcb, NULL, threadfunc, (void*) i);
}
sleep(40);
#ifdef USE_PTHREAD_EXIT
pthread_exit(0);
#endif
return 0;
}
The proper way is to keep track of all of your pthread_id's, but you asked for a quick and dirty way so here it is. Basically:
just keep a total count of running threads,
increment it in the main loop before calling pthread_create,
decrement the thread count as each thread finishes.
Then sleep at the end of the main process until the count returns to 0.
.
volatile int running_threads = 0;
pthread_mutex_t running_mutex = PTHREAD_MUTEX_INITIALIZER;
void * threadStart()
{
// do the thread work
pthread_mutex_lock(&running_mutex);
running_threads--;
pthread_mutex_unlock(&running_mutex);
}
int main()
{
for (i = 0; i < num_threads;i++)
{
pthread_mutex_lock(&running_mutex);
running_threads++;
pthread_mutex_unlock(&running_mutex);
// launch thread
}
while (running_threads > 0)
{
sleep(1);
}
}
If you don't want to keep track of your threads then you can detach the threads so you don't have to care about them, but in order to tell when they are finished you will have to go a bit further.
One trick would be to keep a list (linked list, array, whatever) of the threads' statuses. When a thread starts it sets its status in the array to something like THREAD_STATUS_RUNNING and just before it ends it updates its status to something like THREAD_STATUS_STOPPED. Then when you want to check if all threads have stopped you can just iterate over this array and check all the statuses.
Don't forget though that if you do something like this, you will need to control access to the array so that only one thread can access (read and write) it at a time, so you'll need to use a mutex on it.
you could keep a list all your thread ids and then do pthread_join on each one,
of course you will need a mutex to control access to the thread id list. you will
also need some kind of list that can be modified while being iterated on, maybe a std::set<pthread_t>?
int main() {
pthread_mutex_lock(&mutex);
void *data;
for(threadId in threadIdList) {
pthread_mutex_unlock(&mutex);
pthread_join(threadId, &data);
pthread_mutex_lock(&mutex);
}
printf("All threads completed.\n");
}
// called by any thread to create another
void CreateThread()
{
pthread_t id;
pthread_mutex_lock(&mutex);
pthread_create(&id, NULL, ThreadInit, &id); // pass the id so the thread can use it with to remove itself
threadIdList.add(id);
pthread_mutex_unlock(&mutex);
}
// called by each thread before it dies
void RemoveThread(pthread_t& id)
{
pthread_mutex_lock(&mutex);
threadIdList.remove(id);
pthread_mutex_unlock(&mutex);
}
Thanks all for the great answers! There has been a lot of talk about using memory barriers etc - so I figured I'd post an answer that properly showed them used for this.
#define NUM_THREADS 5
unsigned int thread_count;
void *threadfunc(void *arg) {
printf("Thread %p running\n",arg);
sleep(3);
printf("Thread %p exiting\n",arg);
__sync_fetch_and_sub(&thread_count,1);
return 0L;
}
int main() {
int i;
pthread_t thread[NUM_THREADS];
thread_count=NUM_THREADS;
for (i=0;i<NUM_THREADS;i++) {
pthread_create(&thread[i],0L,threadfunc,&thread[i]);
}
do {
__sync_synchronize();
} while (thread_count);
printf("All threads done\n");
}
Note that the __sync macros are "non-standard" GCC internal macros. LLVM supports these too - but if your using another compiler, you may have to do something different.
Another big thing to note is: Why would you burn an entire core, or waste "half" of a CPU spinning in a tight poll-loop just waiting for others to finish - when you could easily put it to work? The following mod uses the initial thread to run one of the workers, then wait for the others to complete:
thread_count=NUM_THREADS;
for (i=1;i<NUM_THREADS;i++) {
pthread_create(&thread[i],0L,threadfunc,&thread[i]);
}
threadfunc(&thread[0]);
do {
__sync_synchronize();
} while (thread_count);
printf("All threads done\n");
}
Note that we start creating the threads starting at "1" instead of "0", then directly run "thread 0" inline, waiting for all threads to complete after it's done. We pass &thread[0] to it for consistency (even though it's meaningless here), though in reality you'd probably pass your own variables/context.

Resources