c pthreads | creating and joining specific threads without waiting? - c

Question

I'm in a situation where every given time interval (let's say 1 second) I need to generate a thread that follows a pre-determined set of actions. Then after it is done, somehow I need to clean up the resources associated with that thread. But I'm not sure how I can do this while still generating new threads, since pthread_join is blocking, so I can't keep generating new threads while waiting for others to finish.
The typical method I have seen suggested to do something like this is:
int i;
pthread_t threads[NUM_THREADS];
for (i = 0; i < NUM_THREADS; ++i) {
pthread_create(&threads[i], NULL, mythread, NULL);
}
for (i = 0; i < NUM_THREADS; ++i) {
pthread_join(threads[i], NULL);
}
However, I don't want to generate a pre-determined amount of threads at the start and let them run. I want to generate the threads one at a time, and just keep generating (this will be ok since the threads reach a saturation point where they just end at the first step if there's more than 100 of them). One solution I thought of is to have the pthread_joins running in their own thread, but then I'm not sure how to tell it which ones to join. These threads have randomised sleep times within them so there's no specified order in which they ought to finish. What I have in mind as to how the program should run is something like this:
Thread[1] created/running
Thread[2] created/running
Thread[3] created/running
Thread[2] finished -> join/free memory
new Thread[2] created/running (since 2 finished, now create a new thread 2)
So for example, you can never have more than 5 threads running, but every time one does finish you create a new one. The threads don't necessarily need to be in an array, I just thought that would make it easier to manage. I've been thinking about this for hours now and can't think of a solution. Am I just approaching the problem the completely wrong way, and there's something easier?

Answer 1

Main thread:
Initialize counting semaphore to 5
Semaphore down
Launch thread
Loop back
Worker thread:
Work
Cleanup
Semaphore up
Die

Answer 2

pthread_join is a function for joining threads, not to wait for them. Yes, the caller will wait until the thread returns from the starting function, but that is only a side effect.
Waiting for threads are accomplished via condition variables (pthread_cond_wait) or barriers (pthread_barrier_wait).
Another approach would be to introduce a global list, where the finished threads will be stored (before exiting the thread, add the thread to that list). And every interval (i assume from your main function), work off the list and join all threads. I don't think that i have to mention, that the list has to be protected by a mutex.
e.g.
struct thread_data {
struct thread_data *next; //single linked list instead of array
pthread_t id; //thread id
void *priv_data; //some data needed for other functionality
bool in_use; //test flag to indicate
//if the specific data/thread is in use/running
};
//mutex to protect the global garbage data
pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
//global list for garbage collection, protected via 'lock'
struct thread_data *garbage = NULL;
//thread starting function
void* thread_func(void *arg)
{
//1. cast 'arg' to 'struct thread_data'
struct thread_data *data = (struct thread_data*) arg;
//2. do your other stuff
//3. before returning/exiting
pthread_mutex_lock(&lock);
//add data to garbage
data->next = garbage;
garbage = data;
pthread_mutex_unlock(&lock);
return NULL;
}
int main()
{
#define MAX_THREADS 5 //some arbitrary number
struct thread_data data[MAX_THREADS]; //main data storage
while (true) {
//1. your sleep/wait routine
//2. on wake up, collect garbage first
pthread_mutex_lock(&lock);
for (; garbage ; garbage = garbage->next) {
//destroy garbage->priv_data, if not already done somewhere else
pthread_join(garbage->id); //clear thread resources
garbage->in_use = false; //data no longer in use, free to reuse
}
pthread_mutex_unlock(&lock);
//3. iterate over data storage and if a reusable data is found,
//create the thread and pass the specific data
//as argument to the thread starting function
for (int i=0; i < MAX_THREADS; ++i) {
if (!data[i].in_use) {
data[i].in_use = true;
//data[i].priv_data = ...
pthread_create(&data[i].id, NULL, thread_func, &data[i]);
break; //we only want one thread at a time, right?
}
}
}
return 0;
}

Answer 3

You could call pthread_tryjoin_np for existing threads when you're about to create a new one. (This is a GNU extension to POSIX.)
You could create a list of completed threads, and join them and empty the list when you're about to create a new one. An easily-implemented stack could be used as the list.
You could detach the threads so they get cleaned up automatically. You will need a mechanism to wait for threads to complete before existing the main program, but that can easily be accomplished with a cond var.