I am creating 10 threads. Each thread will do some task. There are 7 tasks to be done. Since number of tasks are less than the number of threads, there would always be 3 threads that sleep and do nothing.
My main thread must wait for the tasks to get completed and exit only when all the tasks are done (i.e when the thread exits). I am waiting in a for loop and calling pthread_join but with the 3 threads that are sleeping, how can I wake them up and make them exit?
Here is what I am doing now.
// thread handler function, called when the thread is created
void* handler_func(void* arg) {
while(true){
pthread_mutex_lock(&my_queue_mutex);
while(my_queue_is_empty()) {
pthread_cond_wait(&my_cond_var, &my_queue_mutex);
}
item = get_item_from_queue();
pthread_mutex_unlock(&my_queue_mutex);
}
pthread_exit(NULL);
}
int total_threads_to_create = 10;
int total_requests_to_make = 7;
pthread_t threads[total_threads_to_create];
for(int i = 0; i < total_threads_to_create; i++) {
// create threads
pthread_create(&threads[i], NULL, handler_func, NULL);
}
for(int i=0;i<total_requests_to_make;i++){
// fill up the task queue
add_task_to_my_queue(i + 100);
}
for(int i = 0; i< total_threads_to_create; i++) {
// wait for threads to finish
pthread_join(threads[i], NULL);
}
The simplest thing to do would be to enqueue dummy tasks that indicate "all done" to the workers, since you know the number of workers in advance:
for(int i=0;i<total_threads_to_create;i++){
// a task of -1 means "no more work"
add_task_to_my_queue(-1);
}
Alternatively, you can have a "breakable" queue. This queue wakes up waiting consumers with a compound predicate: either non-empty or finished. Perl's Thread::Queue objects can be ended, for example, and Python's queues can track completed tasks.
Another option would be to keep track of the number of tasks completed yourself, with its own condition variable and mutex or a "countdown latch" or whatever, and not care about the worker threads. They'll be vaporized when the program exits.
Related
In my program the main thread starts multiple child threads, locks a mutex for a counter and then sleeps until the counter reaches N.
In an infinite for loop, the other threads do some work and they increase the counter in each iteration alternately. Once the counter == N one of them has to signal the main thread and all of the child threads have to end.
My problem is that I'm not sure when I should lock the counter mutex and make a signal in the thread function, because once the counter becomes N I managed to wake up the main thread and exit one of the child threads, but the other threads will keep on trying to work when they should all be terminating.
My question is, how can I check if counter == N but send a signal by only one of the threads, and the others just return without any signalling?
Can this be done by only locking the mutex once in each iteration, for the time of checking its value (== N)? If yes, how?
void *thread_function(void *arg) {
int *id = (int *) arg;
for (;;) {
pthread_mutex_lock(&mutex_counter);
counter++;
if (counter == N) {
pthread_cond_signal(&cond);
pthread_mutex_unlock(&mutex_counter);
return NULL;
}
if (counter > N) {
pthread_mutex_unlock(&mutex_counter);
return NULL;
}
pthread_mutex_unlock(&mutex_counter);
sleep(random());
// do some work here
// ...
// finish work
}
The problem with above code is that, despite all threads terminating, they execute the loop one less iteration than needed because the counter is increased before the if clause (so counter will be N, but there were no N loops of work done).
However, if I put the counter to the end of the function, the threads will do more work than needed which messes up my program.
why you lock counter-mutex in main thread? Why you need to send signal from other treads to main? If you have only one global counter and all threads increasing it, you can just start in main thread all threads, remember their thread ids and then wait using while(counter<N); instruction. After all, just kill all threads with pthread_cancel.
I want to spawn N number of threads that execute a function I did that like this
for (int p = 0; p < threads; p++)
pthread_create(&tid[p], NULL, dowork, full);
for now I used
for (int p = 0; p < threads; p++)
pthread_join(tid[p], NULL);
to spawn again n number of threads, but I want to spawn another thread when one of the created threads finish.
For exemple I have 3 threads to do 6 jobs (running the same function 6 times with different parameters)
I create 3 threads if one of them finish create another thread immediately without waiting for all of them to finish.
How can I do that?
Following is a simple solution for fixed threads loops. It should be good enough if each task has similar operation time.
Take each line of comment as a snippet of code.
int finished_tasks = 0;
int threads = 3;
int todo_tasks = 6;
int check_index = 0;
// ...
// Create thread by variable `threads` ...
for (int p = 0; p < threads; p++)
pthread_create(&tid[p], NULL, dowork, full);
while( finished_tasks < todo_tasks ){
// Join one thread by check index;
// finished_tasks++;
// Check if finished tasks meets todo tasks, leave the while
// Immediately create a new thread to do jobs
// Move check index to next one.
}
Since one thread is done, and another thread is created immediately. The total thread counts are fixed to threads.
Pro
Simple
Con
Running thread is deviating from 1~ threads. Because while loop is always waiting for current thread join while other threads might finish faster than current thread.
This solution is good enough if each task has similar operation time. However if operation time deviation is too large, you need to make a synchronization for checking which thread is done first.
I've got a project where I'm adding jobs to a queue and I have multiple threads taking jobs, and calculating their own independent results.
My program handles the SIGINT signal and I'm attempting to join the threads to add up the results, print to screen, and then exit. My problem is is that the threads either seem to stop functioning when I send the signal, or they're getting blocked on the mutex_lock. Here are the important parts of my program so as to be concise.
main.c
//the thread pool has a queue of jobs inside
//called jobs (which is a struct)
struct thread_pool * pool;
void signal_handler(int signo) {
pool->jobs->running = 0; //stop the thread pool
pthread_cond_broadcast(pool->jobs->cond);
for (i = 0; i < tpool->thread_count; i++) {
pthread_join(tpool->threads[i], retval);
//do stuff with retval
}
//print results then exit
exit(EXIT_SUCCESS);
}
int main() {
signal(SIGINT, signal_handler);
//set up threadpool and jobpool
//start threads (they all run the workerThread function)
while (1) {
//send jobs to the job pool
}
return 0;
}
thread_stuff.c
void add_job(struct jobs * j) {
if (j->running) {
pthread_mutex_lock(j->mutex);
//add job to queue and update count and empty
pthread_cond_signal(j->cond);
pthread_mutex_unlock(j->mutex);
}
}
struct job * get_job(struct jobs * j) {
pthread_mutex_lock(j->mutex);
while (j->running && j->empty)
pthread_cond_wait(j->cond, j->mutex);
if (!j->running || j->empty) return NULL;
//get the next job from the queue
//unlock mutex and send a signal to other threads
//waiting on the condition
pthread_cond_signal(j->cond);
pthread_mutex_unlock(j->mutex);
//return new job
}
void * workerThread(void * arg) {
struct jobs * j = (struct jobs *) arg;
int results = 0;
while (j->running) {
//get next job and process results
}
return results;
}
Thanks for your help, this is giving me a real headache!
You should not call pthread_cond_wait or pthread_join from a signal handler which handles asynchronously generated signals such as SIGINT. Instead, you should block SIGINT for all threads, spawn a dedicated thread, and call sigwait there. This means that you detect the arrival of the SIGINT signal outside of a signal handler context, so that you are not restricted to async-signal-safe functions. You also avoid the risk of self-deadlock in case the signal is delivered to one of the worker threads.
At this point, you just need to shut down your work queue/thread pool in an orderly manner. Depending on the details, your existing approach with the running flag might even work unchanged.
I'm trying to implement a multithreaded application but got some doubts. Can anybody please clarify one this?
My main goal is to
Create five threads to execute the work concurrently.
The program for this is:
for(i = 0; i < 5; i++)
{
pthread_create(tid[i], NULL, func, NULL)
}
Then generally we call for the pthread_join():
for(j = 0; j < 5; j++)
{
pthread_join(tid[j], NULL);
}
So here my question is that in this call of pthread_join(), if thread 2 finishes first then thread 2 will wait for thread 1 to finish as we have put in a sequential loop for the pthread_join() function.
for(j=0; j< 5 ; j++)
{
pthread_join(tid[j], NULL);
}
Means wait for thread with tid[0] to finish. If it is already finished, return immediately.
Then wait for thread with tid[1] to finish. If it is already finished, return immediately and so on.
In short, this for loop will make sure, current (possibly main) thread would not exit before children thread.
The for-loop is executed sequentially,
for(j=0; j< 5 ; j++)
{
pthread_join(tid[j], NULL);
}
so regardless of which thread finishes first, the for-loop will wait for thread 0 first, and then thread 1 etc. If thread 2 actually completes first then the result of that thread will simply just hang around until the join is executed -- this is similar to the way the wait call works if you were to wait for specific process ids.
Your loop is not doing something magical. So it calls
pthread_join(tid[0], NULL);
pthread_join(tid[1], NULL);
pthread_join(tid[2], NULL);
pthread_join(tid[3], NULL);
pthread_join(tid[4], NULL);
in the above order.
If you want to join threads in some other order, you need to use explicitly some synchronizations inside your func. You may want to use condition variables and mutexes. Read a good pthread tutorial
When you use pthread_join() function in a loop, your system will wait until first thread is returned, then the second one and so on. If the second thread is finished, it will wait for the first one and then it will take care of the second one.
I have an assignment in class that requires us to use POSIX threads and create n*(n-1)/2 of them to process a dataset of n elements.
You can think of it as basically the classical "handshake" in probability.
I know that for a large data set it's going to make the application CPU-bound and eventually it will spend so much time context switching that it will be useless, but the assignment requires us to do this.
However, my loop to create all the threads ceases creating them after a while.
For the code below, I will see output like:
making thread
thread start
thread done
made thread 1944
making thread
thread start
thread done
made thread 1945
making thread
thread start
thread done
made thread 1946
making thread
for a while, but then I will stop seeing the "thread start" and "thread done" messages, and only see the "making thread, made thread" messages.
Here is the loop that creates the threads:
int tCtr = 0;
tArr = (pthread_t*)malloc(((numbers_read) * (numbers_read - 1)/2) * sizeof(pthread_t));
for(i=0; i<numbers_read; i++){
int j;
for(j=i; j<numbers_read; j++){
// n(n-1)/2
if(i != j){
printf("making thread\n");
struct comparison_struct *data;
data = (struct comparison_struct *)malloc(sizeof(struct comparison_struct));
data->i_value = &numbers[i];
data->j_value = &numbers[j];
data->i_arr_entry = &wArr[i];
data->j_arr_entry = &wArr[j];
pthread_create(&tArr[tCtr], NULL, compare_thread, (void *)data);
printf("made thread %d\n", tCtr);
tCtr++;
}
}
}
for(i=0; i<tCtr; i++){
pthread_join(tArr[i], NULL);
}
free(tArr);
and here is the subroutine containing the thread code:
void *compare_thread(void *vData) {
printf("thread start\n");
struct comparison_struct *data;
data = (struct comparison_struct *)vData;
if(*data->i_value <= *data->j_value){
*data->i_arr_entry = 0;
} else {
*data->j_arr_entry = 0;
}
free(vData);
printf("thread done\n");
return NULL;
}
Anybody have any ideas? I'm new to pthreads and having trouble figuring it out.
I know that if I put the pthread_join call immediately after the pthread_create, the application works - but then it blocks on every thread, which I'd assume would decrease performance because there will only ever actually be 2 threads running at a time.
check the return value of pthread_create, maybe you are hitting a resource limit.
pthread_create() will fail if:
[EAGAIN] The system lacked the necessary resources to create
another thread, or the system-imposed limit on the
total number of threads in a process
[PTHREAD_THREADS_MAX] would be exceeded.
[EINVAL] The value specified by attr is invalid.
If you are reaching a resource limit, you can try to make a thread that joins on the other threads, make a worker queue and give work to each of the threads via the queue, or if you control the resource limit of the system try to increase it.
if I put the pthread_join call immediately after the pthread_create ...then it blocks on every thread...there will only ever actually be 2 threads running at a time.
An alternative to joining the threads is to just create them as detached. Create and initialize a pthread_attr_t, set it detached and pass the attr in with your pthread_create call.
pthread_attr_t attr;
int ret;
ret = pthread_attr_init(&attr);
if (ret)
// error .........
ret = pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
for (....)
{
//........
ret = pthread_create(&tArr[tCtr], &attr, compare_thread, (void *)data);
//.......
}
ret = pthread_attr_destroy(&attr);
There is also no requirement that a thread that creates another thread has to be the one to join it. You can create a thread just to join all other created threads. But that is probably going beyond the call of duty for this assignment.