I'm trying to implement a multithreaded application but got some doubts. Can anybody please clarify one this?
My main goal is to
Create five threads to execute the work concurrently.
The program for this is:
for(i = 0; i < 5; i++)
{
pthread_create(tid[i], NULL, func, NULL)
}
Then generally we call for the pthread_join():
for(j = 0; j < 5; j++)
{
pthread_join(tid[j], NULL);
}
So here my question is that in this call of pthread_join(), if thread 2 finishes first then thread 2 will wait for thread 1 to finish as we have put in a sequential loop for the pthread_join() function.
for(j=0; j< 5 ; j++)
{
pthread_join(tid[j], NULL);
}
Means wait for thread with tid[0] to finish. If it is already finished, return immediately.
Then wait for thread with tid[1] to finish. If it is already finished, return immediately and so on.
In short, this for loop will make sure, current (possibly main) thread would not exit before children thread.
The for-loop is executed sequentially,
for(j=0; j< 5 ; j++)
{
pthread_join(tid[j], NULL);
}
so regardless of which thread finishes first, the for-loop will wait for thread 0 first, and then thread 1 etc. If thread 2 actually completes first then the result of that thread will simply just hang around until the join is executed -- this is similar to the way the wait call works if you were to wait for specific process ids.
Your loop is not doing something magical. So it calls
pthread_join(tid[0], NULL);
pthread_join(tid[1], NULL);
pthread_join(tid[2], NULL);
pthread_join(tid[3], NULL);
pthread_join(tid[4], NULL);
in the above order.
If you want to join threads in some other order, you need to use explicitly some synchronizations inside your func. You may want to use condition variables and mutexes. Read a good pthread tutorial
When you use pthread_join() function in a loop, your system will wait until first thread is returned, then the second one and so on. If the second thread is finished, it will wait for the first one and then it will take care of the second one.
Related
poll() in main() waits for some sort of a trigger from another application and when there is a trigger, pollHandler() is executed. In pollHandler(), I want to start 'n' number of threads based on the number of requests in the poll message.
But now in pollHandler(), when I want to use pthread_join in a different for loop, I don't have access to the thread_ids. I could create an array of pthread ids and use it outside the for loop block accessible to both pthread_create and pthread_join but the poll() function is active and it could get called again and again, thus overwriting the thread ids. How do I keep things clean here - wait for each thread to finish and make room to have more threads?
int pollHandler(){
int num_req = poll.size();
for(int i=0; i < num_req; i++){
// start thread for each req
pthread_t tid;
// thread paramters dynamically allocated and freed later
struct parameters *p = (struct parameters*)malloc(sizeof(struct parameters));
if((pthread_create(&tid, NULL, thread_func, p) != 0){
return -1;
}
}
for(int i=0; i < num_req; i++){
// pthread_join here but no access to thread ids?
}
return 0;
}
int main(){
......
while(1){
poll(); //waits for a trigger from another application
}
}
I want to start 'n' number of threads based on the number of requests in the poll message.
This design is fundamentally flawed: if you get a request with (say) 10,000 requests, it is unlikely that you will be able to create a separate thread to handle each one, and even if you could, thread creation and destruction are inefficient and best avoided.
A much better design is to start a thread pool, and dispatch work to them, waiting for all work to be completed before returning, as Martin James suggested.
That said, here is correct (except error checking is omitted for clarity) way to implement your current design:
int pollHandler(){
int num_req = poll.size();
pthread_t *tids = calloc(num_req * sizeof(pthread_t));
for(int i=0; i < num_req; i++){
// start thread for each req
// thread paramters dynamically allocated and freed later
struct parameters *p = (struct parameters*)malloc(sizeof(struct parameters));
if((pthread_create(&tid[i], NULL, thread_func, p) != 0){
// bug here.
return -1;
}
}
for(int i=0; i < num_req; i++){
pthread_join(tids[i], NULL);
}
free(tids);
return 0;
I could create an array of pthread ids and use it outside the for loop block accessible to both pthread_create and pthread_join but the poll() function is active and it could get called again and again, thus overwriting the thread ids.
Unless pollHandler() can interrupt another pollHandler(), it will not be invoked until previous invocation finishes, so the code above is "safe".
If pollHandler() can run as part of an interrupt, then your code is already hopelessly broken (neither malloc, nor pthread_create are async-signal safe and thus can't be called in signal handler).
P.S. What's up with //bug here?
You can't just return there -- you need to join the threads you've already created. You would also need to free(tids); there as well.
I want to spawn N number of threads that execute a function I did that like this
for (int p = 0; p < threads; p++)
pthread_create(&tid[p], NULL, dowork, full);
for now I used
for (int p = 0; p < threads; p++)
pthread_join(tid[p], NULL);
to spawn again n number of threads, but I want to spawn another thread when one of the created threads finish.
For exemple I have 3 threads to do 6 jobs (running the same function 6 times with different parameters)
I create 3 threads if one of them finish create another thread immediately without waiting for all of them to finish.
How can I do that?
Following is a simple solution for fixed threads loops. It should be good enough if each task has similar operation time.
Take each line of comment as a snippet of code.
int finished_tasks = 0;
int threads = 3;
int todo_tasks = 6;
int check_index = 0;
// ...
// Create thread by variable `threads` ...
for (int p = 0; p < threads; p++)
pthread_create(&tid[p], NULL, dowork, full);
while( finished_tasks < todo_tasks ){
// Join one thread by check index;
// finished_tasks++;
// Check if finished tasks meets todo tasks, leave the while
// Immediately create a new thread to do jobs
// Move check index to next one.
}
Since one thread is done, and another thread is created immediately. The total thread counts are fixed to threads.
Pro
Simple
Con
Running thread is deviating from 1~ threads. Because while loop is always waiting for current thread join while other threads might finish faster than current thread.
This solution is good enough if each task has similar operation time. However if operation time deviation is too large, you need to make a synchronization for checking which thread is done first.
I am creating 10 threads. Each thread will do some task. There are 7 tasks to be done. Since number of tasks are less than the number of threads, there would always be 3 threads that sleep and do nothing.
My main thread must wait for the tasks to get completed and exit only when all the tasks are done (i.e when the thread exits). I am waiting in a for loop and calling pthread_join but with the 3 threads that are sleeping, how can I wake them up and make them exit?
Here is what I am doing now.
// thread handler function, called when the thread is created
void* handler_func(void* arg) {
while(true){
pthread_mutex_lock(&my_queue_mutex);
while(my_queue_is_empty()) {
pthread_cond_wait(&my_cond_var, &my_queue_mutex);
}
item = get_item_from_queue();
pthread_mutex_unlock(&my_queue_mutex);
}
pthread_exit(NULL);
}
int total_threads_to_create = 10;
int total_requests_to_make = 7;
pthread_t threads[total_threads_to_create];
for(int i = 0; i < total_threads_to_create; i++) {
// create threads
pthread_create(&threads[i], NULL, handler_func, NULL);
}
for(int i=0;i<total_requests_to_make;i++){
// fill up the task queue
add_task_to_my_queue(i + 100);
}
for(int i = 0; i< total_threads_to_create; i++) {
// wait for threads to finish
pthread_join(threads[i], NULL);
}
The simplest thing to do would be to enqueue dummy tasks that indicate "all done" to the workers, since you know the number of workers in advance:
for(int i=0;i<total_threads_to_create;i++){
// a task of -1 means "no more work"
add_task_to_my_queue(-1);
}
Alternatively, you can have a "breakable" queue. This queue wakes up waiting consumers with a compound predicate: either non-empty or finished. Perl's Thread::Queue objects can be ended, for example, and Python's queues can track completed tasks.
Another option would be to keep track of the number of tasks completed yourself, with its own condition variable and mutex or a "countdown latch" or whatever, and not care about the worker threads. They'll be vaporized when the program exits.
This question already has an answer here:
Pthread_create() incorrect start routine parameter passing
(1 answer)
Closed 3 years ago.
I tried to build a program which should create threads and assign a Print function to each one of them, while the main process should use printf function directly.
Firstly, I made it without any synchronization means and expected to get a randomized output.
Later I tried to add a mutex to the Print function which was assigned to the threads and expected to get a chronological output but it seems like the mutex had no effect about the output.
Should I use a mutex on the printf function in the main process as well?
Thanks in advance
My code:
#include <stdio.h>
#include <pthread.h>
#include <errno.h>
pthread_t threadID[20];
pthread_mutex_t lock;
void* Print(void* _num);
int main(void)
{
int num = 20, indx = 0, k = 0;
if (pthread_mutex_init(&lock, NULL))
{
perror("err pthread_mutex_init\n");
return errno;
}
for (; indx < num; ++indx)
{
if (pthread_create(&threadID[indx], NULL, Print, &indx))
{
perror("err pthread_create\n");
return errno;
}
}
for (; k < num; ++k)
{
printf("%d from main\n", k);
}
indx = 0;
for (; indx < num; ++indx)
{
if (pthread_join(threadID[indx], NULL))
{
perror("err pthread_join\n");
return errno;
}
}
pthread_mutex_destroy(&lock);
return 0;
}
void* Print(void* _indx)
{
pthread_mutex_lock(&lock);
printf("%d from thread\n", *(int*)_indx);
pthread_mutex_unlock(&lock);
return NULL;
}
All questions of program bugs notwithstanding, pthreads mutexes provide only mutual exclusion, not any guarantee of scheduling order. This is typical of mutex implementations. Similarly, pthread_create() only creates and starts threads; it does not make any guarantee about scheduling order, such as would justify an assumption that the threads reach the pthread_mutex_lock() call in the same order that they were created.
Overall, if you want to order thread activities based on some characteristic of the threads, then you have to manage that yourself. You need to maintain a sense of which thread's turn it is, and provide a mechanism sufficient to make a thread notice when it's turn arrives. In some circumstances, with some care, you can do this by using semaphores instead of mutexes. The more general solution, however, is to use a condition variable together with your mutex, and some shared variable that serves as to indicate who's turn it currently is.
The code passes the address of the same local variable to all threads. Meanwhile, this variable gets updated by the main thread.
Instead pass it by value cast to void*.
Fix:
pthread_create(&threadID[indx], NULL, Print, (void*)indx)
// ...
printf("%d from thread\n", (int)_indx);
Now, since there is no data shared between the threads, you can remove that mutex.
All the threads created in the for loop have different value of indx. Because of the operating system scheduler, you can never be sure which thread will run. Therefore, the values printed are in random order depending on the randomness of the scheduler. The second for-loop running in the parent thread will run immediately after creating the child threads. Again, the scheduler decides the order of what thread should run next.
Every OS should have an interrupt (at least the major operating systems have). When running the for-loop in the parent thread, an interrupt might happen and leaves the scheduler to make a decision of which thread to run. Therefore, the numbers being printed in the parent for-loop are printed randomly, because all threads run "concurrently".
Joining a thread means waiting for a thread. If you want to make sure you print all numbers in the parent for loop in chronological order, without letting child thread interrupt it, then relocate the for-loop section to be after the thread joining.
I have an assignment in class that requires us to use POSIX threads and create n*(n-1)/2 of them to process a dataset of n elements.
You can think of it as basically the classical "handshake" in probability.
I know that for a large data set it's going to make the application CPU-bound and eventually it will spend so much time context switching that it will be useless, but the assignment requires us to do this.
However, my loop to create all the threads ceases creating them after a while.
For the code below, I will see output like:
making thread
thread start
thread done
made thread 1944
making thread
thread start
thread done
made thread 1945
making thread
thread start
thread done
made thread 1946
making thread
for a while, but then I will stop seeing the "thread start" and "thread done" messages, and only see the "making thread, made thread" messages.
Here is the loop that creates the threads:
int tCtr = 0;
tArr = (pthread_t*)malloc(((numbers_read) * (numbers_read - 1)/2) * sizeof(pthread_t));
for(i=0; i<numbers_read; i++){
int j;
for(j=i; j<numbers_read; j++){
// n(n-1)/2
if(i != j){
printf("making thread\n");
struct comparison_struct *data;
data = (struct comparison_struct *)malloc(sizeof(struct comparison_struct));
data->i_value = &numbers[i];
data->j_value = &numbers[j];
data->i_arr_entry = &wArr[i];
data->j_arr_entry = &wArr[j];
pthread_create(&tArr[tCtr], NULL, compare_thread, (void *)data);
printf("made thread %d\n", tCtr);
tCtr++;
}
}
}
for(i=0; i<tCtr; i++){
pthread_join(tArr[i], NULL);
}
free(tArr);
and here is the subroutine containing the thread code:
void *compare_thread(void *vData) {
printf("thread start\n");
struct comparison_struct *data;
data = (struct comparison_struct *)vData;
if(*data->i_value <= *data->j_value){
*data->i_arr_entry = 0;
} else {
*data->j_arr_entry = 0;
}
free(vData);
printf("thread done\n");
return NULL;
}
Anybody have any ideas? I'm new to pthreads and having trouble figuring it out.
I know that if I put the pthread_join call immediately after the pthread_create, the application works - but then it blocks on every thread, which I'd assume would decrease performance because there will only ever actually be 2 threads running at a time.
check the return value of pthread_create, maybe you are hitting a resource limit.
pthread_create() will fail if:
[EAGAIN] The system lacked the necessary resources to create
another thread, or the system-imposed limit on the
total number of threads in a process
[PTHREAD_THREADS_MAX] would be exceeded.
[EINVAL] The value specified by attr is invalid.
If you are reaching a resource limit, you can try to make a thread that joins on the other threads, make a worker queue and give work to each of the threads via the queue, or if you control the resource limit of the system try to increase it.
if I put the pthread_join call immediately after the pthread_create ...then it blocks on every thread...there will only ever actually be 2 threads running at a time.
An alternative to joining the threads is to just create them as detached. Create and initialize a pthread_attr_t, set it detached and pass the attr in with your pthread_create call.
pthread_attr_t attr;
int ret;
ret = pthread_attr_init(&attr);
if (ret)
// error .........
ret = pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
for (....)
{
//........
ret = pthread_create(&tArr[tCtr], &attr, compare_thread, (void *)data);
//.......
}
ret = pthread_attr_destroy(&attr);
There is also no requirement that a thread that creates another thread has to be the one to join it. You can create a thread just to join all other created threads. But that is probably going beyond the call of duty for this assignment.