I'm using c posix threads, and i've encountered a problem I don't quite understand and would like some fresh eyes to see what the problem could be.
In summary, I'm creating 3 threads which check some program state, signal that they are ready, and then wait for further signals. Meanwhile the thread that created these threads is waiting (pthread_cond_wait_) for each thread to signal that it has been setup. However it seems that the mutex is never unlocked once a wait has been signaled, causing the program the halt entirely.
This is the code for the main thread:
pthread_mutex_lock(&scenario->mutex);
if (scenario->flags.sinkThreadSetup == 0) {
pthread_create(&sinkT, NULL, sinkProblemHandler, (void *)scenario);
pthread_cond_wait(&scenario->sinkSetupCondition, &scenario->mutex);
}
if (scenario->flags.freeThreadSetup == 0) {
pthread_create(&freeT, NULL, freeWheelProblemHandler, (void *)scenario);
pthread_cond_wait(&scenario->freeSetupCondition, &scenario->mutex);
}
if (scenario->flags.blockThreadSetup == 0) {
pthread_create(&blockT, NULL, blockProblemHandler, (void *)scenario);
pthread_cond_wait(&scenario->blockSetupCondition, &scenario->mutex);
}
scenario->state = VECTORING;
pthread_mutex_unlock(&scenario->mutex);
// Start wheel threads
pthread_t wheelThreads[NUM_WHEELS];
scenario_wheel_t threadDataArr[NUM_WHEELS];
for (int i =0; i < NUM_WHEELS; i++) {
threadDataArr[i].scenario = scenario;
threadDataArr[i].wheel = &scenario->wheels[i];
pthread_create(&wheelThreads[i], NULL, wheel_start, (void *)&threadDataArr[i]);
}
pthread_mutex_lock(&scenario->mutex);
pthread_cond_wait(&scenario->scenarioComplete_condition, &scenario->mutex);
pthread_mutex_unlock(&scenario->mutex);
Here is the code for the problem handler threads:
void *sinkProblemHandler(void *args) {
scenario_t *scenario = (scenario_t *) args;
while(scenario->flags.terminate != 1) {
pthread_mutex_lock(&scenario->mutex);
if (scenario->state == SETUP && scenario->flags.sinkThreadSetup == 0) {
scenario->flags.sinkThreadSetup = 1;
pthread_cond_signal(&scenario->sinkSetupCondition);
}
if (scenario->state != SINKING) {
pthread_cond_wait(&scenario->conditions.sinking_condition, &scenario->mutex);
if (scenario->flags.terminate == 1) {
pthread_mutex_unlock(&scenario->mutex);
pthread_exit(NULL);
}
printf("SINKHandler: I'M HELPING!");
}
pthread_mutex_unlock(&scenario->mutex);
}
pthread_exit(NULL);
}
void *blockProblemHandler(void *args) {
scenario_t *scenario = (scenario_t *) args;
while(scenario->flags.terminate != 1) {
pthread_mutex_lock(&scenario->mutex);
if (scenario->state == SETUP && scenario->flags.blockThreadSetup == 0) {
scenario->flags.blockThreadSetup = 1;
pthread_cond_signal(&scenario->blockSetupCondition);
}
if (scenario->state != BLOCKED) {
pthread_cond_wait(&scenario->conditions.blocked_condition, &scenario->mutex);
if (scenario->flags.terminate == 1) {
pthread_mutex_unlock(&scenario->mutex);
pthread_exit(NULL);
}
printf("BlockHandler: I'M HELPING!");
}
pthread_mutex_unlock(&scenario->mutex);
}
pthread_exit(NULL);
}
void *freeWheelProblemHandler(void * args) {
scenario_t *scenario = (scenario_t *) args;
while(scenario->flags.terminate != 1) {
pthread_mutex_lock(&scenario->mutex);
if (scenario->state == SETUP && scenario->flags.freeThreadSetup == 0) {
scenario->flags.freeThreadSetup = 1;
pthread_cond_signal(&scenario->freeSetupCondition);
}
if (scenario->state != BLOCKED) {
pthread_cond_wait(&scenario->conditions.freeWheeling_condition, &scenario->mutex);
if (scenario->flags.terminate == 1) {
pthread_mutex_unlock(&scenario->mutex);
pthread_exit(NULL);
}
printf("FreeHandler: I'M HELPING!");
}
pthread_mutex_unlock(&scenario->mutex);
}
pthread_exit(NULL);
}
We can see here that the problem handler threads signal their respective setup condition, and then wait for a different condition (pthread_cond_wait),which should unlock the mutex.
My initial thought was that the problem handler threads were signaling before the main thread has started waiting, thus creating a deadlock. However after attaching to the program, this is not the case, as the program does:
1. Starts a problem thread
2. waits for the problem setup condition (unlocking mutex)
3. problem handler thread lock mutex.
3. problem handler thread then signals setup complete
4. problem handler thread then waits (supposedly unlocking mutex)
5. Program halts completely. (like the mutex has not been unlocked).
Any help would be greatly appreciated.
Thanks
Your worker threads roughly all have the form of
while(var) {
pthread_mutex_lock(&scenario->mutex);
// ...
pthread_mutex_unlock(&scenario->mutex);
}
that is mutex is locked immediately after it is unlocked. The main thread has a little chance to continue execution, as it cannot reacquire the mutex.
This is from man pthread_cond_signal:
The thread(s) that are unblocked shall contend for the mutex according to the scheduling policy (if applicable), and as if each had called pthread_mutex_lock().
So, at a bare minimum you can try pthread_yield() before reacquiring the mutex in the worker threads; and even better reduce the usage of mutex and / or introduce different mutexes to let different sections run in parallel.
Related
I have a threadpool of workers. Each worker executes this routine:
void* worker(void* args){
...
pthread_mutex_lock(&mtx);
while (queue == NULL && stop == 0){
pthread_cond_wait(&cond, &mtx);
}
el = pop(queue);
pthread_mutex_unlock(&mtx);
...
}
main thread:
int main(){
...
while (stop == 0){
...
pthread_mutex_lock(&mtx);
insert(queue, el);
pthread_cond_signal(&cond);
pthread_mutex_unlock(&mtx);
...
}
...
}
Then I have a signal handler that executes this code when it receives a signal:
void exit_handler(){
stop = 1;
pthread_mutex_lock(&mtx);
pthread_cond_broadcast(&cond);
pthread_mutex_unlock(&mtx);
}
I have omitted declarations and initialization, but the original code has them.
After a signal is received most of the time it's all ok, but sometimes it seems that some worker threads stay in the wait loop because they don't see that the variable stop is changed and/or they are not waken up by the broadcast.
So the threads never end.
What I am missing?
EDIT: stop=1 moved inside the critical section in exit_handler. The issue remains.
EDIT2: I was executing the program on a VM with Ubuntu. Since the code appears to be totally right I tried to change VM and OS (XUbuntu) and now it seems to work correctly. Still don't know why, anyone has an idea?
Some guessing here, but it's too long for a comment, so if this is wrong, I will delete. I think you may have a misconception about how pthread_cond_broadcast works (at least something I've been burned with in the past). From the man page:
The pthread_cond_broadcast() function shall unblock all threads
currently blocked on the specified condition variable cond.
Ok, that make sense, _broadcast awakens all threads currently blocked on cond. However, only one of the awakened threads will then be able to lock the mutex after they're all awoken. Also from the man page:
The thread(s) that are unblocked shall contend for the mutex according
to the scheduling policy (if applicable), and as if each had called
pthread_mutex_lock().
So this means that if 3 threads are blocked on cond and _broadcast is called, all 3 threads will wake up, but only 1 can grab the mutex. The other 2 will still be stuck in pthread_cond_wait, waiting on a signal. Because of this, they don't see stop set to 1, and exit_handler (I'm assuming a Ctrl+c software signal?) is done signaling, so the remaining threads that lost the _broadcast competition are stuck in limbo, waiting on a signal that will never come, and unable to read that the stop flag has been set.
I think there are 2 options to work-around/fix this:
Use pthread_cond_timedwait. Even without being signaled, this will return from waiting at the specified time interval, see that stop == 1, and then exit.
Add pthread_cond_signal or pthread_cond_broadcast at the end of your worker function. This way, right before a thread exits, it will signal the cond variable allowing any other waiting threads to grab the mutex and finish processing. There is no harm in signaling a conditional variable if no threads are waiting on it, so this should be fine even for the last thread.
EDIT: Here is an MCVE that proves (as far as I can tell) that my answer above is wrong, heh. As soon as I press Ctrl+c, the program exits "immediately", which says to me all the threads are quickly acquiring the mutex after the broadcast, seeing that stop is false, and exiting. Then main joins on the threads and it's process over.
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <stdbool.h>
#include <signal.h>
#include <unistd.h>
#define NUM_THREADS 3
#define STACK_SIZE 10
pthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t c = PTHREAD_COND_INITIALIZER;
volatile bool stop = false;
int stack[STACK_SIZE] = { 0 };
int sp = 0; // stack pointer,, also doubles as the current stack size
void SigHandler(int sig)
{
if (sig == SIGINT)
{
stop = true;
}
else
{
printf("Received unexcepted signal %d\n", sig);
}
}
void* worker(void* param)
{
long tid = (long)(param);
while (stop == false)
{
// acquire the lock
pthread_mutex_lock(&m);
while (sp <= 0) // sp should never be < 0
{
// there is no data in the stack to consume, wait to get signaled
// this unlocks the mutex when it is called, and locks the
// mutex before it returns
pthread_cond_wait(&c, &m);
}
// when we get here we should be guaranteed sp >= 1
printf("thread %ld consuming stack[%d] = %d\n", tid, sp-1, stack[sp-1]);
sp--;
pthread_mutex_unlock(&m);
int sleepVal = rand() % 10;
printf("thread %ld sleeping for %d seconds...\n", tid, sleepVal);
sleep(sleepVal);
}
pthread_exit(NULL);
}
int main(void)
{
pthread_t threads[NUM_THREADS];
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
srand(time(NULL));
for (long i=0; i<NUM_THREADS; i++)
{
int rc = pthread_create(&threads[i], &attr, worker, (void*)i);
if (rc != 0)
{
fprintf(stderr, "Failed to create thread %ld\n", i);
}
}
while (stop == false)
{
// produce data in bursts
int numValsToInsert = rand() % (STACK_SIZE - sp);
printf("main producing %d values\n", numValsToInsert);
// acquire the lock
pthread_mutex_lock(&m);
for (int i=0; i<numValsToInsert; i++)
{
// produce values for the stack
int val = rand() % 10000;
// I think this should already be guaranteed..?
if (sp+1 < STACK_SIZE)
{
printf("main pushing stack[%d] = %d\n", sp, val);
stack[sp++] = val;
// signal the workers that data is ready
//printf("main signaling threads...\n");
//pthread_cond_signal(&c);
}
else
{
printf("stack full!\n");
}
}
pthread_mutex_unlock(&m);
// signal the workers that data is ready
printf("main signaling threads...\n");
pthread_cond_broadcast(&c);
int sleepVal = 1;//rand() % 5;
printf("main sleeping for %d seconds...\n", sleepVal);
sleep(sleepVal);
}
for (long i=0; i<NUM_THREADS; i++)
{
pthread_join(threads[i], NULL);
}
return 0;
}
I have a function to be executed by thread1,2,3,4.... Once thread1 gets access to the function, I use thread_cond for waiting for the other threads. Once thread1 does its work, I send thread_signal to the same cond. Thread2 is executing the function. But after it finishes execution, the other threads don't get access to the function.
Please help me
#include<stdio.h>
#include<unistd.h>
#include<pthread.h>
int limit = 0;
pthread_mutex_t mutex[100];
pthread_cond_t cond[100];
pthread_t tid[100];
void *enter()
{
if (limit == 1)
{
printf("waiting\n");
pthread_cond_wait(&cond[1],&mutex[1]);
}
gotofunction();
}
void gotofunction()
{
limit++;
/* Do work */
printf("Doing work\n");
sleep(1);
limit--;
printf("Going to give signal\n");
pthread_cond_signal(&cond[1]);
}
int main()
{
int n,i;
scanf("%d",&n);
for (i=0;i<100;i++)
{
pthread_mutex_init(&mutex[i], NULL);
pthread_cond_init(&cond[i], NULL);
}
for (i=0;i<n;i++)
{
pthread_create(&tid[i], NULL, enter, NULL);
sleep(0.5);
}
for (i=1;i<=n;i++)
{
pthread_join(tid[i], NULL);
}
}
Basically thread1 prints "doing work"
Thread2 prints "doing work"
Then nothing happens for the other threads
You must have the mutex that you passed to pthread_cond_wait() locked at the time you call it. That mutex must also be locked while you check and change the condition that the condition variable is paired with (in this case, limit == 1 is that condition).
You should also use the pattern while (condition) { pthread_cond_wait() } (rather than if (condition)), because the condition variable may wake up without the condition actually being true.
Changing your code to lock the mutex around the accesses to limit, and unlock it while simulating the work, looks like this:
void *enter()
{
pthread_mutex_lock(&mutex[1]);
while (limit == 1)
{
printf("waiting\n");
pthread_cond_wait(&cond[1],&mutex[1]);
}
gotofunction();
pthread_mutex_unlock(&mutex[1]);
return NULL;
}
void gotofunction()
{
limit++;
pthread_mutex_unlock(&mutex[1]);
/* Do work */
printf("Doing work\n");
sleep(1);
pthread_mutex_lock(&mutex[1]);
limit--;
printf("Going to give signal\n");
pthread_cond_signal(&cond[1]);
}
Of course when you check against a limit of 1, you could just use a plain mutex instead - but this scheme could be extended to allow N threads to execute work simultaneously, by changing the condition to while (limit >= N).
I have a worker thread doing a polling activity (it's targeting libcurl but that is not relevant).
Many actions from the application can start a worker thread but if it is already running, no new thread should be created. A new request from the application is simply joined with any other polling already ongoing.
In summary, a single worker thread that exits when there is nothing left to do.
The code is:
pthread_t thread = NULL;
struct timespec sleep_val = {0, 20000000};
void *worker_thread(void *threadid)
{
do {
poll();
nanosleep(&sleep_val, NULL);
} while (no_of_handles_running > 0);
pthread_exit(NULL);
thread = NULL;
}
void start_worker_thread_if_needed() {
if (thread == NULL) {
pthread_create(&thread, NULL, worker_thread, NULL);
}
}
My doubt is about thread safety. The function start_worker_thread_if_needed() can be called at any time any number of times.
So, what if start_worker_thread_if_needed() is called exactly when we exit the while loop and therefor interrupts the worker thread. If that happens the condition if (thread == NULL) will be FALSE since the worker thread is interrupted and pthread_exit + thread = NULL has not yet completed.
So now start_worker_thread_if_needed() will exit without creating a new thread but as soon as control is returned to the old worker thread it will continue to the pthread_exit line and the worker thread is destroyed.
The issue will be that the request that was just made triggering start_worker_thread_if_needed() will be lost and the polling will not start until the next call to start_worker_thread_if_needed().
Edit: Here is a suggestion with mutex but I still have the same doubt. What happens if the main thread interrupts just after we exit the while loop and before the worker can take the mutex. Then the main thread does not create a new thread and then the worker exit.
void *worker_thread(void *threadid)
{
do {
poll();
nanosleep(&sleep_val, NULL);
} while (no_of_handles_running > 0);
pthread_mutex_lock(&my_mutex);
thread = NULL;
pthread_mutex_unlock(&my_mutex);
pthread_exit(NULL);
}
void start_worker_thread_if_needed() {
pthread_mutex_lock(&my_mutex);
if (thread == NULL) {
pthread_create(&thread, NULL, worker_thread, NULL);
}
pthread_mutex_unlock(&my_mutex);
}
I think I have a flaw in how I have set this up and it would be highly appreciated if someone could help out with the proper solution.
Thanks to for all the help in the comments above. Kaylums suggestion to use condition variables seems to be a good approach to not risk ending up with the corner case I was concerned about.
The final code (with logs removed) is the one below and it appears to work just fine.
static int no_of_handles_running;
static int RUN_THREAD = 0;
pthread_t thread = NULL;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cv = PTHREAD_COND_INITIALIZER;;
void *worker_thread(void *threadid)
{
RUN_THREAD = 1;
do {
poll(); //Will update no_of_handles_running
if (no_of_handles_running == 0) {
pthread_mutex_lock(&mutex);
pthread_cond_wait(&cv, &mutex);
pthread_mutex_unlock(&mutex);
}
} while (RUN_THREAD);
thread = NULL;
pthread_exit(NULL);
}
void start_worker_thread() {
if (thread == NULL) {
int rc = pthread_create(&thread, NULL, worker_thread, NULL);
if (rc){
return;
}
}
pthread_mutex_lock(&mutex);
pthread_cond_signal(&cv);
pthread_mutex_unlock(&mutex);
}
void stop_worker_thread() {
RUN_THREAD = 0;
pthread_mutex_lock(&mutex);
pthread_cond_signal(&cv);
pthread_mutex_unlock(&mutex);
}
I'm trying to create four threads printing some message.
I'm facing some issue with synchronisation.
Here is what my main() looks like
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;
int count = 4;
int main (void)
{
pthread_t thread1, thread2, thread3, thread4;
pthread_create (&thread4, NULL, function4, NULL);
pthread_create (&thread3, NULL, function3, NULL);
pthread_create (&thread2, NULL, function2, NULL);
pthread_create (&thread1, NULL, function1, NULL);
pthread_join (thread1, NULL);
pthread_join (thread2, NULL);
pthread_join (thread3, NULL);
pthread_join (thread4, NULL);
return 0;
}
function1() prints Function 1, function2() prints Function 2 and so on.
Desired Output should be the following:
Function 1
Function 2
Function 3
Function 4
Actual Output:
Function 1
/* Infinitely runs (Deadlock maybe) */
Actual Question:
Can we use one single condition variable for Synchronization of two or more threads ? If so how ?
If not, how do I tackle this problem ?
Here is the definition of function(n)
void *function1 ()
{
while (1)
{
if (count == 4)
{
pthread_mutex_lock (&mutex);
printf("Function 1\n");
count --;
pthread_mutex_unlock (&mutex);
pthread_cond_signal (&cond);
return NULL;
}
else
pthread_cond_wait(&cond, &mutex);
}
return NULL;
}
void *function2 ()
{
while (1)
{
if (count == 3)
{
pthread_mutex_lock (&mutex);
printf("Function 2\n");
count--;
pthread_mutex_unlock (&mutex);
pthread_cond_signal (&cond);
return NULL;
}
else
pthread_cond_wait(&cond, &mutex);
}
return NULL;
}
void *function3 ()
{
while (1)
{
if(count == 2)
{
pthread_mutex_lock (&mutex);
printf("Function 3\n");
count--;
pthread_mutex_unlock (&mutex);
pthread_cond_signal (&cond);
return NULL;
}
else
pthread_cond_wait(&cond, &mutex);
}
return NULL;
}
void *function4 ()
{
while (1)
{
if(count == 1)
{
pthread_mutex_lock (&mutex);
printf("Function 4\n");
pthread_mutex_unlock (&mutex);
pthread_cond_signal (&cond);
return NULL;
}
else
pthread_cond_wait(&cond, &mutex);
}
return NULL;
}
After a more clear understanding, I have resolved this issue.
This is a case of race condition.
Things wrong with the code:
Use pthread_cond_broadcast instead ofpthread_cond_signal.
From the man pages
The pthread_cond_broadcast() function shall unblock all threads currently blocked on the specified condition variable cond.
The pthread_cond_signal() function shall unblock at least one of the threads that are blocked on the specified condition variable cond
Not acquiring a lock before checking for if (count == X), Need a
place a lock before that (since count is global/shared variable).
Not placing an unlock if the control goes to else. Since a lock
is already placed (earlier before else), you need to unlock
it.
If you fail to do so, other threads will try to lock a mutex,
that is already locked , leading to race conditions.
Function should be as follow
void *function1 ()
{
while (1)
{
/* Take a lock on the Mutex */
pthread_mutex_lock (&mutex);
if (4 == count)
{
printf("Function 1\n"); /* Perform your task */
count--;
/* Once operation on shared variable (count) is completed, release lock*/
pthread_mutex_unlock (&mutex);
/* Broadcast to other threads about completion of task */
pthread_cond_broadcast (&cond);
return NULL;
}
else
{
/* If count doesnt match, wait on the condition (wait for signal from other threads) */
pthread_cond_wait(&cond, &mutex);
/* Unlock the mutex, since it was locked earlier - else goes to deadlock */
pthread_mutex_unlock (&mutex);
}
}
return NULL;
}
Here is the complete working code.
The change required is simple:
instead of:
else
pthread_cond_wait(&cond, &mutex);
do this:
else {
pthread_mutex_lock (&mutex);
pthread_cond_wait(&cond, &mutex);
pthread_mutex_unlock (&mutex);
}
It works for me with these changes, but results in unpredictable behaviour without the changes.
Edit:
This above mentioned simple way still leaves a race condition and doesn't fix the signal vs broadcast problem. To avoid it, the code should be structured like the following:
pthread_mutex_lock (&mutex);
if (count == 4)
{
printf("Function 1\n");
count --;
pthread_mutex_unlock (&mutex);
pthread_cond_broadcast (&cond); // NOTE: broadcast, not signal!
return NULL;
}
else
pthread_cond_wait(&cond, &mutex);
pthread_mutex_unlock (&mutex);
Note that pthread_cond_broadcast is necessary as you want to wake up all threads.
I'm supposed to implement a signal handler which stops all the threads in the program the first time a certain signal is called, and resume them if they are already stopped. I implemented a demon thread running in the background using the following code:
void *handle_sig(void *arg)
{
sigset_t set;
//set set...
struct timespec to;
to.tv_sec = 1;
to.tv_nsec = 0;
while (_keepListening) {
int ret = sigtimedwait(&set, NULL, &to);
if (ret < 0)
if (errno == EAGAIN) {
continue; /* no signal sent*/
} else {
fprintf(stderr, "Failed...\n");
exit(1);
}
if (_state == PAUSE_STATE) { /* was blocked*/
//do something
} else {
` //do something else
}
}
return NULL;
}
The problem is that if a signal is sent and detected, and then another is sent before the while loop is entered again - the second signal is ignored, which can leave the program paused forever. is there a way of detecting if a signal was sent since the last call to sigtimedwait?
Thank you
You might think about making your processing asynchronous.
For example by starting a thread for "doing something" or "doing something else".