I'm approaching to C Windows programming in particular threads, concurrency and synchronization.
To experiment, I'm writing a C program that accepts N parameters.
Each parameter indicates a path to a file system directory tree and the program has to compare the content of all directories to decide whether all directories have the same content or not.
The main runs a "reading" thread for each parameter while a single "comparison" thread compares the name of all the entries found. For each file/directory found, "reading" threads synchronize themselves by activating the "comparison" thread.
I wrote the program with Semaphore objects and now I'm trying with Event objects.
The idea is to use N Events auto-reset and a single Event manual-reset.
The N events are used by the N "reading" threads to signal the "comparison" thread which is in WaitForMultipleObjects for an INFINITE time. When all the signals are available, it starts comparing the entry and then it performs a SetEvent() for the manual-reset object.
The "reading" threads wait for this set and then Reset the event and continue working with the next entry.
Some code for the N reading threads:
void ReadingTraverseDirectory(LPTSTR StartPathName, DWORD i) {
//variables and some work
do {
//take the next entry and put it in current_entry;
gtParams[it].entry = current_entry; //global var for comparison
SetEvent(glphReadingEvent[i]); //signal the comparison thread
WaitForSingleObject(ghComparisonEvent, INFINITE); //wait signal to restart working
ResetEvent(ghComparisonEvent); //reset the event
if (current_entry == TYPE_DIR) {
ReadingTraverseDirectory(current_entry, i); //recur to explor the next dir
}
} while (FindNextFile(SearchHandle, &FindData)); //while there are still entries
//
return;
}
Some code for the comparison thread:
DWORD WINAPI CompareThread(LPVOID arg) {
while (entries are equal){
WaitForMultipleObjects(N, glphReadingEvent, TRUE, 1000);
for (r = 0; r < nworkers - 1; r++){
if (_tcscmp(entries) != 0){
//entries are different. exit and close.
}
}
SetEvent(ghComparisonEvent);
}
}
The problem:
Sometimes it happens that one reading thread is able to work without respecting the synchro with other threads. If I put a printf() or Sleep(1) -between Wait and Set of the comparison thread-, the program works perfectly.
My opinion:
I think the manual-reset Event is not safe for this kind of (barrier)synchronization.
A reading thread may be too fast in ResetEvent() and if the scheduler slows down other threads, it is possible that some of them risk to stay blocked while the one which performed the Reset is able to continue its work.However if this is the case, the comparison thread should block itself on WaitingForMultipleObjects causing a deadlock... actually there is no deadlock but 1 thread is able to cycle more times respect to others.
What I'm trying to understand is why a simple Sleep(1) can solve the issue. Is it matter of scheduling or wrong implementation of synchronization?
Thank you.
Related
My problem is that I cannot reuse cancelled pthread. Sample code:
#include <pthread.h>
pthread_t alg;
pthread_t stop_alg;
int thread_available;
void *stopAlgorithm() {
while (1) {
sleep(6);
if (thread_available == 1) {
pthread_cancel(alg);
printf("Now it's dead!\n");
thread_available = 0;
}
}
}
void *algorithm() {
while (1) {
printf("I'm here\n");
}
}
int main() {
thread_available = 0;
pthread_create(&stop_alg, NULL, stopAlgorithm, 0);
while (1) {
sleep(1);
if (thread_available == 0) {
sleep(2);
printf("Starting algorithm\n");
pthread_create(&alg, NULL, algorithm, 0);
thread_available = 1;
}
}
}
This sample should create two threads - one will be created at the program beginning and will try to cancel second as soon it starts, second should be rerunned as soon at it was cancelled and say "I'm here". But when algorithm thread cancelled once it doesn't start once again, it says "Starting algorithm" and does nothing, no "I'm here" messages any more. Could you please tell me the way to start cancelled(immediately stopped) thread once again?
UPD: So, thanks to your help I understood what is the problem. When I rerun algorithm thread it throws error 11:"The system lacked the necessary resources to create another thread, or the system-imposed limit on the total number of threads in a process PTHREAD_THREADS_MAX would be exceeded.". Actually I have 5 threads, but only one is cancelled, others stop by pthread_exit. So after algorithm stopped and program went to standby mode I checked status of all threads with pthread_join - all thread show 0(cancelled shows PTHREAD_CANCELED), as far as I can understand this means, that all threads stopped successfully. But one more try to run algorithm throws error 11 again. So I've checked memory usage. In standby mode before algorithm - 10428, during the algorithm, when all threads used - 2026m, in standby mode after algorithm stopped - 2019m. So even if threads stopped they still use memory, pthread_detach didn't help with this. Are there any other ways to clean-up after threads?
Also, sometimes on pthread_cancel my program crashes with "libgcc_s.so.1 must be installed for pthread_cancel to work"
Several points:
First, this is not safe:
int thread_available;
void *stopAlgorithm() {
while (1) {
sleep(6);
if (thread_available == 1) {
pthread_cancel(alg);
printf("Now it's dead!\n");
thread_available = 0;
}
}
}
It's not safe for at least reasons. Firstly, you've not marked thread_available as volatile. This means that the compiler can optimise stopAlgorithm to read the variable once, and never reread it. Secondly, you haven't ensured access to it is atomic, or protected it by a mutex. Either declare it:
volatile sig_atomic_t thread_available;
(or similar), or better, protect it by a mutex.
But for the general case of triggering one thread from another, you are better using a condition variable (and a mutex), using pthread_condwait or pthread_condtimedwait in the listening thread, and pthread_condbroadcast in the triggering thread.
Next, what's the point of the stopAlgorithm thread? All it does is cancel the algorithm thread after an unpredictable amount of time between 0 and 6 seconds? Why not just sent the pthread_cancel from the main thread?
Next, do you care where your algorithm is when it is cancelled? If not, just pthread_cancel it. If so (and anyway, I think it's far nicer), regularly check a flag (either atomic and volatile as above, or protected by a mutex) and pthread_exit if it's set. If your algorithm does big chunks every second or so, then check it then. If it does lots of tiny things, check it (say) every 1,000 operations so taking the mutex doesn't introduce a performance penalty.
Lastly, if you cancel a thread (or if it pthread_exits), the way you start it again is simply to call pthread_create again. It's then a new thread running the same code.
This isn't a technical question, but a conceptual one. My program needs to handle several tasks in background. In my case, I consider threads more appropriate than processes for several reasons :
Background tasks aren't heavy, but they have to be processed regularly.
All threads need to manipulate a shared resource. Complete processes would require setting up a shared memory segment, which isn't appropriate in my case (the resource doesn't have a fixed size). Of course, this resource is protected by a mutex.
Another thing I take into consideration is that the main() function needs to be able to end all backgrounds tasks when it wants to (which means joining threads).
Now, here are two implementations :
1 thread, looping inside.
void *my_thread_func(void* shared_ressource)
{
while(1){
do_the_job();
sleep(5);
}
}
// main()
pthread_create(&my_thread, NULL, my_thread_func, (void*)&shared_ressource);
pthread_kill(my_thread, 15);
// pthread_cancel(my_thread);
pthread_join(my_thread, NULL);
Note : In this case, main() needs to signal (or cancel) the thread before joining, otherwise it'll hang. This can be dangerous if the thread doesn't get time to sem_post before it gets terminated.
n threads, looping outside.
void *my_thread_func(void* shared_ressource)
{
do_the_job();
}
// main()
while(1){
pthread_create(&my_thread, NULL, my_thread_func, (void*)&shared_ressource);
pthread_join(my_thread, NULL);
sleep(5);
}
Note : In this case, main() wouldn't naturally hang on pthread_join, it would just have to kill its own continuous loop (using a "boolean" for instance).
Now, I would like some help comparing those two. Threads are lightweight structures, but is the spawning process too heavy for the second implementation ? Or is the infinite loop holding the thread when it shouldn't ? At the moment, I prefer the second implementation because it protects the semaphore : threads do not terminate before they sem_post it. My concern here is optimisation, not functionality.
Having your background threads continuously spawning and dying tends to be inefficient. It is usually much better to have some number of threads stay alive, servicing the background work as it becomes available.
However, it's often better to avoid thread cancellation, too. Instead, I advise using a condition variable and exit flag:
void *my_thread_func(void *shared_resource)
{
struct timespec timeout;
pthread_mutex_lock(&exit_mutex);
do
{
pthread_mutex_unlock(&exit_mutex);
do_the_job();
clock_gettime(CLOCK_REALTIME, &timeout);
timeout.tv_sec += 5;
pthread_mutex_lock(&exit_mutex);
if (!exit_flag)
pthread_cond_timedwait(&exit_cond, &exit_mutex, &timeout);
} while (!exit_flag)
pthread_mutex_unlock(&exit_mutex);
}
When the main thread wants the background thread to exit, it sets the exit flag and signals the condition variable:
pthread_mutex_lock(&exit_mutex);
exit_flag = 1;
pthread_cond_signal(&exit_cond);
pthread_mutex_unlock(&exit_mutex);
pthread_join(my_thread, NULL);
(You should actually strongly consider using CLOCK_MONOTONIC instead of the default CLOCK_REALTIME, because the former isn't affected by changes to the system clock. This requires using pthread_condattr_setclock() and pthread_cond_init() to set the clock used by the condition variable.)
I am currently trying to analyse a issue in third party source code where a thread (code snippet corresponding to THREAD-T1) is in infinite wait state. The suspicion is that the thread is stuck in pthread_cond_wait. The following are the details of the same.
Code description
T1 does an asynchronous call to an API exposed by T2.
Hence T1 moves to a blocking wait on a conditional variable (say cond_t).
The conditional variable cond_t is signalled in the callback event generated by T2.
The above cycle is repeated n times until the API returns success.
To consolidate, the above is a series of steps which makes the asynchronous call similar to a synchronous one by the use of condition variables.
Sample code
#define MAX_RETRY (3)
bool g_b_ret_val;
pthread_mutex_t g_cond_mutex;
pthread_mutex_t g_ret_val_mutex; /* Assume iniitailzed in the main thread */
pthread_cond_t g_cond_t; /* Assume iniitailzed in the main thread */
retry_async_call_routine() /* Thread-T1 */
{
while(( false == g_b_ret_val) && (retry < MAX_RETRY))
{
(void)invoke_async_api();
pthread_mutex_init(&g_cond_mutex, NULL);
pthread_mutex_lock(&g_cond_mutex);
pthread_cond_wait(g_cond_t, &g_cond_mutex);
pthread_mutex_unlock(&g_cond_mutex);
pthread_mutex_destroy(&g_cond_mutex);
retry ++ ;
}
}
callback_routine() /* Thread-T2 */
{
pthread_mutex_lock(&g_ret_val_mutex);
g_b_ret_val = true; /* May be false also on failure */
pthread_mutex_unlock(&g_ret_val_mutex);
pthread_cond_signal(&g_cond_t);
}
Known issues that I see in the code
Missing retest of condition in a while loop on pthread_cond_wait
Missing mutex lock while signalling
Questions
Please point out me on any more loop holes (or) possibility of infinite wait (if any).
g_cond_t is not reset using pthread_cond_destroy between successive waits, what is the behaviour of the same ? ( Any references regarding this)
This code seems absurd. You are not supposed to create and destroy a mutex just so you can wait on the condition variable. A mutex needs to be created before thread-shared data are used, then the mutex must be used to protect the shared data. In this case, that's g_ret_val_mutex which protects g_b_ret_val.
The condition variable itself is just used for waiting (with regular or timed wait) and signaling (signal or broadcast). It generally does not need its own lock, and in fact, having a separate one (as in the above loop) gets in the way of calling pthread_cond_wait, which takes only one mutex to unlock, not two. There's no need to destroy and re-create condition variables unless you need new/different attributes.
The key to "not getting stuck"—avoiding infinite wait—is to guarantee that, whenever a thread calls pthread_cond_wait, there is definitely some other thread that will, in the future, call pthread_cond_signal (or pthread_cond_broadcast). That is, the waiter tests "why to wait" first, with the "why" part locked, then waits only if the "why" part says "you should wait". The wake-up thread may use the same lock to determine that a wake-up is necessary, or—if the wake-up thread is "lazy", as in the above example—simply issues a "wake up call" every time.
The minimal change for correctness would thus seem to be to change the loop to read:
pthread_mutex_lock(&g_ret_val_mutex);
for (retry = 0; retry < MAX_RETRY && !g_b_ret_val; retry++) {
(void)invoke_async_api();
pthread_cond_wait(&g_cond_t, &g_ret_val_mutex);
}
success = g_b_ret_val; /* if false, we failed */
/* or: success = retry < MAX_RETRY; -- same result */
pthread_mutex_unlock(&g_ret_val_mutex);
(Aside: g_cond_t is a terrible name for a variable; the _t suffix is meant for types.)
It's sometimes wise to separate "some thread needs a wake-up" from "final result of that thread is success". If needed, I'd probably add that using a second boolean. Let's call it g_waiting, which we set true when callback_routine() is (supposedly) guaranteed to be called and it should do a wake-up event, and false when it's not guaranteed to be called or the wakeup is not required. This kind of coding allows you to switch to pthread_cond_timedwait, in case the asynchronous event might never occur for some reason.
Given that g_ret_val_mutex protects g_b_ret_val, it's appropriate to use that for the "waiting" flag as well—adding another mutex just offers more opportunities for problems, here. So now we get:
pthread_mutex_lock(&g_ret_val_mutex);
for (retry = 0; retry < MAX_RETRY && !g_b_ret_val; retry++) {
(void)invoke_async_api();
compute_wakeup_time(&abstime);
g_waiting = true;
pthread_cond_timedwait(&g_cond_t, &g_ret_val_mutex, &abstime);
if (g_waiting) {
/* timeout occurred, we never got our callback */
/* may want something special for this case */
} else {
/* wakeup occurred, result is in g_b_ret_val */
}
}
success = g_b_ret_val;
/* or: success = retry < MAX_RETRY; */
g_waiting = false;
pthread_mutex_unlock(&g_ret_val_mutex);
Meanwhile:
callback_routine() /* Thread-T2 */
{
pthread_mutex_lock(&g_ret_val_mutex);
g_b_ret_val = compute_success_or_failure();
if (g_waiting) {
g_waiting = false;
pthread_cond_signal(&g_cond_t);
}
pthread_mutex_unlock(&g_ret_val_mutex);
}
I've moved the signal to "inside" the mutex, although it's OK either way, so that I can do it only if g_waiting is set, and clear g_waiting. Since we hold the mutex, it's OK to clear g_waiting either before or after calling pthread_cond_signal (as long as no other code will interrupt the sequence).
Note: if we do start using timedwait, we need to find out whether it is OK to call invoke_async_api when another earlier invoke was used but no result was returned before the timeout.
I am trying to set-up race condition to see how it happens to get an understanding. I wrote the code below. This compiles without any issue but when I run it, it does not print the count at every run. If is run it twice or thrice , then the count is printed. Is my understanding correct that in this code it is not necessary that race condition actually takes place.
( if this is correct,then I am not sure how this exits since there are no boundary condition!). Can some one give me some idea if my understanding is incorrect or the code?
Thanks.
#include<stdio.h>
#include<pthread.h>
#include<stdlib.h>
void *banking(void *);
int main(){
int accounts[2]={0,0};
pthread_t tid1,tid2;
if(pthread_create(&tid1,NULL,banking,(void *)accounts))
{
perror("pthread_create");
return 1;
}
if(pthread_create(&tid2,NULL,banking,(void *)accounts))
{
perror("pthread_create");
return 1;
}
pthread_join(tid1, NULL);
pthread_join(tid2, NULL);//program now goes into infinite loop.
return 0;
}
void *banking(void * accounts){
int *ptr=accounts;
int count=0;
do{
int temp1=ptr[0];
int temp2=ptr[1];
int amount=rand();
ptr[0]=temp1-amount;
ptr[1]=temp2+amount;
//printf("%d \n %d\n",ptr[0],ptr[1]);
count++;
}while((ptr[0]+ptr[1])==0);
printf("%d\n",count);
//return NULL;
exit(0);
}
I was trying to implement pthread_exit(NULL) to achieve the logic where the thread would exit once it do-while loop is over but as per my understanding the other running thread will not stop this way,because of which the programs goes into an infinite loop. I realized that exit() from any thread terminates process and incorporated exit(0) . The code works fine for some values but at random generates two different 'count' values. This happens once in 10-12 tries. Please suggest if usage of exit in a threaded function is advisable and under what situation will I have two diff values of count.
1> first correct that mistakes which "Paul R" has sauggested. then
2>
you need to use pthread_join function to get succesfully completion of both thread..
here after creating both thread main process may be get over so at that time both threads are also gets over so to over come form this use othread_join for both thread in main()
add this code to end of at your main()
pthread_join(tid1, NULL);
pthread_join(tid2, NULL);
if you still not getting the funda of race condition then read belows part. which i have copied from one refrence book
Suppose that your program has a series of queued jobs that are processed by several
concurrent threads.The queue of jobs is represented by a linked list of struct job
objects.
After each thread finishes an operation, it checks the queue to see if an additional
job is available. If job_queue is non-null, the thread removes the head of the linked list
and sets job_queue to the next job on the list.
The thread function that processes jobs in the queue might look like Listing 4.10.
Listing 4.10 ( job-queue1.c) Thread Function to Process Jobs from the Queue
#include <malloc.h>
struct job {
/* Link field for linked list.
struct job* next;
*/
/* Other fields describing work to be done... */
};
/* A linked list of pending jobs.
struct job* job_queue;
*/
/* Process queued jobs until the queue is empty.
void* thread_function (void* arg)
{
while (job_queue != NULL) {
/* Get the next available job. */
struct job* next_job = job_queue;
/* Remove this job from the list. */
job_queue = job_queue->next;
/* Carry out the work. */
process_job (next_job);
/* Clean up. */
free (next_job);
}
return NULL;
}
*/
4.4
Synchronization and Critical Sections
Now suppose that two threads happen to finish a job at about the same time, but only
one job remains in the queue.The first thread checks whether job_queue is null; find-
ing that it isn’t, the thread enters the loop and stores the pointer to the job object in
next_job. At this point, Linux happens to interrupt the first thread and schedules the
second.The second thread also checks job_queue and finding it non-null, also assigns
the same job pointer to next_job. By unfortunate coincidence, we now have two
threads executing the same job.
To make matters worse, one thread will unlink the job object from the queue,
leaving job_queue containing null.When the other thread evaluates job_queue->next,
a segmentation fault will result.
This is an example of a race condition. Under “lucky” circumstances, this particular
schedule of the two threads may never occur, and the race condition may never
exhibit itself. Only under different circumstances, perhaps when running on a heavily
loaded system (or on an important customer’s new multiprocessor server!) may the
bug exhibit itself.
To eliminate race conditions, you need a way to make operations atomic. An atomic
operation is indivisible and uninterruptible; once the operation starts, it will not be
paused or interrupted until it completes, and no other operation will take place mean-
while. In this particular example, you want to check job_queue; if it’s not empty,
remove the first job, all as a single atomic operation.
I have a function say void *WorkerThread ( void *ptr).
The function *WorkerThread( void *ptr) has infinite loop which reads and writes continously from Serial Port
example
void *WorkerThread( void *ptr)
{
while(1)
{
// READS AND WRITE from Serial Port USING MUXTEX_LOCK AND MUTEX_UNLOCK
} //while ends
}
The other function I worte is ThreadTest
example
int ThreadTest()
{
pthread_t Worker;
int iret1;
pthread_mutex_init(&stop_mutex, NULL);
if( iret1 = pthread_create(&Worker, NULL, WorkerThread, NULL) == 0)
{
pthread_mutex_lock(&stop_mutex);
stopThread = true;
pthread_mutex_unlock(&stop_mutex);
}
if (stopThread != false)
stopThread = false;
pthread_mutex_destroy(&stop_mutex);
return 0;
}
In main function
I have something like
int main(int argc, char **argv)
{
fd = OpenSerialPort();
if( ConfigurePort(fd) < 0) return 0;
while (true)
{
ThreadTest();
}
return 0;
}
Now, when I run this sort of code with debug statement it runs fine for few hours and then throw message like "can't able to create thread" and application terminates.
Does anyone have an idea where I am making mistakes.
Also if there is way to run ThreadTest in main with using while(true) as I am already using while(1) in ThreadWorker to read and write infinitely.
All comments and criticism are welcome.
Thanks & regards,
SamPrat.
You are creating threads continually and might be hitting the limit on number of threads.
Pthread_create man page says:
EAGAIN Insufficient resources to create another thread, or a system-imposed
limit on the number of threads was encountered. The latter case may
occur in two ways: the RLIMIT_NPROC soft resource limit (set via
setrlimit(2)), which limits the number of process for a real user ID,
was reached; or the kernel's system-wide limit on the number of
threads, /proc/sys/kernel/threads-max, was reached.
You should rethink of the design of your application. Creating an infinite number of threads is not a god design.
[UPDATE]
you are using lock to set an integer variable:
pthread_mutex_lock(&stop_mutex);
stopThread = true;
pthread_mutex_unlock(&stop_mutex);
However, this is not required as setting an int is atomic (on probably all architectures?). You should use a lock when you are doing not-atomic operations, eg: test and set
take_lock ();
if (a != 1)
a = 1
release_lock ();
You create a new thread each time ThreadTest is called, and never destroy these threads. So eventually you (or the OS) run out of thread handles (a limited resource).
Threads consume resources (memory & processing), and you're creating a thread each time your main loop calls ThreadTest(). And resources are finite, while your loop is not, so this will eventually throw a memory allocation error.
You should get rid of the main loop, and make ThreadTest return the newly created thread (pthread_t). Finally, make main wait for the thread termination using pthread_join.
Your pthreads are zombies and consume system resources. For Linux you can use ulimit -s to check your active upper limits -- but they are not infinite either. Use pthread_join() to let a thread finish and release the resources it consumed.
Do you know that select() is able to read from multiple (device) handles ? You can also define a user defined source to stop select(), or a timeout. With this in mind you are able to start one thread and let it sleeping if nothing occurs. If you intent to stop it, you can send a event (or timeout) to break the select() function call.
An additional design concept you have to consider is message queues to share information between your main application and/or pthread. select() is compatible with this technique so you can use one concept for data sources (devices and message queues).
Here a reference to a good pthread reading and the best pthread book available: Programming with POSIX(R) Threads, ISBN-13:978-0201633924
Looks like you've not called pthread_join() which cleans up state after non-detached threads are finished. I'd speculate that you've hit some per process resource limit here as a result.
As others have noted this is not great design though - why not re-use the thread rather than creating a new one on every loop?