I have the following program that spawns two threads to print_something() and they both repeatedly print a specific string: thread 1 prints "Hi\n" and thread 2 prints "Bye\n":
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
void *print_something(int *k)
{
int n = 100;
int i;
if (*k) {
for (i = 0; i < 100; i++) {
printf("Hi\n");
}
} else {
for (i = 0; i < 100; i++) {
printf("Bye\n");
}
}
}
int main()
{
int x = 1, y = 0;
pthread_t t1, t2;
pthread_create(&t1, NULL, print_something, &x);
pthread_create(&t2, NULL, print_something, &y);
pthread_join(t1, NULL);
pthread_join(t2, NULL);
printf("End of program.\n");
return 0;
}
I expected them to run synchronously wherein the output in the terminal would be random such as:
Hi
Hi
Bye
Hi
Bye
...
But instead I always get thread 1 to finish its printing first before thread 2 will start printing:
Hi
Hi
...
Hi
Hi
Bye
Bye
...
Bye
Bye
End of program.
Why is the first thread blocking the second thread from printing?
Why is the first thread blocking the second thread from printing?
Who says it's blocking? Maybe starting a new thread takes long enough that the first additional thread (running in parallel with the original thread) finishes its printing (to stdout's buffer) before the second additional thread arrives at the point of trying to print anything.
On the other hand, POSIX does specify that the stdio functions perform operations on streams as if there was a lock associated with each stream that a thread must obtain upon entry to the function and releases upon exit. Thus, the first thread may indeed be blocking the second via the lock associated with stdout.
Moreover, when a thread unlocks a lock and then immediately tries to re-acquire the same lock, there is a high probability for that thread to succeed immediately despite other threads contending for the lock. As a result, when an entire loop body starts with acquiring a lock and ends with releasing that lock -- as is the case in your code for the lock associated stdout -- it is common for one thread to be able to monopolize the lock for many loop iterations.
I expected them to run synchronously wherein the output in the terminal would be random such as:
That's an unreasonable expectation. If two people each need to put in a hundred screws and are sharing a screwdriver, do you think they should hand off the screwdriver after each screw? It only makes sense to hand off the screwdriver when the one holding the screwdriver is tired.
Each thread spends the vast majority of its time accessing the console output stream. It can only do this by excluding the other thread. The behavior you expect would be atrocious.
Would they run on the same core? That would require a context switch after every line of output -- the worst performance possible for this code. Would they run on two cores? That would mean each core is waiting for the other core to finish with the console for about half the time -- also horrible performance.
Simply put, you expected your system to find a terrible way to do what you asked it to do. It found a much more efficient way -- letting one thread keep the console, finish what it was doing, and then letting the other go.
Related
This question already has an answer here:
Pthread_create() incorrect start routine parameter passing
(1 answer)
Closed 3 years ago.
I tried to build a program which should create threads and assign a Print function to each one of them, while the main process should use printf function directly.
Firstly, I made it without any synchronization means and expected to get a randomized output.
Later I tried to add a mutex to the Print function which was assigned to the threads and expected to get a chronological output but it seems like the mutex had no effect about the output.
Should I use a mutex on the printf function in the main process as well?
Thanks in advance
My code:
#include <stdio.h>
#include <pthread.h>
#include <errno.h>
pthread_t threadID[20];
pthread_mutex_t lock;
void* Print(void* _num);
int main(void)
{
int num = 20, indx = 0, k = 0;
if (pthread_mutex_init(&lock, NULL))
{
perror("err pthread_mutex_init\n");
return errno;
}
for (; indx < num; ++indx)
{
if (pthread_create(&threadID[indx], NULL, Print, &indx))
{
perror("err pthread_create\n");
return errno;
}
}
for (; k < num; ++k)
{
printf("%d from main\n", k);
}
indx = 0;
for (; indx < num; ++indx)
{
if (pthread_join(threadID[indx], NULL))
{
perror("err pthread_join\n");
return errno;
}
}
pthread_mutex_destroy(&lock);
return 0;
}
void* Print(void* _indx)
{
pthread_mutex_lock(&lock);
printf("%d from thread\n", *(int*)_indx);
pthread_mutex_unlock(&lock);
return NULL;
}
All questions of program bugs notwithstanding, pthreads mutexes provide only mutual exclusion, not any guarantee of scheduling order. This is typical of mutex implementations. Similarly, pthread_create() only creates and starts threads; it does not make any guarantee about scheduling order, such as would justify an assumption that the threads reach the pthread_mutex_lock() call in the same order that they were created.
Overall, if you want to order thread activities based on some characteristic of the threads, then you have to manage that yourself. You need to maintain a sense of which thread's turn it is, and provide a mechanism sufficient to make a thread notice when it's turn arrives. In some circumstances, with some care, you can do this by using semaphores instead of mutexes. The more general solution, however, is to use a condition variable together with your mutex, and some shared variable that serves as to indicate who's turn it currently is.
The code passes the address of the same local variable to all threads. Meanwhile, this variable gets updated by the main thread.
Instead pass it by value cast to void*.
Fix:
pthread_create(&threadID[indx], NULL, Print, (void*)indx)
// ...
printf("%d from thread\n", (int)_indx);
Now, since there is no data shared between the threads, you can remove that mutex.
All the threads created in the for loop have different value of indx. Because of the operating system scheduler, you can never be sure which thread will run. Therefore, the values printed are in random order depending on the randomness of the scheduler. The second for-loop running in the parent thread will run immediately after creating the child threads. Again, the scheduler decides the order of what thread should run next.
Every OS should have an interrupt (at least the major operating systems have). When running the for-loop in the parent thread, an interrupt might happen and leaves the scheduler to make a decision of which thread to run. Therefore, the numbers being printed in the parent for-loop are printed randomly, because all threads run "concurrently".
Joining a thread means waiting for a thread. If you want to make sure you print all numbers in the parent for loop in chronological order, without letting child thread interrupt it, then relocate the for-loop section to be after the thread joining.
My problem is that I cannot reuse cancelled pthread. Sample code:
#include <pthread.h>
pthread_t alg;
pthread_t stop_alg;
int thread_available;
void *stopAlgorithm() {
while (1) {
sleep(6);
if (thread_available == 1) {
pthread_cancel(alg);
printf("Now it's dead!\n");
thread_available = 0;
}
}
}
void *algorithm() {
while (1) {
printf("I'm here\n");
}
}
int main() {
thread_available = 0;
pthread_create(&stop_alg, NULL, stopAlgorithm, 0);
while (1) {
sleep(1);
if (thread_available == 0) {
sleep(2);
printf("Starting algorithm\n");
pthread_create(&alg, NULL, algorithm, 0);
thread_available = 1;
}
}
}
This sample should create two threads - one will be created at the program beginning and will try to cancel second as soon it starts, second should be rerunned as soon at it was cancelled and say "I'm here". But when algorithm thread cancelled once it doesn't start once again, it says "Starting algorithm" and does nothing, no "I'm here" messages any more. Could you please tell me the way to start cancelled(immediately stopped) thread once again?
UPD: So, thanks to your help I understood what is the problem. When I rerun algorithm thread it throws error 11:"The system lacked the necessary resources to create another thread, or the system-imposed limit on the total number of threads in a process PTHREAD_THREADS_MAX would be exceeded.". Actually I have 5 threads, but only one is cancelled, others stop by pthread_exit. So after algorithm stopped and program went to standby mode I checked status of all threads with pthread_join - all thread show 0(cancelled shows PTHREAD_CANCELED), as far as I can understand this means, that all threads stopped successfully. But one more try to run algorithm throws error 11 again. So I've checked memory usage. In standby mode before algorithm - 10428, during the algorithm, when all threads used - 2026m, in standby mode after algorithm stopped - 2019m. So even if threads stopped they still use memory, pthread_detach didn't help with this. Are there any other ways to clean-up after threads?
Also, sometimes on pthread_cancel my program crashes with "libgcc_s.so.1 must be installed for pthread_cancel to work"
Several points:
First, this is not safe:
int thread_available;
void *stopAlgorithm() {
while (1) {
sleep(6);
if (thread_available == 1) {
pthread_cancel(alg);
printf("Now it's dead!\n");
thread_available = 0;
}
}
}
It's not safe for at least reasons. Firstly, you've not marked thread_available as volatile. This means that the compiler can optimise stopAlgorithm to read the variable once, and never reread it. Secondly, you haven't ensured access to it is atomic, or protected it by a mutex. Either declare it:
volatile sig_atomic_t thread_available;
(or similar), or better, protect it by a mutex.
But for the general case of triggering one thread from another, you are better using a condition variable (and a mutex), using pthread_condwait or pthread_condtimedwait in the listening thread, and pthread_condbroadcast in the triggering thread.
Next, what's the point of the stopAlgorithm thread? All it does is cancel the algorithm thread after an unpredictable amount of time between 0 and 6 seconds? Why not just sent the pthread_cancel from the main thread?
Next, do you care where your algorithm is when it is cancelled? If not, just pthread_cancel it. If so (and anyway, I think it's far nicer), regularly check a flag (either atomic and volatile as above, or protected by a mutex) and pthread_exit if it's set. If your algorithm does big chunks every second or so, then check it then. If it does lots of tiny things, check it (say) every 1,000 operations so taking the mutex doesn't introduce a performance penalty.
Lastly, if you cancel a thread (or if it pthread_exits), the way you start it again is simply to call pthread_create again. It's then a new thread running the same code.
I'm learning thread synchronization and this is the demo to show how to lock critical data when a thread is executing:
http://ideone.com/7Do0l
(To run this code, compile it with the -pthread parameter in Linux/MacOS environment)
The program works as expected, but the sleep() function doesn't pause the execution between threads. My idea is to have one thread do the calculation at a time, then 1 second later another thread comes into play. Here is the code segment I'm fighting with:
while(1) {
//sleep(1); //(1) (Sleep for one second)
sem_wait(&mutex);
//sleep(1); //(2)
printf("Thread #%d is doing math. %d + 1 = %d.\n", (int) id, s, s+1);
s++;
//sleep(1); //(3)
sem_post(&mutex);
//sleep(1); //(4)
}
There are four positions I have tried to put the sleep() in. (1) and (4) result in no pauses between single threads but between two bunches of ten threads. (2) and (3) result in one thread gets executed repeatedly for very long time before another gets called.
Is there a remedy to this?
Update
There is a trick to make the program produce the result: generating the sleeping time randomly for each thread, but it's not consistent since two random numbers could be the same by accident.
Put it in the 3rd position, since you want a one second delay between printf messages.
If you want to make sure that all threads are initialized before any of them can enter into the critical section, modify the main function of the linked code to this
int main() {
pthread_t thread[10];
int i;
sem_init(&mutex, 0, 1);
sem_wait(&mutex);
for (i = 0; i<10; ++i)
pthread_create(&(thread[i]), NULL, job, (void*) i);
sem_post(&mutex);
sleep(100);
}
That's not really the kind of problem threads are designed to solve. You'd have to have a separate semaphore for each thread, have one thread loop through those, calling sem_post on a different one each second, and the rest just calling sem_wait. May as well just use the one thread.
I did some research and found that the only way to produce the desired output is the one I mentioned in the Update part. That is, instead of hard coded the sleep timer, just give each thread a random number:
// Sleep time in microseconds.
int st = rand() % 500000;
usleep(st);
And actually I have been over-worrying about two threads doing the same thing at once. Even though the two adjacent random timers could be accidentally the same, two threads never get executed on the same core of the CPU at the same time, in case the CPU is of multiple cores, no two instructions can modify the same memory content concurrently.
I am working on a program with a fixed number of threads in C using posix threads.
How can i be notified when a thread has been terminated due to some error?
Is there a signal to detect it?
If so, can the signal handler create a new thread to keep the number of threads the same?
Make the threads detached
Get them to handle errors gracefully. i.e. Close mutexs, files etc...
Then you will have no probolems.
Perhaps fire a USR1 signal to the main thread to tell it that things have gone pear shaped (i was going to say tits up!)
Create your threads by passing the function pointers to an intermediate function. Start that intermediate function asynchronously and have it synchronously call the passed function. When the function returns or throws an exception, you can handle the results in any way you like.
With the latest inputs you've provided, I suggest you do something like this to get the number of threads a particular process has started-
#include<stdio.h>
#define THRESHOLD 50
int main ()
{
unsigned count = 0;
FILE *a;
a = popen ("ps H `ps -A | grep a.out | awk '{print $1}'` | wc -l", "r");
if (a == NULL)
printf ("Error in executing command\n");
fscanf(a, "%d", &count );
if (count < THRESHOLD)
{
printf("Number of threads = %d\n", count-1);
// count - 1 in order to eliminate header.
// count - 2 if you don't want to include the main thread
/* Take action. May be start a new thread etc */
}
return 0;
}
Notes:
ps H displays all threads.
$1 prints first column where PID is displayed on my system Ubuntu. The column number might change depending on the system
Replace a.out it with your process name
The backticks will evaluate the expression within them and give you the PID of your process. We are taking advantage of the fact that all POSIX threads will have same PID.
I doubt Linux would signal you when a thread dies or exits for any reason. You can do so manually though.
First, let's consider 2 ways for the thread to end:
It terminates itself
It dies
In the first method, the thread itself can tell someone (say the thread manager) that it is being terminated. The thread manager will then spawn another thread.
In the second method, a watchdog thread can keep track of whether the threads are alive or not. This is done more or less like this:
Thread:
while (do stuff)
this_thread->is_alive = true
work
Watchdog:
for all threads t
t->timeout = 0
while (true)
for all threads t
if t->is_alive
t->timeout = 0
t->is_alive = false
else
++t->timeout
if t->timeout > THRESHOLD
Thread has died! Tell the thread manager to respawn it
If for any reason one could not go for Ed Heal's "just work properly"-approach (which is my favorite answer to the OP's question, btw), the lazy fox might take a look at the pthread_cleanup_push() and pthread_cleanup_pop() macros, and think about including the whole thread function's body in between such two macros.
The clean way to know whether a thread is done is to call pthread_join() against that thread.
// int pthread_join(pthread_t thread, void **retval);
int retval = 0;
int r = pthread_join(that_thread_id, &retval);
... here you know that_thread_id returned ...
The problem with pthread_join() is, if the thread never returns (continues to run as expected) then you are blocked. That's therefore not very useful in your case.
However, you may actually check whether you can join (tryjoin) as follow:
//int pthread_tryjoin_np(pthread_t thread, void **retval);
int retval = 0;
int r = pthread_tryjoin_np(that_thread_id, &relval);
// here 'r' tells you whether the thread returned (joined) or not.
if(r == 0)
{
// that_thread_id is done, create new thread here
...
}
else if(errno != EBUSY)
{
// react to "weird" errors... (maybe a perror() at least?)
}
// else -- thread is still running
There is also a timed join which will wait for the amount of time you specified, like a few seconds. Depending on the number of threads to check and if your main process just sits around otherwise, it could be a solution. Block on thread 1 for 5 seconds, then thread 2 for 5 seconds, etc. which would be 5,000 seconds per loop for 1,000 threads (about 85 minutes to go around all threads with the time it takes to manage things...)
There is a sample code in the man page which shows how to use the pthread_timedjoin_np() function. All you would have to do is put a for loop around to check each one of your thread.
struct timespec ts;
int s;
...
if (clock_gettime(CLOCK_REALTIME, &ts) == -1) {
/* Handle error */
}
ts.tv_sec += 5;
s = pthread_timedjoin_np(thread, NULL, &ts);
if (s != 0) {
/* Handle error */
}
If your main process has other things to do, I would suggest you do not use the timed version and just go through all the threads as fast as you can.
I'm running a highly threaded application (500+ threads). I need to trace some data from them, and to do so I was printing from the thread. The output is only cut off it seems. I've also made sure to flush stdout often and I've also tried using a mutex to coordinate output. None of those solutions have worked.
This is the thread in question:
void* troutine(void* tmp) {
a = RDTSC();
chance = Park(state);
b = RDTSC();
printf("%s.%i.%c : %lli\n", IMPLEMENTATION, *(int*)tmp, 'T', b-a);
usleep(RAND(50));
a = RDTSC();
Leave(chance, state);
b = RDTSC();
printf("%s.%i.%c : %lli\n", IMPLEMENTATION, *(int*)tmp, 'T', b-a);
fflush(stdout);
pthread_exit(NULL);
}
Only about half the print statements actually print, which is the problem. I need to make sure they all print, the order doesn't matter, and none of the output is interweaved.
EDIT main.c
for(i = 0; i < 4000; i++)
while(!pthread_create(&tmp, NULL, &troutine, (void*)&testNum));
The while loop is so that I ensure the creation of 4k threads as sometimes pthread_create fails with so many threads active. Also, even when I only set the loop to make i < 4 threads, I still get ~300 lines of output (as opposed to 8).
(1) You will fall out of main() before your threads finish. Either join the threads or put a pthread_exit() in main() so it doesn't kill your running threads when it exits.
for(i = 0; i < 4000; i++)
while(!pthread_create(&tmp, NULL, &troutine, (void*)&testNum));
(2) Pthread_create returns 0 on success. So the above while loop is saying "while successful, keep creating threads". That would explain so much output when i is only 4.
Edit 2: Another possibility is that your problem is outside of this code and that something is calling exit (if not crashing) so that half of your threads never finish. It would really help to know more about what you mean by "cut-off".
[As R mentions, this shouldn't be necessary. Only leaving it so the comment thread makes sense.]
When you say you're using a lock, are you using some kind of global mutex like:
pthread_mutex_lock(mutex);
printf("%s.%i.%c : %lli\n", IMPLEMENTATION, *(int*)tmp, 'T', b-a);
pthread_mutex_unlock(mutex);
because I don't see that in your example. Note that mutex needs to be defined in the above example, and also needs to be a pointer.