I have a gateway server and 2 clients.(made in Oracle VM VirtualBox).
In my gateway server, I have listener.c listening all packets.(in a while(1) loop).
If the client sends a valid token, I have to put it in my authorized MACs list and all packets coming from authorized MACs must be forwarded for 180 secs. (in other words, given internet access for 180 secs).
Before 180 sec., the thing with a single user it was working.
/* authorized MACs list */
char *auth_macs[5];
int client;
pthread_mutex_t lock;
/* after 3 min remove the client from auth list */
void *timer(void *arg){
sleep(180);
pthread_mutex_lock(&lock);
auth_macs[client] = " ";
client--;
pthread_mutex_unlock(&lock);
return NULL;
}
This is how I tried to implement my timer thread. client is a global variable that changes in main function.
if(has_token == 1){
client++;
sprintf(client_ip, "./accept.sh %s", sender);
system(client_ip);
auth_macs[client] = client_mac;
/* start timer thread */
pthread_t tid;
pthread_create(&tid, NULL, timer,NULL);
pthread_join(tid, NULL);
}
This is where I start this thread. accept.sh is a shell script to allow forwarding.
My problem is I thought sleep(180) in timer thread was only going to stop itself. But instead, listener.c stops receiving packets.
How can I fix this? I want timer to wait 180 secs but still be able to receive packets in main function.
sleep() only suspends the calling thread. So, it doesn't affect the main thread.
What's problematic is the pthread_join() call:
pthread_create(&tid, NULL, timer,NULL);
pthread_join(tid, NULL);
This effectively renders the multi-threading pointless. Because there's only thread that's ever going to make progress as the main thread waits until the created thread is completed.
You need to remove pthread_join() call and possibly it outside the while(1) loop if main thread needed to join. Alternatively, you can call pthread_exit(0) outside the thread creation loop so that main completes execution while the rest of the threads, if still alive when main thread
breaks its loop, can continue execution.
Related
I'm currently learning concurrent programming. I have two threads and I want them to act like this:
the first thread runs N times while the second one waits
when the first one is done, the second does its job and the first waits
when the second thread is done, repeat.
I'm trying to do it in C using pthread library. Here's some pseudocode (hopefully understandable)
int cnt = 0;
void* thread1(){
while(1){
// thread 1 code
cnt ++;
if(cnt == N){
let_thread2_work();
wait_thread2();
}
}
}
void* thread2(){
while(1){
wait_thread1();
// thread2 code
cnt = 0;
let_thread1_work();
}
}
Can anyone please help me ?
Like David Schwartz commented, this doesn't make a ton of sense with just thread1 and thread2 waiting on each other, not actually doing any work in parallel. But maybe eventually you want multiple "thread1"s, all processing jobs at the same time until they finish a batch of N jobs, then they all stop and wait for "thread2" to do some kind of post-processing before the pool of worker threads start back up.
In this situation I would consider using a couple condition variables, one for your worker threads to communicate to your post-processing thread that they're waiting, and one for your post-processing thread to tell the workers to start working again. You can declare a condition variable and a helper mutex in the global scope right next to your "cnt" variable, which I'm calling "jobs_done" for clarity:
#include<pthread.h>
#DEFINE NUM_WORKERS 1 // although it doesn't make sense to just have 1
#DEFINE BATCH_SIZE 50 // this is "N"
// we're going to keep track of how many jobs we have done in
// this variable
int jobs_done = 0;
// when a worker thread checks jobs_done and it's N or greater,
// that means we have to wait for the post-processing thread to set
// jobs_done back to 0. so the worker threads "wait" on the
// condition variable, and the post-processing thread "broadcasts"
// to the condition variable to wake them all up again once it's
// done its work
pthread_cond_t jobs_ready_cv;
// we're going to use this helper mutex. whenever any thread
// reads or writes to the jobs_done variable, we have to lock this
// mutex. that includes the worker threads when they check to see if
// they're ready to wake up again.
pthread_mutex_t jobs_mx;
// here's how the worker threads will communicate to the post-process
// thread that a batch is done. to make sure that all N jobs are fully
// complete before postprocessing happens, we'll use this variable to
// keep track of how many threads are waiting for postprocessing to finish.
int workers_done = 0;
// we'll also use a separate condition variable and separate mutex to
// communicate to the postprocess thread.
pthread_cond_t workers_done_cv;
pthread_mutex_t workers_done_mx;
Then in your setup code, initialize the condition variables and helper mutexes:
int main() { // or something
pthread_cond_init(&jobs_ready_cv, NULL);
pthread_mutex_init(&jobs_mx, NULL);
pthread_cond_init(&workers_done_cv, NULL);
pthread_mutex_init(&workers_done_mx, NULL);
...
}
So, your worker threads (or "thread1"), before taking a job, will check to see how many jobs have been taken. If N (here, BATCH_SIZE) have been taken, then it updates a variable to indicate that it has no work left to do. If it finds that all of the worker threads are done, then it signals the postprocess thread ("thread2") through workers_done_cv. Then, the thread waits for a signal from the postprocess thread through `
void* worker_thread(){
while(1){
/* first, we check if the batch is complete. we do this first
* so we don't accidentally take an extra job.
*/
pthread_mutex_lock(&jobs_mx);
if (jobs_done == BATCH_SIZE) {
/* if BATCH_SIZE jobs have been done, first let's increment workers_done,
* and if all workers are done, let's notify the postprocess thread.
* after that, we release the workers_done mutex so the postprocess
* thread can wake up from the workers_done condition variable.
*/
pthread_mutex_lock(&workers_done_mx);
++workers_done;
if (workers_done == NUM_WORKERS) {
pthread_cond_broadcast(&workers_done_cv);
}
pthread_mutex_unlock(&workers_done_mx);
/* now we wait for the postprocess thread to do its work. this
* unlocks the mutex. when we get the signal to start doing jobs
* again, the mutex will relock automatically when we wake up.
*
* note that we use a while loop here to check the jobs_done
* variable after we wake up. That's because sometimes threads
* can wake up on accident even if no signal or broadcast happened,
* so we need to make sure that the postprocess thread actually
* reset the variable. google "spurious wakeups"
*/
while (jobs_done == BATCH_SIZE) {
pthread_cond_wait(&jobs_ready_cv, &jobs_mx);
}
}
/* okay, now we're ready to take a job.
*/
++jobs_done;
pthread_mutex_unlock(&jobs_mx);
// thread 1 code
}
}
Meanwhile, your postprocess thread waits on the workers_done_cv immediately, and doesn't wake up until the last worker thread is done and calls pthread_cond_broadcast(&workers_done_cv). Then, it does whatever it needs to, resets the counts, and broadcasts to the worker threads to wake them back up.
void* postprocess_thread(){
while(1){
/* first, we wait for our worker threads to be done
*/
pthread_mutex_lock(&workers_done_mx);
while (workers_done != NUM_WORKERS) {
pthread_cond_wait(&workers_done_cv, &workers_done_mx);
}
// thread2 code
/* reset count of stalled worker threads, release mutex */
workers_done = 0;
pthread_mutex_unlock(&workers_done_mx);
/* reset number of jobs done and wake up worker threads */
pthread_mutex_lock(&jobs_mx);
jobs_done = 0;
pthread_mutex_unlock(&jobs_mx);
pthread_cond_broadcast(&jobs_ready_cv);
}
}
Also take heed of David Schwartz's advice that you probably don't actually need the postprocessing thread to wait on the worker threads. If you don't need this, then you can get rid of the condition variable that makes the worker threads wait for the postprocess thread, and this implementation becomes a lot simpler.
edit: mutex protected the assignment to jobs_done in postprocess_thread(), added a forgotten ampersand
One solution is to use mutex:
https://www.geeksforgeeks.org/mutex-lock-for-linux-thread-synchronization/
Another solution, but using semaphore:
https://www.geeksforgeeks.org/difference-between-binary-semaphore-and-mutex/?ref=rp
Or creating own lightthread schedule.
for exemple:
http://www.dunkels.com/adam/pt/ [edited]
I'm building a project in c language (using openwrt as OS) to upload files to FTP server. and i'm using MQTT for the incoming data. So for each topic i subscribe to, i save this data and then upload it to the FTP server and to keep things going smoothly, each time i need to upload a file i just use a thread to do the job.
and to make sure the program doesn't run too much threads, each topic is allowed to create one thread. and i'm using a variable (like mutex but it's not pthread_mutex_t because i don't need to block a thread, i want to skip that step and upload the next file). i though with this technique i'm safe but after running the program for 15 min i get this error 11 which said resource temporarily unavailable when the program try to create a thread (pthread_create).
one of my tries to figure out what could be the problem is.
i used the pthread_join() function which is not an option in my condition but just to make sure that each thread is finished and not running in permanent loop. the program was running over than an hour and the error didn't show up again. and of course each thread was finished as intended.
and i'm 90% sure that each topic is creating only on thread and the next one will be create only if the previous one finished. (i was tracking the status of the variable before and after the thread creation).
i set the max thread from here "/proc/sys/kernel/threads-max" to 2000 (2000 is more than enough since i don't have too much topics)
upload function (this will create the thread):
void uploadFile(<args...>, bool* locker_p){
*locker_p = true;
args->uploadLocker_p = uploadLocker_p;
<do something here>
pthread_t tid;
int error = pthread_create(&tid, NULL, uploadFileThread, (void*)args);
if(0 != error){
printf("Couldn't run thread,(%d) => %s\n", error, strerror(error));
}
else{
printf("Thread %d\n", tid);
}
}
upload thread:
void *uploadFileThread(void *arg){
typeArgs* args = (typeArgs*)arg;
<do something like upload the file>
*(args->uploadLocker_p) = false;
free(args);
return NULL;
//pthread_exit(0);
}
The default stack size for the created threads is eating too much virtual memory.
Essentially, the kernel is telling your process that it has so much virtual memory already in use that it doesn't dare give it any more, because there isn't enough RAM and swap to back it up if the process were to suddenly use it all.
To fix, create an attribute that limits the per-thread stack to something sensible. If your threads do not use arrays as local variables, or do deep recursion, then 2*PTHREAD_STACK_MIN (from <limits.h>) is a good size.
The attribute is not consumed by the pthread_create() call, it is just a configuration block, and you can use the same one for any number of threads you create, or create a new one for each thread.
Example:
pthread_attr_t attrs;
pthread_t tid;
int err;
pthread_attr_init(&attrs);
pthread_attr_setstacksize(&attrs, 2 * PTHREAD_STACK_MIN);
err = pthread_create(&tid, &attrs, uploadFileThread, (void *)args);
pthread_attr_destroy(&attrs);
if (err) {
/* Failed, errno in err; use strerror(err) */
} else {
/* Succeeded */
}
Also remember that if your uploadFileThread() allocates memory, it will not be freed automatically when the thread exits. It looks like OP already knows this (as they have the function free the argument structure when it's ready to exit), but I thought it a good idea to point it out.
Personally, I like to use a thread pool instead. The idea is that the upload workers are created beforehand, and they'll wait for a new job. Here is an example:
pthread_mutex_t workers_lock;
pthread_mutex_t workers_wait;
volatile struct work *workers_work;
volatile int workers_idle;
volatile sig_atomic_t workers_exit = 0;
where struct work is a singly-linked list protected by workers_lock, workers_idle is initialized to zero and incremented when waiting for new work, workers_wait is a condition variable signaled when new work arrives under workers_lock, and workers_exit is a counter that when nonzero, tells that many workers to exit.
A worker would be basically something along
void worker_do(struct work *job)
{
/* Whatever handling a struct job needs ... */
}
void *worker_function(void *payload __attribute__((unused)))
{
/* Grab the lock. */
pthread_mutex_lock(&workers_lock);
/* Job loop. */
while (!workers_exit) {
if (workers_work) {
/* Detach first work in chain. */
struct work *job = workers_work;
workers_work = job->next;
job->next = NULL;
/* Work is done without holding the mutex. */
pthread_mutex_unlock(&workers_lock);
worker_do(job);
pthread_mutex_lock(&workers_lock);
continue;
}
/* We're idle, holding the lock. Wait for new work. */
++workers_idle;
pthread_cond_wait(&workers_wait, &workers_lock);
--workers_idle;
}
/* This worker exits. */
--workers_exit;
pthread_mutex_unlock(&workers_lock);
return NULL;
}
The connection handling process can use idle_workers() to check the number of idle workers, and either grow the worker thread pool, or reject the connection as being too busy. The idle_workers() is something like
static inline int idle_workers(void)
{
int result;
pthread_mutex_lock(&workers_lock);
result = workers_idle;
pthread_mutex_unlock(&workers_lock);
return result;
}
Note that each worker only holds the lock for very short durations, so the idle_workers() call won't block for long. (pthread_cond_wait() atomically releases the lock when it starts waiting for a signal, and only returns after it re-acquires the lock.)
When waiting for a new connection in accept(), set the socket nonblocking and use poll() to wait for new connections. If the timeout passes, examine the number of workers, and reduce them if necessary by calling reduce_workers(1) or similar:
void reduce_workers(int number)
{
pthread_mutex_lock(&workers_lock);
if (workers_exit < number) {
workers_exit = number;
pthread_cond_broadcast(&workers_wait);
}
pthread_mutex_unlock(&workers_lock);
}
To avoid having to call pthread_join() for each thread – and we really don't even know which threads have exited here! – to reap/free the kernel and C library metadata related to the thread, the worker threads need to be detached. After creating a worker thread tid successfully, just call pthread_detach(tid);.
When a new connection arrives and it is determined to be one that should be delegated to the worker threads, you can, but do not have to, check the number of idle threads, create new worker threads, reject the upload, or just append the work to the queue, so that it will "eventually" be handled.
This code plays a sound clip by creating a thread to do it. When bleep() runs, it sets the global variable bleep_playing to TRUE. In its main loop, if it notices that bleep_playing has been set to FALSE, it terminates that loop, cleans up (closing files, freeing buffers), and exits. I don't know the correct way to wait for a detached thread to finish. pthread_join() doesn't do the job. The while loop here continually checks bleep_id to see if it's valid. When it isn't, execution continues. Is this the correct and portable way to tell a thread to clean up and terminate before the next thread is allowed to be created?
if (bleep_playing) {
bleep_playing = FALSE;
while (pthread_kill(bleep_id, 0) == 0) {
/* nothing */
}
}
err = pthread_create(&bleep_id, &attr, (void *) &bleep, &effect);
I
Hmm... pthread_join should do the job. As far as I remember the thread has to call pthread_exit...?
/* bleep thread */
void *bleep(void *)
{
/* do bleeping */
pthread_exit(NULL);
}
/* main thread */
if (pthread_create(&thread, ..., bleep, ...) == 0)
{
/*
** Try sleeping for some ms !!!
** I've had some issues on multi core CPUs requiring a sleep
** in order for the created thread to really "exist"...
*/
pthread_join(&thread, NULL);
}
Anyway if it isn't doing its thing you shouldn't poll a global variable since it will eat up your CPU. Instead create a mutex (pthread_mutex_*-functions) which is initially locked and freed by the "bleep thread". In your main thread you can wait for that mutex which makes your thread sleep until the "bleep thread" frees the mutex.
(or quick & and dirty: sleep for a small amount of time while waiting for bleep_playing becoming FALSE)
I have a capture program which in addition do capturing data and writing it into a file also prints some statistics.The function that prints the statistics
static void report(void)
{
/*Print statistics*/
}
is called roughly every second using an ALARM that expires every second.So The program is like
void capture_program()
{
pthread_t report_thread
while()
{
if(pthread_create(&report_thread,NULL,report,NULL)){
fprintf(stderr,"Error creating reporting thread! \n");
}
/*
Capturing code
--------------
--------------
*/
if(doreport)
/*wakeup the sleeping thread.*/
}
}
void *report(void *param)
{
//access some register from hardware
//sleep for a second
}
The expiry of the timer sets the doreport flag.If this flag is set report() is called which clears the flag.
How do I wake up the sleeping thread (that runs the report()) when the timer goes off in the main thread?
You can sleep a thread using sigwait, and then signal that thread to wake up with pthread_kill. Kill sounds bad, but it doesn't kill the thread, it sends a signal. This method is very fast. It was much faster than condition variables. I am not sure it is easier, harder, safer or more dangerous, but we needed the performance so we went this route.
in startup code somewhere:
sigemptyset(&fSigSet);
sigaddset(&fSigSet, SIGUSR1);
sigaddset(&fSigSet, SIGSEGV);
to sleep, the thread does this:
int nSig;
sigwait(&fSigSet, &nSig);
to wake up (done from any other thread)
pthread_kill(pThread, SIGUSR1);
or to wake up you could do this:
tgkill(nPid, nTid, SIGUSR1);
Our code calls this on the main thread before creating child threads. I'm not sure why this would be required.
pthread_sigmask(SIG_BLOCK, &fSigSet, NULL);
How do I wake up the sleeping thread (that runs the report()) when the
timer goes off in the main thread?
I think a condition variable is the mechanism you are looking for. Have the report-thread block on the condition variable, and the main thread signal the condition variable whenever you want the report-thread to wake up (see the link for more detailed instructions).
I had a similar issue when coding an UDP chat server: there is a thread_1 that only works when an alarm interruption (timeout to see if the client is still alive) OR another thread_2 (this thread meets client requests) signals arrives. What I did was put this thread_1 to sleep (sleep(n*TICK_TIMER), where TICK_TIMER is the alarm expiration value, n is some integer >1), and wake up this thread with SIGALRM signal. See sleep() doc
The alarm handler ( to use this you have to init it: "signal(SIGALRM, tick_handler); alarm(5);")
void tick_handler(){tick_flag++; alarm(5); }
will send a SIGALRM when timeout occurs.
And the command to wake this sleep thread_1 from another thread_2 is:
pthread_kill(X,SIGALRM);
where X is a pthread_t type. If your thread_1 is your main thread, you can get this number by pthread_t X = pthread_self();
I cant terminate the thread, it keeps sending things even after I close the terminal...
void *RTPfun(void * client_addr);
int main(int argc, char *argv[])
{
pthread_t RTPthread;
pthread_create(&RTPthread, NULL, &RTPfun, (void*)client_addr);
...
...
pthread_exit(&RTPfun);
return 0;
}
void *RTPfun(void * client_addr)
{
...
...
return 0;
}
Can someone tell me what am I doing wrong?
Thanks!
pthread_exit kills your current thread.
Notice, that if you kill the main thread as you do, it does not terminate the process. Other threads keep running.
You probably want to use pthread_cancel.
More generally though, killing threads is a bad idea. Correct way is to ask your threads politely to terminate and wait till they do.
If you call exit() from main, it will terminate main thread with all other thread.
If you call the method pthread_exit() from your main it will terminate main thread and let other thread will run continuously.
In your case you are calling pthread_exit() from main so your main thread get terminated, and other thread running until thread gets finish the job.
To cancel thread Add Below in RTPfun and add pthread_cancel in main.
/* call this when you are not ready to cancel the thread */
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, NULL);
/* call this when you are ready to cancel the thread */
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
Working sample code:
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
void *RTPfun(void * client_addr);
int main(int argc, char *argv[])
{
pthread_t RTPthread;
int client_addr;
pthread_create(&RTPthread, NULL, &RTPfun, (void*)client_addr);
sleep(2);
pthread_cancel(RTPthread);
pthread_join(RTPthread, NULL);
return 0;
}
void *RTPfun(void * client_addr)
{
int count = 0;
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, NULL);
while(1) {
if(count > 10) {
printf("thread set for cancel\n");
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
}
sleep(1);
printf("count:%d\n", count);
count ++;
}
return 0;
}
Used sleep in code for just understanding.
My understanding:
From "man pthread_exit" clearly talks about the description and rational behavior.
we will need to clean-up all respected resource that are been used in the thread created.
" The pthread_exit() function shall terminate the calling thread and make the value value_ptr available to any successful join with the terminating thread."
we shall pass "value_ptr" to exit(value_ptr) --> this will help to analyse what was the results of exit process.
obviously exit() - call exiting from the process, will release resources that used for.
In other way, you can create pthread in detached state, this attribute will reclaim the resources implicitly when pthread_exit . Refer "man pthread_detach".
We don't need to use pthread_join .. either go for pthread_detach again its simple go ahead set attribute .
/* set the thread detach state */
ret = pthread_attr_setdetachstate(&tattr,PTHREAD_CREATE_DETACHED);
Thanks.
Sankar
Your pthread_exit() exits the current thread. The parameter is used to pass a return value to any thread that then wants to join with it - and not to specify which thread to exit, as your code is implying.
The nicest way to do what you want is to use pthread_cancel() from your main thread. It takes the thread to cancel as a parameter, and then sends a cancellation request to that thread. Notice though, that by default cancellation is deferred, so your thread will keep on running until it hits a function that is a cancellation point - if you don't use any of those functions, you can insert an explicit cancellation point with a call to pthread_testcancel().
If you need to do some cleanup (for instance to free allocated memory, unlock mutexes, etc), you can insert a cleanup handler that is automatically called when canceling the thread. Have a look at pthread_cleanup_push() for this.
You can also set your thread to use asynchronous cancellation - your thread can then be canceled immediately, without hitting a cancellation point. However, asynchronous cancellation should only be used if you aren't using any system calls at all (that is, it's okay if you're purely doing calculations on already available data - but not if you're for instance using printf, file I/O, socket communication or similar), as otherwise you'll risk your system ending up in an inconsistent state.
After calling pthread_cancel(), your main thread should call pthread_join() on the canceled thread, to make sure all thread resources are cleaned up (unless you create the thread as detached).
You can of course also just have a shared doExit flag between the two threads, that the main thread can set, and which the other thread looks at from time to time. It's basically a manual way of using pthread_cancel().