I am sorry for the basicness of this question, but I am having an issue here. I have a client-server program. I don't know before hand how many connections will come but they are not infinite. And at the end , after all connections are closed some results are output. But the problem I am having is, accepting connections is in an infinite while loop, how is it stoppedd to output the result.
Thanks
you need to have some form of condition to break out of you loop, in your case, a timeout would probably work the best, basically meaning, if you don't get any new clients for x seconds, you stop looking for clients, same goes for any for of connection error.
Anything more requires looking at the code you are using.
Handling EINTR on error from accept(2) with terminating the program and hitting ^C usually works.
You could install a handler for the SIGTERM signal which would set a global volatile sig_atomic_t variable, and test that variable in your multiplexing loop (probably around poll or select). Remember that signal handlers cannot call many functions (only the async-signal-safe ones).
Catching nicely SIGTERM is expected from most Linux or Posix servers.
You could consider using an event handling library like libev, libevent etc.
Although my background is with Windows NT the function "names" are ones that name generic threading or process functions that should be available in any multi-threading environment.
If the main thread can determine when the child thread in question should terminate it can either do this by having the child thread loop on a boolean - such as "terminate_conditon" - or by terminating the thread throught its handle.
// child thread
terminate_condition=FALSE;
while (!terminate_condition)
{
// accept connections
}
child_thread_done=TRUE;
// output results
exit_thread ();
// main thread
child_thread_done=FALSE;
child_thread=create_thread (...);
// monitor connections to determine when done
terminate_condition=TRUE;
while (!child_thread_done)
{
sleep (1);
}
// or maybe output results here?
exit_process ();
This controlled termination solution requires that only one thread writes to the child_thread_done boolean and that any other thread only reads.
Or
// child thread
while (1)
{
// accept connections
}
// main thread
child_thread=create_thread (...);
// monitor connections to determine when done
kill_thread (child_thread);
// output results
exit_process ();
The second form is messier since it simply kills the child thread. In general it is better to have the child thread perform a controlled termination, especially if it has allocated resources (which become the responsibility of the process as a whole rather than just the allocating thread).
If there are many child threads working with connections a synchronized termination mechanism is necessary: either a struct with as many members as there are child threads (a terminating thread sets its "terminated" boolean to true, terminates and the main thread monitors the struct to make sure all child "terminated" booleans are true before proceeding) or a counter containing the number of child threads operating (when a child is about to terminate it takes exclusive control of the counter via a spinlock, decrements it and frees the lock before terminating: the main thread doesn't do anything before the counter contains zero).
Related
Is there a way to close all created threads if I don't have a list of their identifiers?
It is assumed that I only need the main thread, and the rest can be closed.
It's usually a good idea to have threads in charge of their own lifetime, periodically checking for some event indicating they should shut down. This usually make the architecture of your code much easier to understand.
What I'm talking about is along the lines of (pseudo-code):
def main():
# Start up all threads.
synchronised runFlag = true
for count = 1 to 10:
start thread threadFn, receiving id[count]
sleep for a bit
# Tell them all to exit, then wait.
synchronised runFlag = false
for count = 1 to 10:
wait for thread id[count] to exit
exit program
def threadFn():
initialise
# Thread will do its stuff until told to stop.
while synchronised runFlag:
do something relatively quick
exit thread
The periodic checking is a balance between efficiency of the thread loop and the amount of time you may have to wait for the thread to exit.
And, yes, I'm aware that pseudo-code uses identifiers (that you specifically stated you didn't have), but that's just one example of how to effect shutdown. You could equally, for example:
maintain a (synchronised) thread count incremented as a thread starts and decremented when it stops, then wait for it to reach zero;
have threads continue to run while a synchronised counter hasn't changed from the value it was when the thread started (you could just increment the counter in main then freely create a new batch of threads, knowing that the old ones would eventually disappear since the counter is different).
do one of a half dozen other things, depending on your needs :-)
This "lifetime handled by thread" approach is often the simplest way to achieve things since the thread is fully in control of when things happen to it. The one thing you don't want is a thread being violently killed from outside while it holds a resource lock of some sort.
Some threading implementations have ways to handle that with, for example, cancellability points, so you can cancel a thread from outside and it will die at such time it allows itself to. But, in my experience, that just complicates things.
In any case, pthread_cancel requires a thread ID so is unsuitable based on your requirements.
Is there a way to close all created threads if I don't have a list of their identifiers?
No, with POSIX threads there is not.
It is assumed that I only need the main thread, and the rest can be closed.
What you could do is have main() call fork() and let the calling main() (the parent) return, which will end the parent process along with all its thread.
The fork()ed off child process would live on as a copy of the original parent process' main() but without any other threads.
If going this route be aware, that the threads of the process going down might very well run into undefined behaviour, so that strange things might happen including messy left-overs.
All in all a bad approach.
Is there a way to close all created threads if I don't have a list of their identifiers? It is assumed that I only need the main thread, and the rest can be closed.
Technically, you can fork your process and terminate the parent. Only the thread calling fork exists in the new child process. However, the mutexes locked by other threads remain locked and this is why forking a multi-threaded process without immediately calling exec may be unwise.
Hi I'm stuck with this problem and I can't figure out which is the best solution for that:
I have a process with some threads. All the threads have signal masked and only one (SignalHandler) that check if there are pending signals.
while(!shouldEnd){
if(!sigwait(&set, &signum)){
if(signum == SIGINT){
fprintf(stderr, "Ricevuto SIGINT\n");
}
else{
//TODO
}
}
}
Now I would like to terminate even other threads when it receive a SIGINT (for instance). How can I do it properly ? The main problem is that the others thread might be waiting on a condition variable or waiting in a accept() (for socket connections). I think would be a good thing put a variable like "shouldEnd" and set it to 1 when threads should stop working. Is that a good idea ? And what about the waiting one ? Mostly for the one waiting for a socket connection ?
So, first and foremost, we don't know what you're using for threads. You might consider using libBoost for this, as it will make your life easier with RAII style locks and whatnot.
Anyhow, only the elected thread of your process (typically main(), in most examples), will catch the signal, unless you've enabled signal masking. When you want the threads to shut down cleanly, you just need to:
Wakeup the threads waiting on accept() by modifying your FD set to include a pipe that can also wake up the blocking call.
Simply signal the condvars the other threads are waiting on, and set some sort of mutex-protected boolean/flag to notify the thread that it should exit early (ie: mutexLock()l; bool bExitEarly = true; signal(condVar); mutexUnlock();).
Assuming you spawned the threads as joinable (ie: non-detached), just make sure you have a copy of the pointer to each thread object, and call thread_join() on each of them after you've signaled them to stop. This will ensure that the threads are fully stopped before main() exits. If you don't do this, main() could exit before the threads are done, and just forcibly kill off the threads while they are in the middle of their shutdown logic, which is messy, and could cause your program to crash or worse.
This is kind of generic question - however I met this problem several times already and I still haven't found the best possible solution.
Let's imagine you have program (e.g. HTTP application server) that is multithreaded and that communicates over sockets (TCP, Unix, ...). Main thread is using asynchronous IO and select() or poll() POSIX calls to dispatch traffic from/to sockets. There are also worker threads that process requests and provides responses. To send response back to the client, worker thread synchronises with main thread (that polls) 'somehow'. Core of the questions is 'how' - in terms of what is efficient. I can use pipe() - socket based IPC mechanism - but this seems to me as quite huge overhead. I tend to use some pthread IPC techniques like mutex, condition variables etc. … but these will not work with select() or poll().
Is there a common technique in POSIX (and surroundings) that address this conflict?
I guess on Windows there is WaitForMultipleObjects() function that allows that.
Example program is crafted to illustrate an issue, I know that I can design master/worker pattern in a different way but this is not what I'm asking for. I have other cases where I'm in the same situation.
You could use a signal to poke the worker thread, which will interrupt the select() call and return EINTR. This gets even easier to do with pselect().
For this to work:
decide on a signal (or allocate a real-time signal)
attach an empty handler function to it (if the signal were ignored, the system call would be automatically restarted)
block the signal, at least in the worker thread.
use the signal mask argument in pselect() to unblock the signal while waiting.
Between threads, you can use pthread_kill to deliver the signal to the worker thread specifically. When another process should send the signal, you can either make sure the signal is blocked in all but the worker thread (so it will be delivered there), or use the signal handler to find out whether the signal was sent to the worker thread, and use pthread_kill to forward it explicitly (the worker thread still doesn't need to do anything in the signal handler).
Due to laziness on my part, I don't have a source code viewer online, but you can clone the LibreVISA git tree, and take a look at src/messagepump.cpp, where this method is used to poke the worker thread after another thread added a file descriptor to the watch list.
Simon Richthers answer is v good.
Another alternative might be to make main thread only responsible for listening for new connections and starting up a worker thread with the connection information so that the worker is responsible for all subsequent ‘transactions’ from this source.
My understanding is:
Main thread uses select.
Worker threads processes requests forwarded to it by main thread.
So need to synchronize between workers and main thread e.g. when
worker finishes a transaction need to send response back to main
thread which in turn forwards the response back to the source.
Why don't you remove the problem of having to synchronize between the worker thread and the main thread by making the worker thread responsible for all transactions from a particular connection?
Thus the main thread is only responsible for listening for new connections and starting up a worker thread with the connection information i.e. the file descriptor for the new connection.
First of all, the way to wake another thread is to use the pthread_cond_wait / pthread_cond_timedwait calls in thread A to wait, and for thread B to use pthread_cond_broadcast / pthread_cond_signal to pick it up. So, for instance if B is a producer and A is the consumer, the producer might add items to a linked list protected with a mutex. There would be an associated conditional variable such that after the addition of the item, it could wake thread B such that it went to see if any new items had arrived on the list, and if so removed them. I say 'associated' as then the same mutex can be associated with the condition variable as protects the list.
So far so good. Now you mention asynchronous I/O. What I've wanted to do several times is select() or poll() on a set of FDs and a set of condition variables, so the select(), poll() is interrupted when the condition variable is broadcasted to. There is no easy way of doing this directly; you cannot simply mix and match.
You thus need to do one of two things. Either:
work around the problem (for instance, use a self-connected pipe() to send one byte to wake the select() up either instead of the condition variable, as well as the condition variable, or from some additional thread waiting on the condition variable; or
convert to a more threaded model. IE use one thread for sending, one thread for receiving, and use a producer / consumer model, so the sender thread simply removes from a list / buffer and sends (blocking if necessary), and the received waits for I/O (blocking if necessary) and adds it to the list (this is what you put in italics at the end).
The second is a major design change for those of us brought up on asynchronous I/O, and the first is ugly. You are not the first to be dismayed by this, but I've not found an easy way around it. Re the first an inefficiency, if you only write one character to wake the select loop to the self-pipe, I don't think you are going to see too much inefficiency.
I'm running a multi-threaded C program (process?) , making use of semaphores & pthreads. The threads keep interacting, blocking, waking & printing prompts on stdout continuously, without any human intervention. I want to be able to exit this process (gracefully after printing a message & putting down all threads, not via a crude CTRL+C SIGINT) by pressing a keyboard character like #.
What are my options for getting such an input from the user?
What more relevant information could I provide that will help to solve this problem?
Edit:
All your answers sound interesting, but my primary question remains. How do I get user input, when I don't know which thread is currently executing? Also, semaphore blocking using sem_wait() breaks if signalled via SIGINT, which may cause a deadlock.
There is no difference in reading standard input from threads except if more than one thread is trying to read it at the same time. Most likely your threads are not all calling functions to read standard input all the time, though.
If you regularly need to read input from the user you might want to have one thread that just reads this input and then sets flags or posts events to other threads based on this input.
If the kill character is the only thing you want or if this is just going to be used for debugging then what you probably want to do is occasionally poll for new data on standard input. You can do this either by setting up standard input as non-blocking and try to read from it occasionally. If reads return 0 characters read then no keys were pressed. This method has some problems, though. I've never used stdio.h functions on a FILE * after having set the underlying file descriptor (an int) to non-blocking, but suspect that they may act odd. You could avoid the use of the stdio functions and use read to avoid this. There is still an issue I read about once where the block/non-block flag could be changed by another process if you forked and exec-ed a new program that had access to a version of that file descriptor. I'm not sure if this is a problem on all systems. Nonblocking mode can be set or cleared with a 'fcntl' call.
But you could use one of the polling functions with a very small (0) timeout to see if there is data ready. The poll system call is probably the simplest, but there is also select. Various operating systems have other polling functions.
#include <poll.h>
...
/* return 0 if no data is available on stdin.
> 0 if there is data ready
< 0 if there is an error
*/
int poll_stdin(void) {
struct pollfd pfd = { .fd = 0, .events = POLLIN };
/* Since we only ask for POLLIN we assume that that was the only thing that
* the kernel would have put in pfd.revents */
return = poll(&pfd, 1, 0);
}
You can call this function within one of your threads until and as long as it retuns 0 you just keep on going. When it returns a positive number then you need to read a character from stdin to see what that was. Note that if you are using the stdio functions on stdin elsewhere there could actually be other characters already buffered up in front of the new character. poll tells you that the operating system has something new for you, not what C's stdio has.
If you are regularly reading from standard input in other threads then things just get messy. I'm assuming you aren't doing that (because if you are and it works correctly you probably wouldn't be asking this question).
You would have a thread listening for keyboard input, and then it would join() the other threads when receiving # as input.
Another way is to trap SIGINT and use it to handle the shutdown of your application.
The way I would do it is to keep a global int "should_die" or something, whose range is 0 or 1, and another global int "died," which keeps track of the number of threads terminated. should_die and died are both initially zero. You'll also need two semaphores to provide mutex around the globals.
At a certain point, a thread checks the should_die variable (after acquiring the mutex, of course). If it should die, it acquires the died_mutex, ups the died count, releases the died_mutex, and dies.
The main initial thread periodically wakes up, checks that the number of threads that have died is less than the number of threads, and goes back to sleep. The main thread dies when all the other threads have checked in.
If the main thread doesn't spawn all the threads itself, a small modification would be to have "threads_alive" instead of "died". threads_alive is incremented when a thread forks, and decremented when the thread dies.
In general, terminating a multithreaded operation cleanly is a pain in the butt, and besides special cases where you can use things like the semaphore barrier design pattern, this is the best I've heard of. I'd love to hear it if you find a better, cleaner one.
~anjruu
In general, I have threads waiting on a set of events and one of those events is the termination event.
In the main thread, when I have triggered the termination event, I then wait on all the threads having exited.
SIGINT is actually not that difficult to handle and is often used for graceful termination. You need a signal handler and a way to tell all the threads that it's time to stop. One global flag that threads check in their loops and the signal handler sets might do. Same approach works for "on user command" termination, though you need a way to get the input from the terminal - either poll in a dedicated thread, or again, set the terminal to generate a signal for you.
The tricky part is to unblock waiting threads. You have to carefully design the notification protocol of who tells who to stop and what they need to do - put dummy message into a queue, set a flag and signal a cv, etc.
I use pthread_create to create several child threads. At a time, the main thread wants to kill all child threads or there will be segment falut. Which function should I use to finish that? I searched the answer from google and got function like pthread_kill. But I did not know which signal should I send to the child thread to kill them. My running environment is RHEL 5.4 and programming language is C.
In general, you don't really want to violently kill a child thread, but instead you want to ask it to terminate. That way you can be sure that the child is quitting at a safe spot and all its resources are cleaned up.
I generally do this with a small piece of shared state between parent and child to allow the parent to communicate a "quit request" to each child. This can just be a boolean value for each child, protected by a mutex. The child checks this value periodically (every loop iteration, or whatever convenient checkpoints you have in your child thread). Upon seeing "quit_request" being true, the child thread cleans up and calls pthread_exit.
On the parent side, the "kill_child" routine looks something like this:
acquire shared mutex
set quit_request to true
pthread_join the child
The pthread_join may take some time, depending on how frequently the child checks its quit request. Make sure your design can handle whatever the delay may be.
It is possible to "cancel" a thread using pthread_cancel. However, this isn't typically best practice though under extreme circumstances like a SEGFAULT it may be conisdered a reasonable approach.
You should send SIG_TERM to each of your threads, using
int pthread_kill(pthread_t thread, int sig);
A quick way to get rid of all threads (besides the main) is to fork() and keep going with the child.
Not hyper clean...
if (fork()) exit(0); // deals also with -1...
You can use a global variable for the entire program.
int _fCloseThreads;
Set it to 1 when you want the threads to quit execution. Have the threads check that variable in their "loop" and nicely quit when it is set to 1. No need to protect it with a mutex.
You need to wait for the threads to quit. You can use join. Another way is to increment a counter when a thread enters its thread proc and then decriment the counter when it exits. The counter would need to be a global of sorts. Use gcc atomic ops on the counter. The main thread, after setting fCloseThreads, can wait on the counter to go to zero by looping, sleeping, and checking the count.
Finally, you might checkout pthread_cleanup_push and pop. They are a model for allowing a thread to cancel anywhere in its code (uses a longjump) and then call a final cleanup function before exiting threadproc. You basicly put cleanup_push at the top of your threadproc and cleanup_pop at the bottom, create an unwind function, and then at certain cancelation points a thread canceled by a call to pthread_cancel() will longjump back to threadproc and call the unwind function.