I have wrote a server program that can handle multiple client by using fork(). I have a signal handler, error checking where needed, and everything works correctly. I have it set up where if a client enter 'quit', the server should stop accepting connections, let the opened clients finish their communication, and close once all clients are closed. To do this, whenever 'quit' is entered, I have an int flag that is set to 0. Since the variables in each child process are only for that process, and do not affect the other child processes or the parent process, I can't keep track of when the flag is set to 0. In my signal handler I am checking
if( count == 0 && flag == 0)
//close server and exit program
count is the number of clients opened which after they are all closed will obviously be zero ( this has been error checked to make sure it is correct). Does anyone know of a way I can create flag that can be set inside one client, and have that same value for every client? If that makes sense.. I am coding in C by the way.
You need to implement a single server for everyone to speak to, he will broadcast back to all the clients to update them with the client count, but in essence without a lot of complication to speak and keep track of all the other clients is a bit of a challenge.
This would also be a place to route and traffic messages between different clients.
Hopefully this shines some light.
The child process receiving the quit command needs to send a signal (such as SIGUSR1 -- I don't recommend SIGTERM for this purpose unless you are going to be closing out all the clients as soon as possible) to its parent, and let the parent set the flag in its memory space in that signal handler. (The parent can let everyone know its pid by just storing the results of getpid() somewhere.)
There are many IPC mechanisms that can do this... in particular semaphores come to mind. You could create a set of two semaphores, and use one for your child count and one for your quit flag. When you start your program, the client count would be initialized to zero, and the quit flag would be initialized to one. When each child is started, it should add one to the client count... and subtract one when it exits. When a client receives the quit command, it would zero the quit flag. The main program will know when everything is done when both semaphores reach zero.
Or (if possible) you could just start threads instead of forking processes, and use two global ints for count and flag and a global mutex to protect them. The pthreads API is fairly easy to use for simple things where not a lot of inter-thread communication needs to occur, and creating a thread is faster than fork+exec.
Related
I'm experimenting with a fictional server/client application where the client side launches request threads by a (possibly very large) period of time, with small in-between delays. Each request thread writes on the 'public' fifo (known by all client and server threads) the contents of the request, and receives the server answer in a 'private' fifo that is created by the server with a name that is implicitly known (in my case, it's 'tmp/processId.threadId').
The public fifo is opened once in the main (request thread spawner) thread so that all request threads may write to it.
Since I don't care about the return value of my request threads and I can't make sure how many request threads I create (so that I store their ids and join them later), I opted to create the threads in a detached state, exit the main thread when the specified timeout expires and let the already spawned threads live on their own.
All of this is fine, however, I'm not closing the public fifo anywhere after all spawned request threads finish: after all, I did exit the main thread without waiting. Is this a small kind of disaster, in which case I absolutely need to count the active threads (perhaps with a condition variable) and close the fifo when it's 0? Should I just accept that the file is not explicitly getting closed, and let the OS do it?
All of this is fine, however, I'm not closing the public fifo anywhere
after all spawned request threads finish: after all, I did exit the
main thread without waiting. Is this a small kind of disaster, in
which case I absolutely need to count the active threads (perhaps with
a condition variable) and close the fifo when it's 0? Should I just
accept that the file is not explicitly getting closed, and let the OS
do it?
Supposing that you genuinely mean a FIFO, such as might be created via mkfifo(), no, it's not a particular issue that the process does not explicitly close it. If any open handles on it remain when the process terminates, they will be closed. Depending on the nature of the termination, it might be that pending data are not flushed, but that is of no consequence if the FIFO is used only for communication among the threads of one process.
But it possibly is an issue that the process does not remove the FIFO. A FIFO has filesystem persistence. Once you create one, it lives until it no longer has any links to the filesystem and is no longer open in any process (like any other file). Merely closing it does not cause it to be removed. Aside from leaving clutter on your filesystem, this might cause issues for concurrent or future runs of the program.
If indeed you are using your FIFOs only for communication among the threads of a single process, then you would probably be better served by pipes.
I managed to solve this issue setting up a cleanup rotine with atexit, which is called when the process terminates, ie. all threads finish their work.
Is there a nice way out of an infinite loop? (I tried catching a SIGINT, but it won't let me clean up before exiting)
I'm doing a client-server.
Each time a client connects, the server forks and designates a child to the client (this child will redirect stdin and stdout to a fd of a fifo or socket and then execl)
In the server I am currently listening for clients in an infinite loop
for(;;)
{
listen_client();
create_child();
}
(more or less)
The application works fine (each client can communicate perfectly with it's designated child-server).
The problem comes on server-closure.
I was wondering if there was a nice way for the server to close (I'm not used on using endless loops, not sure if it's the right way and not sure if there is a nice way out).
I tried a sig_handler (sigaction), but I can't seem to make it work (ideally, I would close all fd, kill all childs and delete all fifos before exiting).
EDIT
with a global variable that keeps track of childs and a modification in sig_handler() it seems to be working, still not sure it's a nice way
(code in sig handler)
while(childs > 0)
{
pid = wait(NULL);
//cleanup mess of child
childs--
}
my doubts are:
are children always dying automatically? or should I send a kill or something (in which case I'd guess I would need to keep track of every pid).
is there a risk of creating zombies?
is there a risk of entering an endless loop? (I never decrease "childs" other than here, I'm not sure if wait also counts the children dead long ago).
I know this "works", but I'd rather learn to do it the "right" way (if there is such thing)
You could disable async signal generation, create a signalfd(2) and poll(2) on that. That way you can handle all signals correctly and keep a linear program flow including correct cleanup. IMO this is a very convenient solution.
I am programming a http server. There is the main daemon spawning a bunch of listeners, which are threads or processes, depending on user settings. Upon creation of a listener, the socket descriptor is passed to it, and its job is just to listen for connections (duh). A semaphore is wrapping the call to listen as to avoid the thundering herd effect.
My problem is how to quit the server. In this situation, where the listeners are blocked on a semaphore, how does the daemon is going to tell them to close? The daemon can't just kill them, maybe someone is responding to a request...
I want to keep the design as simple as possible, but I can't find a solution to this problem.
Here are some ugly workaround:
Set a timeout for the semaphore. Wake up. Should I close? No? Ok, back to sleep;
Just kill them;
Array of booleans in shared memory, meaning responding/blocked, the daemon kills accordingly. The best so far, but not so simple.
What do you say?
Thanks.
A clean way to solve this problem is to make each listener wait on two semaphores. The first one it the current one you now use, and a second one, that when become signaled, means it's time to quit. I believe your system is linux since you used the term daemon. The function select does just that - waits on multiple objects (file-descriptors like), and returns when one of them becomes signaled. You also know from the function which one got signaled, so here is your solution.
On Windows the function is WaitForMultipleObjects()
Send a SIGTERM or, if you prefer, SIGUSR to children and implement handling of this signal so that they finish current request and exit gracefully.
If they wait on semaphore, you should use interruptible mode so that receiving a signal will wake them up.
In the past I've used a global that client handling threads could use to find out if they need to 'clean up shop' and then waited on them to all finish but I'd also be interested to know if there's an even better way. (Not sure what language but in most, you can check to see if your thread is still running.)
I am sorry for the basicness of this question, but I am having an issue here. I have a client-server program. I don't know before hand how many connections will come but they are not infinite. And at the end , after all connections are closed some results are output. But the problem I am having is, accepting connections is in an infinite while loop, how is it stoppedd to output the result.
Thanks
you need to have some form of condition to break out of you loop, in your case, a timeout would probably work the best, basically meaning, if you don't get any new clients for x seconds, you stop looking for clients, same goes for any for of connection error.
Anything more requires looking at the code you are using.
Handling EINTR on error from accept(2) with terminating the program and hitting ^C usually works.
You could install a handler for the SIGTERM signal which would set a global volatile sig_atomic_t variable, and test that variable in your multiplexing loop (probably around poll or select). Remember that signal handlers cannot call many functions (only the async-signal-safe ones).
Catching nicely SIGTERM is expected from most Linux or Posix servers.
You could consider using an event handling library like libev, libevent etc.
Although my background is with Windows NT the function "names" are ones that name generic threading or process functions that should be available in any multi-threading environment.
If the main thread can determine when the child thread in question should terminate it can either do this by having the child thread loop on a boolean - such as "terminate_conditon" - or by terminating the thread throught its handle.
// child thread
terminate_condition=FALSE;
while (!terminate_condition)
{
// accept connections
}
child_thread_done=TRUE;
// output results
exit_thread ();
// main thread
child_thread_done=FALSE;
child_thread=create_thread (...);
// monitor connections to determine when done
terminate_condition=TRUE;
while (!child_thread_done)
{
sleep (1);
}
// or maybe output results here?
exit_process ();
This controlled termination solution requires that only one thread writes to the child_thread_done boolean and that any other thread only reads.
Or
// child thread
while (1)
{
// accept connections
}
// main thread
child_thread=create_thread (...);
// monitor connections to determine when done
kill_thread (child_thread);
// output results
exit_process ();
The second form is messier since it simply kills the child thread. In general it is better to have the child thread perform a controlled termination, especially if it has allocated resources (which become the responsibility of the process as a whole rather than just the allocating thread).
If there are many child threads working with connections a synchronized termination mechanism is necessary: either a struct with as many members as there are child threads (a terminating thread sets its "terminated" boolean to true, terminates and the main thread monitors the struct to make sure all child "terminated" booleans are true before proceeding) or a counter containing the number of child threads operating (when a child is about to terminate it takes exclusive control of the counter via a spinlock, decrements it and frees the lock before terminating: the main thread doesn't do anything before the counter contains zero).
I'm running a multi-threaded C program (process?) , making use of semaphores & pthreads. The threads keep interacting, blocking, waking & printing prompts on stdout continuously, without any human intervention. I want to be able to exit this process (gracefully after printing a message & putting down all threads, not via a crude CTRL+C SIGINT) by pressing a keyboard character like #.
What are my options for getting such an input from the user?
What more relevant information could I provide that will help to solve this problem?
Edit:
All your answers sound interesting, but my primary question remains. How do I get user input, when I don't know which thread is currently executing? Also, semaphore blocking using sem_wait() breaks if signalled via SIGINT, which may cause a deadlock.
There is no difference in reading standard input from threads except if more than one thread is trying to read it at the same time. Most likely your threads are not all calling functions to read standard input all the time, though.
If you regularly need to read input from the user you might want to have one thread that just reads this input and then sets flags or posts events to other threads based on this input.
If the kill character is the only thing you want or if this is just going to be used for debugging then what you probably want to do is occasionally poll for new data on standard input. You can do this either by setting up standard input as non-blocking and try to read from it occasionally. If reads return 0 characters read then no keys were pressed. This method has some problems, though. I've never used stdio.h functions on a FILE * after having set the underlying file descriptor (an int) to non-blocking, but suspect that they may act odd. You could avoid the use of the stdio functions and use read to avoid this. There is still an issue I read about once where the block/non-block flag could be changed by another process if you forked and exec-ed a new program that had access to a version of that file descriptor. I'm not sure if this is a problem on all systems. Nonblocking mode can be set or cleared with a 'fcntl' call.
But you could use one of the polling functions with a very small (0) timeout to see if there is data ready. The poll system call is probably the simplest, but there is also select. Various operating systems have other polling functions.
#include <poll.h>
...
/* return 0 if no data is available on stdin.
> 0 if there is data ready
< 0 if there is an error
*/
int poll_stdin(void) {
struct pollfd pfd = { .fd = 0, .events = POLLIN };
/* Since we only ask for POLLIN we assume that that was the only thing that
* the kernel would have put in pfd.revents */
return = poll(&pfd, 1, 0);
}
You can call this function within one of your threads until and as long as it retuns 0 you just keep on going. When it returns a positive number then you need to read a character from stdin to see what that was. Note that if you are using the stdio functions on stdin elsewhere there could actually be other characters already buffered up in front of the new character. poll tells you that the operating system has something new for you, not what C's stdio has.
If you are regularly reading from standard input in other threads then things just get messy. I'm assuming you aren't doing that (because if you are and it works correctly you probably wouldn't be asking this question).
You would have a thread listening for keyboard input, and then it would join() the other threads when receiving # as input.
Another way is to trap SIGINT and use it to handle the shutdown of your application.
The way I would do it is to keep a global int "should_die" or something, whose range is 0 or 1, and another global int "died," which keeps track of the number of threads terminated. should_die and died are both initially zero. You'll also need two semaphores to provide mutex around the globals.
At a certain point, a thread checks the should_die variable (after acquiring the mutex, of course). If it should die, it acquires the died_mutex, ups the died count, releases the died_mutex, and dies.
The main initial thread periodically wakes up, checks that the number of threads that have died is less than the number of threads, and goes back to sleep. The main thread dies when all the other threads have checked in.
If the main thread doesn't spawn all the threads itself, a small modification would be to have "threads_alive" instead of "died". threads_alive is incremented when a thread forks, and decremented when the thread dies.
In general, terminating a multithreaded operation cleanly is a pain in the butt, and besides special cases where you can use things like the semaphore barrier design pattern, this is the best I've heard of. I'd love to hear it if you find a better, cleaner one.
~anjruu
In general, I have threads waiting on a set of events and one of those events is the termination event.
In the main thread, when I have triggered the termination event, I then wait on all the threads having exited.
SIGINT is actually not that difficult to handle and is often used for graceful termination. You need a signal handler and a way to tell all the threads that it's time to stop. One global flag that threads check in their loops and the signal handler sets might do. Same approach works for "on user command" termination, though you need a way to get the input from the terminal - either poll in a dedicated thread, or again, set the terminal to generate a signal for you.
The tricky part is to unblock waiting threads. You have to carefully design the notification protocol of who tells who to stop and what they need to do - put dummy message into a queue, set a flag and signal a cv, etc.