I'm implementing a system that runs game servers. I have a process (the "game controller") that creates two pipe pairs and forks a child. The child dups its STDIN to one pipe, and dups its STDOUT and STDERR to the other, then runs execlp() to run the game code.
The game controller will have two threads. The first will block in accept() on a named UNIX socket receiving input from another application, and the second thread is blocking read()ing the out and error pipe from the game server.
Occasionally, the first thread will receive a command to send a string to the stdin pipe of the game server. At this point, somehow I need to stop the second thread from read()ing so that the first thread can read the reply from the out and error pipe.
(It is worth noting that I will know how many characters/lines long the reply is, so I will know when to stop reading and let the second thread resume reading, resetting the process.)
How can I temporarily switch the read control to another thread, as above?
There are a couple of options. One would be to have the second thread handle all of the reading, and give the first thread a way to signal it to tell it to pass the input back. But this will be somewhat complicated; you will need to set up a method for signalling between the threads, and make sure that the first thread tells the second thread that it wants the input before the second thread reads it and handles it itself. There will be potential for various kinds of race conditions that could make your code unpredictable.
Another option is to avoid using threads at all. Just use select(2) (or poll(2)) to allow one thread to wait for activity on several file descriptors at once. select allows you to indicate the set of file descriptors that you are interested in. As soon as any activity happens on one of them (a connection is available to accept, data is available to read), select will return, indicating the set of file descriptors that are ready. You can then accept or read on the appropriate descriptors, and when you are done, loop and call select again to wait for the next I/O event.
Related
I'm experimenting with a fictional server/client application where the client side launches request threads by a (possibly very large) period of time, with small in-between delays. Each request thread writes on the 'public' fifo (known by all client and server threads) the contents of the request, and receives the server answer in a 'private' fifo that is created by the server with a name that is implicitly known (in my case, it's 'tmp/processId.threadId').
The public fifo is opened once in the main (request thread spawner) thread so that all request threads may write to it.
Since I don't care about the return value of my request threads and I can't make sure how many request threads I create (so that I store their ids and join them later), I opted to create the threads in a detached state, exit the main thread when the specified timeout expires and let the already spawned threads live on their own.
All of this is fine, however, I'm not closing the public fifo anywhere after all spawned request threads finish: after all, I did exit the main thread without waiting. Is this a small kind of disaster, in which case I absolutely need to count the active threads (perhaps with a condition variable) and close the fifo when it's 0? Should I just accept that the file is not explicitly getting closed, and let the OS do it?
All of this is fine, however, I'm not closing the public fifo anywhere
after all spawned request threads finish: after all, I did exit the
main thread without waiting. Is this a small kind of disaster, in
which case I absolutely need to count the active threads (perhaps with
a condition variable) and close the fifo when it's 0? Should I just
accept that the file is not explicitly getting closed, and let the OS
do it?
Supposing that you genuinely mean a FIFO, such as might be created via mkfifo(), no, it's not a particular issue that the process does not explicitly close it. If any open handles on it remain when the process terminates, they will be closed. Depending on the nature of the termination, it might be that pending data are not flushed, but that is of no consequence if the FIFO is used only for communication among the threads of one process.
But it possibly is an issue that the process does not remove the FIFO. A FIFO has filesystem persistence. Once you create one, it lives until it no longer has any links to the filesystem and is no longer open in any process (like any other file). Merely closing it does not cause it to be removed. Aside from leaving clutter on your filesystem, this might cause issues for concurrent or future runs of the program.
If indeed you are using your FIFOs only for communication among the threads of a single process, then you would probably be better served by pipes.
I managed to solve this issue setting up a cleanup rotine with atexit, which is called when the process terminates, ie. all threads finish their work.
I have two threads that are communicating to a serial port. One thread is continuously polling the serial port. The polling is done by first writing to the serial port write("do you have a message") and then reading the reply. This has to happen in consecutively.
I have another thread that is used to send commands over the serial port. So all this thread is doing is writing a message to the port, and then reading a confirmation that the message was sent.
So, while one thread is used to read and another used to write, they both do a read() and write().
My issue is, my current implementation uses and read mutex and a write mutex, which of course means you can get a write, write, read, read ordering, which is not the behavior I want.
I tried only using one lock and encompassing the write/reads, but this is causing one thread (polling thread) to never let the other thread grab the lock, or it randomly happens, even if I put sleeps after an unlock of one thread.
So, is there a way to pause one thread... say after it's done a read/write combo, and let another thread do it's thing? What's the best way of going about this issue? I do not want to use pthread_join because 1. It will pause my main code that is creating these threads and 2. The polling thread is never meant to exit unless there is an error.
Thanks
I've got a program where main process forks into 4 children that cooperate with each other:
process0 is opening FIFO file (O_WRONLY), reading 1 byte at a time from STDIN using read function, writing into FIFO using write and closing FIFO file
process1 is waiting for the shared memory to be empty (I'm using first byte of shared memory table if(tab[0] == 0) to check if its empty) opening FIFO file (O_RDONLY), reading from it, translating this one byte into hex and saving it into shared memory. It then closes fifo and sets tab[0] which is shared memory table to 1.
process2 is reading from shared memory if tab[0] == 1. After reading it writes data into pipe
process3 is reading from pipe and writing into STDIN
this all works perfect. The problem started when I wanted to add signals. I'm using semaphores to synchronize p0 and p1, signals to synchronize p1 and p2 and message queue to synchronize p2 and p3. It also works fine except for the times when for example process1 is in the sleeping mode. It goes into this mode when it wants to read from fifo and has to wait for data to be transferred.. I guess. I've been reading about it.
processes hierarchy
Here's what I've found that I think may be the cause:
"When the process makes a system call while in user mode (1), it moves into state 2 where it begins to run in kernel mode. Assume at this point that the system call made was to read a file on the hard disk. Because the read is not carried out immediately, the process goes to sleep, waiting on the event that the system has read the disk and the data is ready. It is now in state 4. When the data is ready, the process is awakened. This does not mean it runs immediately, but rather it is once again ready to run in main memory (3). "
I think I understand it but how can I avoid this? I want my program to react to signals always. Is there some way to change processes state from sleeping to running when I send the signal through "kill" ? Can I somehow tell the process to stop "waiting on the event that the system has read the disk and the data is ready" ?
here's my code if someone wants to look at it:
Program's code
If I understand your question correctly, process 1 is getting hung up at line 442 (source) and not responding to signals because it's in the read.
The obvious answer is just don't do a blocking read(). Check the descriptor to see if anything is there to read and move on if not. Read up on fcntl/ioctl and how to do non-blocking reads.
Your observations is likely that (most) system call defaults to restarting if a signal occurs. Shortly said, that means code stuck in a system call wakes up to user-space when a signal is delivered, but resumes the system call after the signal handler has been run.
If you use sigaction() to establish a signal handler instead of the signal() function., system calls will not restart when a signal is caught , but rather fail and set errno to EINTR.
That means you you have to handle the fact that system calls can "fail" anywhere due to a signal being delivered.
(signal() by default will cause system calls on linux restart, it can be controlled by some feature macros before including the signal.h header. Whether sigaction() causes the system call to restart is controlled by the flag SA_RESTART when establishing a signal handler)
I'm running a multi-threaded C program (process?) , making use of semaphores & pthreads. The threads keep interacting, blocking, waking & printing prompts on stdout continuously, without any human intervention. I want to be able to exit this process (gracefully after printing a message & putting down all threads, not via a crude CTRL+C SIGINT) by pressing a keyboard character like #.
What are my options for getting such an input from the user?
What more relevant information could I provide that will help to solve this problem?
Edit:
All your answers sound interesting, but my primary question remains. How do I get user input, when I don't know which thread is currently executing? Also, semaphore blocking using sem_wait() breaks if signalled via SIGINT, which may cause a deadlock.
There is no difference in reading standard input from threads except if more than one thread is trying to read it at the same time. Most likely your threads are not all calling functions to read standard input all the time, though.
If you regularly need to read input from the user you might want to have one thread that just reads this input and then sets flags or posts events to other threads based on this input.
If the kill character is the only thing you want or if this is just going to be used for debugging then what you probably want to do is occasionally poll for new data on standard input. You can do this either by setting up standard input as non-blocking and try to read from it occasionally. If reads return 0 characters read then no keys were pressed. This method has some problems, though. I've never used stdio.h functions on a FILE * after having set the underlying file descriptor (an int) to non-blocking, but suspect that they may act odd. You could avoid the use of the stdio functions and use read to avoid this. There is still an issue I read about once where the block/non-block flag could be changed by another process if you forked and exec-ed a new program that had access to a version of that file descriptor. I'm not sure if this is a problem on all systems. Nonblocking mode can be set or cleared with a 'fcntl' call.
But you could use one of the polling functions with a very small (0) timeout to see if there is data ready. The poll system call is probably the simplest, but there is also select. Various operating systems have other polling functions.
#include <poll.h>
...
/* return 0 if no data is available on stdin.
> 0 if there is data ready
< 0 if there is an error
*/
int poll_stdin(void) {
struct pollfd pfd = { .fd = 0, .events = POLLIN };
/* Since we only ask for POLLIN we assume that that was the only thing that
* the kernel would have put in pfd.revents */
return = poll(&pfd, 1, 0);
}
You can call this function within one of your threads until and as long as it retuns 0 you just keep on going. When it returns a positive number then you need to read a character from stdin to see what that was. Note that if you are using the stdio functions on stdin elsewhere there could actually be other characters already buffered up in front of the new character. poll tells you that the operating system has something new for you, not what C's stdio has.
If you are regularly reading from standard input in other threads then things just get messy. I'm assuming you aren't doing that (because if you are and it works correctly you probably wouldn't be asking this question).
You would have a thread listening for keyboard input, and then it would join() the other threads when receiving # as input.
Another way is to trap SIGINT and use it to handle the shutdown of your application.
The way I would do it is to keep a global int "should_die" or something, whose range is 0 or 1, and another global int "died," which keeps track of the number of threads terminated. should_die and died are both initially zero. You'll also need two semaphores to provide mutex around the globals.
At a certain point, a thread checks the should_die variable (after acquiring the mutex, of course). If it should die, it acquires the died_mutex, ups the died count, releases the died_mutex, and dies.
The main initial thread periodically wakes up, checks that the number of threads that have died is less than the number of threads, and goes back to sleep. The main thread dies when all the other threads have checked in.
If the main thread doesn't spawn all the threads itself, a small modification would be to have "threads_alive" instead of "died". threads_alive is incremented when a thread forks, and decremented when the thread dies.
In general, terminating a multithreaded operation cleanly is a pain in the butt, and besides special cases where you can use things like the semaphore barrier design pattern, this is the best I've heard of. I'd love to hear it if you find a better, cleaner one.
~anjruu
In general, I have threads waiting on a set of events and one of those events is the termination event.
In the main thread, when I have triggered the termination event, I then wait on all the threads having exited.
SIGINT is actually not that difficult to handle and is often used for graceful termination. You need a signal handler and a way to tell all the threads that it's time to stop. One global flag that threads check in their loops and the signal handler sets might do. Same approach works for "on user command" termination, though you need a way to get the input from the terminal - either poll in a dedicated thread, or again, set the terminal to generate a signal for you.
The tricky part is to unblock waiting threads. You have to carefully design the notification protocol of who tells who to stop and what they need to do - put dummy message into a queue, set a flag and signal a cv, etc.
I'm trying to understand how asynchronous file operations being emulated using threads. I've found next-to-nothing materials to read about the subject.
Is it possible that:
a process uses a thread to open a regular file (HDD).
the parent gets the file descriptor from the thread, now it may close the thread.
the parent uses the file descriptor with a new thread, reading X bytes from the file.
the parent gets the file descriptor with the seek-position of the current file state.
the parent may repeat these operations, without the need to open, or seek, every time it wishes to "continue" reading a new chunk of the file?
This is just a wild guess of mine, would appreciate if anybody mind to shed more light to clarify how it's being emulated efficiently.
UPDATE:
By efficient I actually mean that I don't want the thread to "wait" since the moment the file been opened. Think of a HTTP non-blocking daemon which serves a client with a huge file, you want to use the thread to read chunks of the file without blocking the daemon - but you don't want to keep the thread busy while "waiting" for the actual transfer to take place, you want to use the thread for other blocking operations of other clients.
To understand asynchronous I/O better, it may be helpful to think in terms of overlapping operation. That is, the number of pending operations (operations that have been started but not yet completed) can simutaneously go above one.
A diagram that explains asynchronous I/O might look like this: http://msdn.microsoft.com/en-us/library/aa365683(VS.85).aspx
If you are using the asynchronous I/O capabilities provided by the underlying Operating System, then it is possible to asynchronously read from multiple files without spawning a equal number of threads.
If your underlying Operating System does not provide asynchronous I/O, or if you decide not to use it, in other words, you wish to emulate asynchronous operation by only using blocking I/O (the regular Read/Write provided by the Operating System) then it is necessary to spawn as many threads as the number of simutaneous I/O operations. This is because when a thread is making a function call to blocking I/O, the thread cannot continue its execution until the operation finishes. In order to start another blocking I/O operation, that operation has to be issued from another thread that is not already occupied.
When you open/create a file fire up a thread. Now store that thread id/ptr as your file handle.
Basically the thread will do nothing except sit in a loop waiting for an "event". A semaphore would be good here. When you want to do a read then you add the read command to a queue (remember to critical section the stack add), return a unique id, and then you increment the semaphore. If the thread is asleep it will now wake up and grab the first message off the queue and process it. When it has completed you remove the command from the queue.
To poll if a file read has completed you can, simply, check to see if its in the command queue. If its not there then the command has completed.
Furthermore if you want to allow synchronous reads as well then you can wait after sending the message through for an "event" to get triggered by the completion. You then check to see if the unique id is the queue and if it isn't you return control. If it still is then you go back to a wait state until the relevant unique id has been processed.