I can't seem to find a useful definition for "blocking" (or for that matter "non-blocking") when used in relation to POSIX C functions.
For example read() may be called in blocking or non-blocking mode on a FIFO pipe. If called in blocking mode, it will block until it's opened elsewhere for writing.
Will this blocking just seize up the thread? Or the process? Or will it pause the rendering of the multiverse?
Blocking means that the thread is de-scheduled off the CPU while waiting for an event to happen. When a thread is de-scheduled it doesn't consume any CPU cycles and allows other threads to make progress or put the CPU in a lower power state if there are no other threads waiting to run.
One thread blocking doesn't affect other threads you may have in the process. A blocking call only blocks the calling thread.
For example, read blocks when there is no data in the pipe to read. When data arrives it "unblocks" and the read call returns.
In the kernel each file description and other objects one can block on (e.g. mutex or condition_variable) have a list of waiting threads. When a thread blocks on an object it is appended to that object's wait list and de-scheduled off the CPU. Whenever an event for the object occurs the kernel checks the wait list for waiting threads for such an event and if there are any one or multiple threads get scheduled again and the blocking calls eventually return.
In non-blocking mode such calls do not block but return immediately an error code with errno being set to EWOULDBLOCK or EAGAIN, which are nowadays two different names for the same errno value. (pthread calls do not set errno but return the error value directly).
Related
I have a pthread which does an indefinite polling on some file descriptors, with negative timeout parameter.
From the main thread, I would like to be able to indicate to the polling thread that it should exit.
I did some research and found the following approaches:
Use pthread_cancel(): This was discouraged as it might lead to unintended consequences of not clearing any held resources or mutexes
Use a variable as a flag and set it in the main thread. The polling thread checks this flag every iteration and if set, exits by calling pthread_exit(). This approach won't work because my polling thread doesn't loop, but simply blocks indefinitely, waiting on the file descriptors.
Can anyone suggest an elegant solution to this problem ?
Thanks!
ASM
You can create an anonymous pipe with pipe(2) and have the file-descriptor-watching thread add the read end to its polled file descriptors, exiting when that file descriptor reports EOF. The main thread then just closes the write end when it wants the thread to exit.
You can send the blocking thread a signal (e.g. SIGUSR1) with pthread_kill(). The select call should then return a negative value and set errno to EINTR. Don't forget to set a handler for the signal you send.
I have a C application using poll to wait for some data.
Currently I am implementing the rest of my application into this one and I use time based interrupts (SIGRTMIN). As expected poll() returns if one of my other timers call back.
How can I stop poll from doing that? I am reading a lot about ppoll(), but not sure how to use that... Can I use this to stop this function from returning when a timer event is fired?
(I do not have any problems with the poll being delayed a few ms)
If a thread / process blocking in poll() receives an unblocked signal then poll() will be interrupted. If you don't want that to happen then you can block the desired signal before calling poll(), and then unblock it after poll() returns (see sigprocmask()). Note, however, that that won't cause poll() to be delayed -- quite the opposite. If anything, it will cause receipt of the signal to be delayed. If poll() blocks long enough then it could cause multiple RT signals to queue up, so that after you unblock that signal you receive it multiple times in quick succession.
You should consider instead checking poll()'s return value (which you should always do anyway) and retrying if it is EINTR.
From the MSDN Documentation:
The transport providers allow an application to invoke send and receive operations from within the context of the socket I/O completion routine, and guarantee that, for a given socket, I/O completion routines will not be nested. This permits time-sensitive data transmissions to occur entirely within a preemptive context.
In our system we do have one thread calling WSARecvFrom() for multiple sockets. There is one CompletionRoutine for that thread handling all call backs from WSARecvFrom() opverlapped I/O.
Our tests showed that this Completion Routine is called like triggered from an Interrupt. Called for a socket while still processing the completeion Routine from an other socket.
How do we can prevent that this completion Routine gets not called while it is still processing Input from an other socket?
What Serialisation of data processing can we use ?
Note there are hundrets of sockets receiving and sending realtime data. Synchronisation with waiting for multiple objects is not applicable as there is a maximum of 64 defined by the Win32 API.
We can not use a Semaphore because when newly called the old ongoing processing is interreupted so a Semaphore would no be realeased and new processing blocks for ever.
Critical Sections or Mutex is not an Option because the Completion Routine Call back is made from within the same thread so CS or mutex would accept anyway and would not wait till the old processing is finished.
Does anyone have an Idea or even better approach to serialze (synchronize) data processing ?
If you read the WSARecvFrom() documentation again more carefully, it also says:
The completion routine follows the same rules as stipulated for Windows file I/O completion routines. The completion routine will not be invoked until the thread is in an alertable wait state such as can occur when the function WSAWaitForMultipleEvents with the fAlertable parameter set to TRUE is invoked.
The Alertable I/O documentation then states:
When the thread enters an alertable state, the following events occur:
The kernel checks the thread's APC queue. If the queue contains callback function pointers, the kernel removes the pointer from the queue and sends it to the thread.
The thread executes the callback function.
Steps 1 and 2 are repeated for each pointer remaining in the queue.
When the queue is empty, the thread returns from the function that placed it in an alertable state.
So it should be practically impossible for a given thread to overlap multiple pending completion routines on top of each other, because the thread receives and processes the routines in a serialized manner. The only way I could see that being different is if a completion routine is doing something to put the thread into a second alertable state while a previous alertable state is still in effect. I'm not sure what Windows does in that situation, but you should avoid doing it anyway.
Note there are hundrets of sockets receiving and sending realtime data. Synchronisation with waiting for multiple objects is not applicable as there is a maximum of 64 defined by the Win32 API
The WaitForMultipleObjects() documentation tells you how to work around that limitation:
To wait on more than MAXIMUM_WAIT_OBJECTS handles, use one of the following methods:
• Create a thread to wait on MAXIMUM_WAIT_OBJECTS handles, then wait on that thread plus the other handles. Use this technique to break the handles into groups of MAXIMUM_WAIT_OBJECTS.
• Call RegisterWaitForSingleObject to wait on each handle. A wait thread from the thread pool waits on MAXIMUM_WAIT_OBJECTS registered objects and assigns a worker thread after the object is signaled or the time-out interval expires.
I wouldn't wait on the sockets anyway, that is not very efficient. Using completion routines is fine as long as they are doing safe things.
Otherwise, I would suggest you stop using completion routines and switch to using an I/O Completion Port for the socket I/O instead. Then you are in more control of when the completion results are reported to you, because you have to call GetQueuedCompletionStatus() yourself to get the results of each I/O operation. You can have multiple sockets associated with a single IOCP, and then have a small pool of threads (typically one thread per CPU core works best) all calling GetQueuedCompletionStatus() on that IOCP. This way, you can process multiple I/O results in parallel, as they will be in different thread contexts and cannot overlap each other in the same thread. This does mean, however, that you can perform an I/O operation in one thread and the result may show up in a different thread. Just make sure your completion processing is thread-safe.
First of all let me thanks for all the helpful hints and comments to my question.
We did stop now using completion routines. We changed the application to use completion ports.
The biggest problem we had with completion routines is that every time the thread goes into an alertable state the completion routines can (and will) be called again from the OS. As seen in the Debugger also calling WSASendTo() from inside the completion routine puts the thread into an alertable state. So the completion routine is executed again before the previous execution of the completion routine comes to its end.
This makes it nearly impossible to synchronize data processing from multiple different sockets.
The approach using Completion Ports seems to be the perfect one. You then have control about what are doing when you are released from GetQueuedCompletionStatus() for processing a data buffer. You have to and you can do the synchronization of data processing by yourself in a linear fashion without being interrupted and newly executed while trying to process the data.
I've got a program where main process forks into 4 children that cooperate with each other:
process0 is opening FIFO file (O_WRONLY), reading 1 byte at a time from STDIN using read function, writing into FIFO using write and closing FIFO file
process1 is waiting for the shared memory to be empty (I'm using first byte of shared memory table if(tab[0] == 0) to check if its empty) opening FIFO file (O_RDONLY), reading from it, translating this one byte into hex and saving it into shared memory. It then closes fifo and sets tab[0] which is shared memory table to 1.
process2 is reading from shared memory if tab[0] == 1. After reading it writes data into pipe
process3 is reading from pipe and writing into STDIN
this all works perfect. The problem started when I wanted to add signals. I'm using semaphores to synchronize p0 and p1, signals to synchronize p1 and p2 and message queue to synchronize p2 and p3. It also works fine except for the times when for example process1 is in the sleeping mode. It goes into this mode when it wants to read from fifo and has to wait for data to be transferred.. I guess. I've been reading about it.
processes hierarchy
Here's what I've found that I think may be the cause:
"When the process makes a system call while in user mode (1), it moves into state 2 where it begins to run in kernel mode. Assume at this point that the system call made was to read a file on the hard disk. Because the read is not carried out immediately, the process goes to sleep, waiting on the event that the system has read the disk and the data is ready. It is now in state 4. When the data is ready, the process is awakened. This does not mean it runs immediately, but rather it is once again ready to run in main memory (3). "
I think I understand it but how can I avoid this? I want my program to react to signals always. Is there some way to change processes state from sleeping to running when I send the signal through "kill" ? Can I somehow tell the process to stop "waiting on the event that the system has read the disk and the data is ready" ?
here's my code if someone wants to look at it:
Program's code
If I understand your question correctly, process 1 is getting hung up at line 442 (source) and not responding to signals because it's in the read.
The obvious answer is just don't do a blocking read(). Check the descriptor to see if anything is there to read and move on if not. Read up on fcntl/ioctl and how to do non-blocking reads.
Your observations is likely that (most) system call defaults to restarting if a signal occurs. Shortly said, that means code stuck in a system call wakes up to user-space when a signal is delivered, but resumes the system call after the signal handler has been run.
If you use sigaction() to establish a signal handler instead of the signal() function., system calls will not restart when a signal is caught , but rather fail and set errno to EINTR.
That means you you have to handle the fact that system calls can "fail" anywhere due to a signal being delivered.
(signal() by default will cause system calls on linux restart, it can be controlled by some feature macros before including the signal.h header. Whether sigaction() causes the system call to restart is controlled by the flag SA_RESTART when establishing a signal handler)
In Linux, if two threads are created and both of them are running, when one of them calls recv() or any IO syscall that blocks when no data is available, what would happen to the whole process?
Will the other thread block also? I guess this depends on how threading is implemented. If thread library is in user space and kernel totally unaware of the threads within process, then process is the scheduling entity and thus both threads got blocked.
Further, if the other thread doesn't block because of this, can it then send() data via the same socket, which is blocking the recv thread? Duplexing?
Any ideas?
You're absolutely right that the blocking behavior will depend on if the thread is implemented in kernel space, or in user space. If threading is implemented purely in user space (that is, the kernel is completely uninvolved with the threading), then any blocking entry point into the kernel will need to be wrapped with some non-blocking variant that can simulate blocking semantics to its calling "thread" (e.g. using AIO to send / recv data instead of blocking, and the completion callback makes the thread runnable, again).
In Linux (and every other extant major OS I can think of), threading is implemented at the kernel level, or similar, and a blocking call into the kernel will not cause all other threads to block.
Yes, you can send() to a socket for which another thread is blocked on recv().
Blocking calls in one thread should not affect other threads.
If the blocked thread locks a mutex prior to entering the blocked call and the second thread attempts to lock the same mutex, then they the second thread would need to wait for the blocking call to finish and for the first thread to release the lock.
It's completely possible to implement threads in user-space such that one thread can proceed while another thread block on I/O.
The non-blocked thread should be able to send on the socket while the other thread is blocking on it (I've written such code).