I'm programming with socket which is in blocking mode, I have a question about send method.
In the man page of send method, it says:
[EINTR] A signal interrupts the system call before any data is transmitted.
It means if a signal interrupts the system call before any data is transmitted, the send would return -1 and errno would be set to EINTR.
My question is that if a part of data has been transmitted when a signal interrupts the system call, what will return. It seems that it shouldn't return -1 because it has send some data. I think it will return the number of data has been transmitted which means send method in the blocking mode may return less number of data than you passed as third parameter.
ssize_t send(int socket, const void *buffer, size_t length, int flags);
The other answers are pretty clear, but after reading some of your comments, I would like to add some further information.
First of all, you got the idea behind EINTR wrong. Getting interrupted by a signal in a syscall is not to be perceived as an error. The rationale behind EINTR in slow syscalls (slow syscalls are those that can block forever, like open(2) on some file types - terminal devices for example - accept(2), read(2) and write(2) on some devices - sockets included - etc.) is that if your program was blocked in a syscall and a signal was caught (while still blocked), then it is very likely (but not mandatory) that the signal handler changed state in your program and things are different, so the call returns prematurely with EINTR to give you a chance of doing anything that you might want to do. It is not an error like EINVAL or EBADF or other "real" errors - it's just the way the kernel tells you that a signal was caught.
If you don't want to do anything, then either set the SA_RESTART flag on the sa_flags field of struct sigaction when setting up the signal handler (which causes the syscall to be automatically restarted), or explicitly call send(2) again when it returns -1 with errno set to EINTR.
The bottom line is, there isn't an inherent limitation on the kernel that forces it to return to userspace when signals are caught. Instead, EINTR is just a convenient behavior that developers might find useful.
If the kernel was in the midst of transmitting data and a signal was raised, it's no big deal: if data is being transmitted, the syscall is doing progress. The kernel is in process context executing a syscall in behalf of the program that called it, so technically the process is not sleeping anymore. If a signal arrives, it will be pending until the kernel decides that it's time to deliver it to the process - most likely, this happens once send(2) returns.
The documentation is clear.
RETURN VALUE
On success, these calls return the number of bytes sent. On error, -1 is returned, and errno is set appropriately.
[...]
EINTR A signal occurred before any data was transmitted;
send() either returns
the number of bytes sent
or -1
If -1 is returned the reason is indicated via the value of errno.
If errno equals EINTR a signal interupted send() while no data had been received so far.
From this info above one can safely conclude that if data had been received the send() function would not return -1, not matter whether a signal was received or not.
[EINTR] A signal interrupts the system call before any data is transmitted.
This means that if send() start transmitting data, it won't be interrupted by any signals. So, transmission will block the receiving of signal until it finishes. The situation that send() may return less bytes of data than you passed as third parameter is usually due to the network problems, such as packets lost.
Related
I have a small server program that accepts connections on a TCP or local UNIX socket, reads a simple command and (depending on the command) sends a reply.
The problem is that the client may have no interest in the answer and sometimes exits early. So writing to that socket will cause a SIGPIPE and make my server crash.
What's the best practice to prevent the crash here? Is there a way to check if the other side of the line is still reading? (select() doesn't seem to work here as it always says the socket is writable). Or should I just catch the SIGPIPE with a handler and ignore it?
You generally want to ignore the SIGPIPE and handle the error directly in your code. This is because signal handlers in C have many restrictions on what they can do.
The most portable way to do this is to set the SIGPIPE handler to SIG_IGN. This will prevent any socket or pipe write from causing a SIGPIPE signal.
To ignore the SIGPIPE signal, use the following code:
signal(SIGPIPE, SIG_IGN);
If you're using the send() call, another option is to use the MSG_NOSIGNAL option, which will turn the SIGPIPE behavior off on a per call basis. Note that not all operating systems support the MSG_NOSIGNAL flag.
Lastly, you may also want to consider the SO_SIGNOPIPE socket flag that can be set with setsockopt() on some operating systems. This will prevent SIGPIPE from being caused by writes just to the sockets it is set on.
Another method is to change the socket so it never generates SIGPIPE on write(). This is more convenient in libraries, where you might not want a global signal handler for SIGPIPE.
On most BSD-based (MacOS, FreeBSD...) systems, (assuming you are using C/C++), you can do this with:
int set = 1;
setsockopt(sd, SOL_SOCKET, SO_NOSIGPIPE, (void *)&set, sizeof(int));
With this in effect, instead of the SIGPIPE signal being generated, EPIPE will be returned.
I'm super late to the party, but SO_NOSIGPIPE isn't portable, and might not work on your system (it seems to be a BSD thing).
A nice alternative if you're on, say, a Linux system without SO_NOSIGPIPE would be to set the MSG_NOSIGNAL flag on your send(2) call.
Example replacing write(...) by send(...,MSG_NOSIGNAL) (see nobar's comment)
char buf[888];
//write( sockfd, buf, sizeof(buf) );
send( sockfd, buf, sizeof(buf), MSG_NOSIGNAL );
In this post I described possible solution for Solaris case when neither SO_NOSIGPIPE nor MSG_NOSIGNAL is available.
Instead, we have to temporarily suppress SIGPIPE in the current thread that executes library code. Here's how to do this: to suppress SIGPIPE we first check if it is pending. If it does, this means that it is blocked in this thread, and we have to do nothing. If the library generates additional SIGPIPE, it will be merged with the pending one, and that's a no-op. If SIGPIPE is not pending then we block it in this thread, and also check whether it was already blocked. Then we are free to execute our writes. When we are to restore SIGPIPE to its original state, we do the following: if SIGPIPE was pending originally, we do nothing. Otherwise we check if it is pending now. If it does (which means that out actions have generated one or more SIGPIPEs), then we wait for it in this thread, thus clearing its pending status (to do this we use sigtimedwait() with zero timeout; this is to avoid blocking in a scenario where malicious user sent SIGPIPE manually to a whole process: in this case we will see it pending, but other thread may handle it before we had a change to wait for it). After clearing pending status we unblock SIGPIPE in this thread, but only if it wasn't blocked originally.
Example code at https://github.com/kroki/XProbes/blob/1447f3d93b6dbf273919af15e59f35cca58fcc23/src/libxprobes.c#L156
Handle SIGPIPE Locally
It's usually best to handle the error locally rather than in a global signal event handler since locally you will have more context as to what's going on and what recourse to take.
I have a communication layer in one of my apps that allows my app to communicate with an external accessory. When a write error occurs I throw and exception in the communication layer and let it bubble up to a try catch block to handle it there.
Code:
The code to ignore a SIGPIPE signal so that you can handle it locally is:
// We expect write failures to occur but we want to handle them where
// the error occurs rather than in a SIGPIPE handler.
signal(SIGPIPE, SIG_IGN);
This code will prevent the SIGPIPE signal from being raised, but you will get a read / write error when trying to use the socket, so you will need to check for that.
You cannot prevent the process on the far end of a pipe from exiting, and if it exits before you've finished writing, you will get a SIGPIPE signal. If you SIG_IGN the signal, then your write will return with an error - and you need to note and react to that error. Just catching and ignoring the signal in a handler is not a good idea -- you must note that the pipe is now defunct and modify the program's behaviour so it does not write to the pipe again (because the signal will be generated again, and ignored again, and you'll try again, and the whole process could go on for a long time and waste a lot of CPU power).
Or should I just catch the SIGPIPE with a handler and ignore it?
I believe that is right on. You want to know when the other end has closed their descriptor and that's what SIGPIPE tells you.
Sam
What's the best practice to prevent the crash here?
Either disable sigpipes as per everybody, or catch and ignore the error.
Is there a way to check if the other side of the line is still reading?
Yes, use select().
select() doesn't seem to work here as it always says the socket is writable.
You need to select on the read bits. You can probably ignore the write bits.
When the far end closes its file handle, select will tell you that there is data ready to read. When you go and read that, you will get back 0 bytes, which is how the OS tells you that the file handle has been closed.
The only time you can't ignore the write bits is if you are sending large volumes, and there is a risk of the other end getting backlogged, which can cause your buffers to fill. If that happens, then trying to write to the file handle can cause your program/thread to block or fail. Testing select before writing will protect you from that, but it doesn't guarantee that the other end is healthy or that your data is going to arrive.
Note that you can get a sigpipe from close(), as well as when you write.
Close flushes any buffered data. If the other end has already been closed, then close will fail, and you will receive a sigpipe.
If you are using buffered TCPIP, then a successful write just means your data has been queued to send, it doesn't mean it has been sent. Until you successfully call close, you don't know that your data has been sent.
Sigpipe tells you something has gone wrong, it doesn't tell you what, or what you should do about it.
Under a modern POSIX system (i.e. Linux), you can use the sigprocmask() function.
#include <signal.h>
void block_signal(int signal_to_block /* i.e. SIGPIPE */ )
{
sigset_t set;
sigset_t old_state;
// get the current state
//
sigprocmask(SIG_BLOCK, NULL, &old_state);
// add signal_to_block to that existing state
//
set = old_state;
sigaddset(&set, signal_to_block);
// block that signal also
//
sigprocmask(SIG_BLOCK, &set, NULL);
// ... deal with old_state if required ...
}
If you want to restore the previous state later, make sure to save the old_state somewhere safe. If you call that function multiple times, you need to either use a stack or only save the first or last old_state... or maybe have a function which removes a specific blocked signal.
For more info read the man page.
Linux manual said:
EPIPE The local end has been shut down on a connection oriented
socket. In this case the process will also receive a SIGPIPE
unless MSG_NOSIGNAL is set.
But for Ubuntu 12.04 it isn't right. I wrote a test for that case and I always receive EPIPE withot SIGPIPE. SIGPIPE is genereated if I try to write to the same broken socket second time. So you don't need to ignore SIGPIPE if this signal happens it means logic error in your program.
is it necessary to check for errno == EINTR if you read massive amounts of data? I use the pread() function to read. In all my time I have never seen EINTR returned, but I have seen some code online where it is explicitely checks for it.
so really is it necessary to check for EINTR and maybe repeat the call?
EINTR is returned when as system call is interrupted as a result of your process receiving a signal. If your process was blocked in the kernel, waiting for the read to complete, and a signal is caught, this may wake the kernel; this depends on if the operation is interruptable. The sleeping I/O routine is woken and is expected to return EINTR to user-space.
Just before the kernel returns to user space, it checks for pending signals. If a signal is pending, it will take the action associated with that signal. Possible actions include: dispatching the signal to a signal handler, killing your process, or ignoring the signal. Assuming this does not kill your process and/or your signal handler returns normally, the system call will return EINTR.
If you were not expecting this, you typically want to try the action again, but this can also be used as a way to gracefully abort an I/O operation. For example, alarm(2) can be used to implement a timeout, where SIGALRM is delivered if the I/O does not complete in a timely manner. In your signal handler, you could set a flag indicating a timeout and when your read operation returns EINTR, you can check for your timeout flag.
The reason is - on a busy system, for example, it is possible to have an interrupt on the read.
So, on your desktop you may never see it. On an overloaded server, you can.
Se Chapter 5 of Advanced Programming in the UNIX Environment - Stevens and Rago. There is a complete explanation.
I have a thread that continuously reads a serial port for data.
If the main program receives a SIGINT it calls g_thread_join() on the serial port thread.
However, since the read is blocking the serial port thread won't return and program stalls untill i get a byte on the serial line and then it can exit.
Is there a way to pass the SIGINT on to read() so that it can be forced to return when the main thread demands it?
To have read() return EINTR, unset SA_RESTART in the member sa_flags of the struct sigaction passed into the call to sigaction() when installing the signal handler for SIGINT.
An alternative approach woud be to avoid a blocking read() at all. Please see the answers to this question: how to avoid blocking from the read function?
The accepted answer for this previous question on implementing timeout for read() when reading from a serial port shows how to use select(2) to perform reading with timeout. The SIGINT signal you send will thus be eventually acted upon.
I've seen a number of questions regarding send() that discuss the underlying protocol. I'm fully aware that for TCP any message may be broken up into parts as it's sent and there's no guarantee that the receiver will get the message in one atomic operation. In this question I'm talking solely about the behavior of the send() system call as it interacts with the networking layer of the local system.
According to the POSIX standard, and the send() documentation I've read, the length of the message to be sent is specified by the length argument. Note that: send() sends one message, of length length. Further:
If space is not available at the sending socket to hold the message to
be transmitted, and the socket file descriptor does not have
O_NONBLOCK set, send() shall block until space is available. If space
is not available at the sending socket to hold the message to be
transmitted, and the socket file descriptor does have O_NONBLOCK set,
send() shall fail.
I don't see any possibility in this definition for send() to ever return any value other than -1 (which means no data is queued in the kernel to be transmitted) or length, which means the entire message is queued in the kernel to be transmitted. I.e., it seems to me that send() must be atomic with respect to locally queuing the message for delivery in the kernel.
If there is enough room in the socket queue in the kernel for the entire message and no signal occurs (normal case), it's copied and returns length.
If a signal occurs during send(), then it must return -1. Obviously we cannot have queued part of the message in this case, since we don't know how much was sent. So nothing can be sent in this situation.
If there is not enough room in the socket queue in the kernel for the entire message and the socket is blocking, then according to the above statement send() must block until space becomes available. Then the message will be queued and send() returns length.
If there is not enough room in the socket queue in the kernel for the entire message and the socket is non-blocking, then send() must fail (return -1) and errno will be set to EAGAIN or EWOULDBLOCK. Again, since we return -1 it's clear that in this situation no part of the message can be queued.
Am I missing something? Is it possible for send() to return a value which is >=0 && <length? In what situation? What about non-POSIX/UNIX systems? Is the Windows send() implementation conforming with this?
Your point 2 is over-simplified. The normal condition under which send returns a value greater than zero but less than length (note that, as others have said, it can never return zero except possibly when the length argument is zero) is when the message is sufficiently long to cause blocking, and an interrupting signal arrives after some content has already been sent. In this case, send cannot fail with EINTR (because this would prevent the application from knowing it had already successfully sent some data) and it cannot re-block (since the signal is interrupting, and the whole point of that is to get out of blocking), so it has to return the number of bytes already sent, which is less than the total length requested.
According to the Posix specification and all the man 2 send pages I have ever seen in 30 years, yes, send() can return any value > 0 and <= length. Note that it cannot return zero.
According to a discussion a few years ago on news:comp.protocols.tcp-ip where all the TCP implementors are, a blocking send() won't actually return until it has transferred all the data to the socket send buffer: in other words, the return value is either -1 or length. It was agreed that this was true of all known implementations, and also true of write(), writev(), sendmsg(), writev(),
I know how the thing works on Linux, with the GNU C Library. Point 4 of your question reads differently in this case. If you set the flag O_NONBLOCK for the file descriptor, and if it is not possible to queue the entire message in the kernel atomically, send() returns the number of bytes actually sent (it can be between 1 and length), and errno is set to EWOULDBLOCK.
(With a file descriptor working in the blocking mode, send() would block.)
It is possible for send() to return a value >= 0 && < length. This could happen if the send buffer has less room than the length of the message upon a call to send(). Similarly, if the current receiver window size known to the sender is smaller than the length of the message, only part of the message may be sent. Anecdotally, I've seen this happen on Linux through the a localhost connection when the receiving process was slow to unload the data it was receiving from its receive buffer.
My sense is that one's actual experience will vary a good bit by implementation. From this Microsoft link, it's clear that a non-error return value less than the length can occur.
It is also possible to get a return value of zero (again, at least with some implementations) if a zero-length message is sent.
This answer is based on my experience, as well as drawing upon this SO answer particularly.
Edit: From this answer and its comments, evidently an EINTR failure may only result if the interruption comes before any data is sent, which would be another possible way to get such a return value.
On a 64-bit Linux system:
sendto(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4294967296, 0, NULL, 0) = 2147479552
So, even trying to send lowy 4GB, Linux chickens out and sends less than 2GB. So, if you think that you'll ask it to send 1TB and it patiently will sit there, keep wishing.
Similarly, on an embedded system with just a few KBs free, don't think that it'll fail or will wait for something - it'll send as much as it can, and tell you how much that was, letting you to retry with the rest (or do something else in the meantime).
Everyone agrees that in case of EINTR, there can be a short send. But EINTR can happen at any time, so there can always be a short send.
And finally, POSIX says that the number of bytes sent is returned, period. And whole Unix and POSIX which formalizes it is built on the concept of short read/writes, which allows implementations of POSIX systems to scale from the tiniest embedded to supercomputers with proverbial "bigdata". So, no need to try to read between the lines and find indulgences to a particular adhoc implementation you have on your hands. There're many more implementations out there, and as long as you follow the word of the standard, your app will be portable among them.
To clarify a little, where it says:
shall block until space is available.
there are several ways to wake up from that block/sleep:
Enough space becomes available.
A signal interrupts the current blocking operation.
SO_SNDTIMEO is set for the socket and the timeout expires.
Other, e.g. the socket is closed in another thread.
So things end up thus:
If there is enough room in the socket queue in the kernel for the entire message and no signal occurs (normal case), it's copied and returns length.
If a signal occurs during send(), then it must return -1. Obviously we cannot have queued part of the message in this case, since we don't know how much was sent. So nothing can be sent in this situation.
If there is not enough room in the socket queue in the kernel for the entire message and the socket is blocking, then according to the above statement send() must block until space becomes available. Then the message will be queued and send() returns length. Then send() can be interrupted by a signal, the send timeout can elapse,... causing a short send/partial write. Reasonable implementations will return -1 and set errno to an adequate value if nothing was copied to the send buffer.
If there is not enough room in the socket queue in the kernel for the entire message and the socket is non-blocking, then send() must fail (return -1) and errno will be set to EAGAIN or EWOULDBLOCK. Again, since we return -1 it's clear that in this situation no part of the message can be queued.
I have found a server by select(), which I want to receive from some clients.
But I find that the server will get blocked in read() by gdb.
So I thought of solving it by adding a SIGALRM, but
when a timeout occurs, it's still blocked in read().
This happens because, system calls are automatically restarted, the read()
is not interrupted when the SIGALRM signal handler returns.
Is this interpretation correct?
The usual solution to this problem is to use SOCK_NONBLOCK to socket(2) or O_NONBLOCK to fcntl(2)'s F_SETFL command. Once the socket is marked non-blocking, it'll never block when you try to read from it, and you won't need to try to straddle the divide between blocking or non-blocking. Are you sure select(2) set the filedescriptor? The select(2) manpage does describe one reason why you see what you're seeing, but it doesn't seem likely:
Under Linux, select() may report a socket file descriptor as
"ready for reading", while nevertheless a subsequent read
blocks. This could for example happen when data has arrived
but upon examination has wrong checksum and is discarded.
There may be other circumstances in which a file descriptor is
spuriously reported as ready. Thus it may be safer to use
O_NONBLOCK on sockets that should not block.
If you really just want to prevent the automatic restart, look into SA_RESTART in sigaction(2) to prevent restartable system calls from restarting.