In C, I understood that if we close a socket, it means the socket will be destroyed and can be re-used later.
How about shutdown? The description said it closes half of a duplex connection to that socket. But will that socket be destroyed like close system call?
This is explained in Beej's networking guide. shutdown is a flexible way to block communication in one or both directions. When the second parameter is SHUT_RDWR, it will block both sending and receiving (like close). However, close is the way to actually destroy a socket.
With shutdown, you will still be able to receive pending data the peer already sent (thanks to Joey Adams for noting this).
None of the existing answers tell people how shutdown and close works at the TCP protocol level, so it is worth to add this.
A standard TCP connection gets terminated by 4-way finalization:
Once a participant has no more data to send, it sends a FIN packet to the other
The other party returns an ACK for the FIN.
When the other party also finished data transfer, it sends another FIN packet
The initial participant returns an ACK and finalizes transfer.
However, there is another "emergent" way to close a TCP connection:
A participant sends an RST packet and abandons the connection
The other side receives an RST and then abandon the connection as well
In my test with Wireshark, with default socket options, shutdown sends a FIN packet to the other end but it is all it does. Until the other party send you the FIN packet you are still able to receive data. Once this happened, your Receive will get an 0 size result. So if you are the first one to shut down "send", you should close the socket once you finished receiving data.
On the other hand, if you call close whilst the connection is still active (the other side is still active and you may have unsent data in the system buffer as well), an RST packet will be sent to the other side. This is good for errors. For example, if you think the other party provided wrong data or it refused to provide data (DOS attack?), you can close the socket straight away.
My opinion of rules would be:
Consider shutdown before close when possible
If you finished receiving (0 size data received) before you decided to shutdown, close the connection after the last send (if any) finished.
If you want to close the connection normally, shutdown the connection (with SHUT_WR, and if you don't care about receiving data after this point, with SHUT_RD as well), and wait until you receive a 0 size data, and then close the socket.
In any case, if any other error occurred (timeout for example), simply close the socket.
Ideal implementations for SHUT_RD and SHUT_WR
The following haven't been tested, trust at your own risk. However, I believe this is a reasonable and practical way of doing things.
If the TCP stack receives a shutdown with SHUT_RD only, it shall mark this connection as no more data expected. Any pending and subsequent read requests (regardless whichever thread they are in) will then returned with zero sized result. However, the connection is still active and usable -- you can still receive OOB data, for example. Also, the OS will drop any data it receives for this connection. But that is all, no packages will be sent to the other side.
If the TCP stack receives a shutdown with SHUT_WR only, it shall mark this connection as no more data can be sent. All pending write requests will be finished, but subsequent write requests will fail. Furthermore, a FIN packet will be sent to another side to inform them we don't have more data to send.
There are some limitations with close() that can be avoided if one uses shutdown() instead.
close() will terminate both directions on a TCP connection. Sometimes you want to tell the other endpoint that you are finished with sending data, but still want to receive data.
close() decrements the descriptors reference count (maintained in file table entry and counts number of descriptors currently open that are referring to a file/socket) and does not close the socket/file if the descriptor is not 0. This means that if you are forking, the cleanup happens only after reference count drops to 0. With shutdown() one can initiate normal TCP close sequence ignoring the reference count.
Parameters are as follows:
int shutdown(int s, int how); // s is socket descriptor
int how can be:
SHUT_RD or 0
Further receives are disallowed
SHUT_WR or 1
Further sends are disallowed
SHUT_RDWR or 2
Further sends and receives are disallowed
This may be platform specific, I somehow doubt it, but anyway, the best explanation I've seen is here on this msdn page where they explain about shutdown, linger options, socket closure and general connection termination sequences.
In summary, use shutdown to send a shutdown sequence at the TCP level and use close to free up the resources used by the socket data structures in your process. If you haven't issued an explicit shutdown sequence by the time you call close then one is initiated for you.
I've also had success under linux using shutdown() from one pthread to force another pthread currently blocked in connect() to abort early.
Under other OSes (OSX at least), I found calling close() was enough to get connect() fail.
"shutdown() doesn't actually close the file descriptor—it just changes its usability. To free a socket descriptor, you need to use close()."1
Close
When you have finished using a socket, you can simply close its file descriptor with close; If there is still data waiting to be transmitted over the connection, normally close tries to complete this transmission. You can control this behavior using the SO_LINGER socket option to specify a timeout period; see Socket Options.
ShutDown
You can also shut down only reception or transmission on a connection by calling shutdown.
The shutdown function shuts down the connection of socket. Its argument how specifies what action to perform:
0
Stop receiving data for this socket. If further data arrives, reject it.
1
Stop trying to transmit data from this socket. Discard any data waiting to be sent. Stop looking for acknowledgement of data already sent; don’t retransmit it if it is lost.
2
Stop both reception and transmission.
The return value is 0 on success and -1 on failure.
in my test.
close will send fin packet and destroy fd immediately when socket is not shared with other processes
shutdown SHUT_RD, process can still recv data from the socket, but recv will return 0 if TCP buffer is empty.After peer send more data, recv will return data again.
shutdown SHUT_WR will send fin packet to indicate the Further sends are disallowed. the peer can recv data but it will recv 0 if its TCP buffer is empty
shutdown SHUT_RDWR (equal to use both SHUT_RD and SHUT_WR) will send rst packet if peer send more data.
linux: shutdown() causes listener thread select() to awake and produce error. shutdown(); close(); will lead to endless wait.
winsock: vice versa - shutdown() has no effect, while close() is successfully catched.
Related
In TCP we say one side of the connection performs an "active close" and the other side performs a "passive close".
In terms of the Linux sockets API, how do you differentiate the active close and the passive close?
For example, suppose we have two connected Linux TCP sockets, A and P, that have exchanged information over the application-level protocol and they are both aware that it is time to close their sockets (neither expect to send or receive any more data to or from each other).
We want socket A to perform the active close, and for P to be the passive close.
There are a few things A and P could do. For example:
call shutdown(SHUT_WR)
call recv and expect to get 0 back
call close.
something else
What combination of these things and in what order should A do?... and what combination of these things and in what order should P do?
In terms of the Linux sockets API, how do you differentiate the active
close and the passive close?
The 'active' close is simply whichever side of the socket sends a FIN or RST packet first, typically by calling close().
What combination of these things and in what order should A do?... and
what combination of these things and in what order should P do?
In practice, most of this is application- and application-protocol specific. I will describe the minimum/typical requirement to answer your question, but your mileage may vary depending on what you are specifically trying to accomplish.
You may first call shutdown() on Socket A if you want to terminate communication in one direction or the other (or both) on Socket A. From your description, both programs already know they're done, perhaps due to application protocol messages, so this may not be necessary.
You must call close() on Socket A in order to close the socket and release the file descriptor.
On Socket P, you simply keep reading until recv() returns 0, and then you must call close() to close the socket and release the file descriptor.
For further reading, there are a number of good tutorials out there, and Beej's Guide to Network Programming is quite popular.
Active open is when you issue connect(2) explicitly to make a connection to a remote site. The call blocks until you get the socket opened on the other side (except if you issued O_NONBLOCK fcntl(2) call before calling connect(2).
Passive open is when you have a socket listen(2)ing on a connection and you have not yet issued an accept(2) system call. The accept(2) call normally blocks until you have a completely open connection and gives you a socket descriptor to communicate over it, or gives you inmediately a socket descriptor if the connection handshake has already finished when you issue the accept(2) syscall (this is a passive open). The limit in the number of passively open connections the kernel can accept on your behalf while you prepare yourself to make the accept(2) system call is what is called the listen(2) value.
Active close is what happens when you explicitly call shutdown(2) or close(2) system calls. As with passive open, there's nothing you can do to make a passive close (it's something that happens behind the scenes, product of other side's actions). You detect a passive close when the socket generates an end of file condition (this is, read(2) always returns 0 bytes on reading) meaning the other end has done a shutdown(2) (or close(2)) and the connection is half (or full) closed. When you explicitly shutdown(2) or close(2) your side, it's an active close.
NOTE
if the other end does an explicit close(2) and you continue writing on the socket, you'll get an error due to the impossibility of sending that data (in this case we can talk about a passive close(2) ---one that has occured without any explicit action from our side) but the other end can do a half close calling shutdown(2). This makes the tcp to send a FIN segment only and conserves the socket descriptor to allow the thread to receive any pending data in transit or not yet sent. Only when it receives and acknowledges the other end's FIN segment will it signal you that no more data remains in transit.
According to Unix Network Programming when a socket writes twice to a closed socket (after a FIN packet), then in the first time it succeeded to send, but receives an RST packet from the other host. Since the host receives an RST, the socket is destroyed. Thus in the second time it writes, the SIGPIPE signal is received, and an EPIPE error is returned.
However, in send man pages ECONNRESET can be returned, which means that an RST packet is received. When it returns ECONNRESET -there no signal is returned.
What are the cases ECONNRESET can be returned? and why does there is no SIGPIPE signal in this case?
Note: I have checked I similar question here. However, when I run in my linux computer, send returned the EPIPE error, and not ECONNRESET.
If the peer closed the connection while there were still unhandled data in the socket buffer it will send a RST packet back. This will cause a flag to be set on the socket and the next send will return ECONNRESET as the result . EPIPE instead is returned (or SIGPIPE triggered) on send if the connection was closed by the peer with no outstanding data. In both cases the local socket is still open (i.e. the file descriptor is valid), but the underlying connection is closed.
Example: Imagine a server which reads a single byte and then closes the connection:
EPIPE: The client sends first one byte. After the server read the byte and closed the connection the client will send some more data and then again some data. The latest send call will trigger EPIPE/SIGPIPE.
ECONNRESET: The client sends first more than one byte. The server will read a single byte and close the connection with more bytes in the sockets receive buffer. This will trigger a connection RST packet from the server and on the next send the client will receive ECONNRESET.
A TCP connection can be seen as two data pipelines between two endpoints. One data pipeline for sending data from A to B and one data pipeline for sending data from B to A. These two pipelines belong to a single connection but they don't otherwise influence each other. Sending data on one pipeline has no effect on data being sent on the other pipeline. If data on one pipeline is reply data to data sent previously on the other pipeline, this is something only your application will know, TCP knows nothing about that. The task of TCP is to make sure that data reliably makes it from one end of the pipeline to the other end and that as fast as possible, that is all that TCP cares for.
As soon as one side is done sending data, it tells the other side it is done by tranmitting it a packet with the FIN flag set. Sending a FIN flag means "I have sent all the data I wanted to send to you, so my send pipeline is now closed". You can trigger that intentionally in your code by calling shutdown(socketfd, SHUT_WR). If the other side will then call recv() on the socket, it won't get an error but receive will say that it read zero bytes, which means "end of stream". End of stream is not an error, it only means that no more data will ever arrive there, no matter how often you are going to call recv() on that socket.
Of course, this doesn't affect the other pipeline, so when A -> B is closed, B -> A can still be used. You can still receive from that socket, even though you closed your sending pipeline. At some point, though, also B will be done with sending data and also transmit a FIN. Once both pipelines are closed, the connection as a whole is closed and this would be a graceful shutdown, as both sides have been able to send all the data they wanted to send and no data should have been lost, since as long as there was unconfirmed data in flight, the other side would not have said it is done but wait for that data to be reliably transferred first.
Alternatively there is the RST flag which closes the entire connection at once, regardless if the other side was done sending and regardless if there was unconfirmed data in flight, so a RST has a high potential of causing data to be lost. As that is an exceptional situation that may require special handling, it would be useful for programmers to know if that was the case, that's why there exists two errors:
EPIPE - You cannot send over that pipe as that pipe is not valid anymore. However, all data that you were sending before it broke was still reliably delivered, you just cannot send any new data.
ECONNRESET - Your pipe is broken and it may be the case that data you were trying to send before got lost in the middle of transfer. If that is a problem, you better handle it somehow.
But these two errors do not map one to one to the FIN and RST flag. If you receive a RST in a situation where the system sees no risk of data loss, there is no reason to drive you round the bend for nothing. So if all data you sent before was ACKed to be correctly received and then the connection was closed by a RST when you tried to send new data, no data was lost. This includes the current data you tried to send as this data wasn't lost, it was never sent on the way, that's a difference as you still have it around whereas data you were sending before may not be around anymore. If your car breaks down in the middle of a road trip then this is quite a different situation than if you you are still at home as your car engine refused to even start. So in the end it's your system that decides if a RST triggers a ECONNRESET or a EPIPE.
Okay, but why would the other side send you a RST in the first place? Why not always closing with FIN? Well, there exists a couple of reasons but the two most prominent ones are:
A side can only signal the other one that it is done sending but the only way to signal that it is done with the entire connection is to send a RST. So if one side wants to close a connection and it wants to close it gracefully, it will first send a FIN to signal that it won't send new data anymore and then give the other side some time to stop sending data, allowing in-flight data to pass through and to finally send a FIN as well. However, what if the other side doesn't want to stop and keeps sending and sending? This behavior is legal as a FIN doesn't mean that the connection needs to close, it only means one side is done. The result is that the FIN is followed by RST to finally close that connection. This may have caused in-flight data to be lost or it may not, only the recipient of the RST will know for sure as if data was lost, it must have been on his side since the sender of the RST was surely not sending any more data after the FIN. For a recv() call, this RST has no effect as there was a FIN before signaling "end of stream", so recv() will report having read zero bytes.
One side shall close the connection, yet it sill has unsent data. Ideally it would wait till all unsent data has been sent and then transmit a FIN, however, the time it is allowed to wait is limited and after that time has passed, there is still unsent data left. In that case it cannot send a FIN as that FIN would be a lie. It would tell the other side "Hey, I sent all the data I wanted to send" but that's not true. There was data that should have been sent but as the close was required to be instant, this data had to be discarded and as a result, this side will directly send a RST. Whether this RST triggers a ECONNRESET for the send() call depends again on the fact, if the recipient of the RST had unsent data in flight or not. However, it will for sure trigger a ECONNRESET error on the next recv() call to tell the program "The other side actually wanted to send more data to you but it couldn't and thus some of that data was lost", since this may again be a situation that handling somehow, as the data you've received was for sure incomplete and this is something you should be made aware of.
If you want to force a socket to be always closed directly with RST and never with FIN/FIN or FIN/RST, you can just set the Linger time to zero.
struct linger l = { .l_onoff = 1, .l_linger = 0 };
setsockopt(socketfd, SOL_SOCKET, SO_LINGER, &l, sizeof(l));
Now the socket must close instantly and without any delay, no matter how little and the only way to close a TCP socket instantly is to send a RST. Some people think "Why enabling it and setting time to zero? Why not just disabling it instead?" but disabling has a different meaning.
The linger time is the time a close() call may block to perform pending send actions to close a socket gracefully. If enabled (.l_onoff != 0), a call to close() may block for up to .l_linger seconds. If you set time to zero, it may not block at all and thus terminates instantly (RST). However, if you disable it, then close() will never block either but then the system may still linger on close, yet this lingering happens in the background, so your process won't notice it any longer and thus also cannot know when the socket has really closed, as the socketfd becomes invalid at once, even if the underlying socket in kernel still exists.
I have a client application communicating with a QEMU process through a QMP Unix domain socket. Sometimes after the client calls close() on the socket connection, 'netstat -ap unix' still shows it in CONNECTED state. I do check the return value of the close() call and it returns successfully with a value of 0, but the connection still seems to be lingering.
Since QMP doesn't really support multiple connections on its socket, all the subsequent calls to connect to the socket fail since they wait indefinitely for the lingering connection to be closed.
Is there a way to make sure from the code that the socket is really closed, and is there a way to force the socket to close?
It could be that the file descriptor has been duped, forked, or leaked.
Call shutdown(sock, SHUT_RDWR) on it to close the connection for sure before closeing.
Have you tried closing the socket from the other end? It's asynchronous, but it gives both sides a chance to ensure socket closure.
You can send a close command through to the listener on the other end and have it recycle the socket. When the socket gets closed, you should end up getting a SIGPIPE. Catch the SIGPIPE and close your end of the socket. If you end up with an EPIPE doing that, then ignore it. That just means you were already notified of the socket closure.
You could just try SO_LINGER via setsockopt(2) option with a timeout of 0. This way, when you close the socket is forcibly closed, sending a RST instead of going into the FIN/ACK closing behavior.
The purpose of the SO_LINGER option is to control how the socket is shut down when the function close(2) is called. This option applies only to connection-oriented protocols such as TCP.
The default behavior of the kernel is to allow the close(2) function to return immediately to the caller. Any unsent TCP/IP data will be transmitted and delivered if possible, but no guarantee is made. Because the close(2) call returns control immediately to the caller, the application has no way of knowing whether the last bit of data was actually delivered.
The SO_LINGER option can be enabled on the socket, to cause the application to block in the close(2) call until all final data is delivered to the remote end. Furthermore, this assures the caller that both ends have acknowledged a normal socket shutdown. Failing this, the indicated option timeout occurs and an error is returned to the calling application.
One final scenario can be applied, by use of different SO_LINGER option values. If the calling application wants to abort communications immediately, appropriate values can be set in the linger structure. Then, a call to close(2) will initiate an abort of the communication link, discarding all pending data and immediately close the socket.
when a tcp client establishes a tcp connection with my TCP server, after it sends several packets, I want to make a hard/abortive close on this TCP connection, how to do it in linux C?
the hard/abortive close here means the tcp server will send a RST immediately to the client. No FIN/ACK at all.
thanks!
From Socket man page
SO_LINGER
Sets or gets the SO_LINGER option. The argument is a linger structure.
struct linger {
int l_onoff; /* linger active */
int l_linger; /* how many seconds to linger for */
};
When enabled, a close(2) or shutdown(2) will not return until all queued messages
for the socket have been successfully sent or the linger timeout has been reached. Otherwise, the call returns immediately and the closing is done in the background. When the socket is closed as part of exit(2), it always lingers in the background.
Another settings :
The effect of an setsockopt(..., SO_LINGER,...) depends on what the values in the linger structure (the third parameter passed to setsockopt()) are:
Case 1: linger->l_onoff is zero (linger->l_linger has no meaning):
This is the default.
On close(), the underlying stack attempts to gracefully shutdown the connection after ensuring all unsent data is sent. In the case of connection-oriented protocols such as TCP, the stack also ensures that sent data is acknowledged by the peer. The stack will perform the above-mentioned graceful shutdown in the background (after the call to close() returns), regardless of whether the socket is blocking or non-blocking.
Case 2: linger->l_onoff is non-zero and linger->l_linger is zero:
A close() returns immediately. The underlying stack discards any unsent data, and, in the case of connection-oriented protocols such as TCP, sends a RST (reset) to the peer (this is termed a hard or abortive close). All subsequent attempts by the peer's application to read()/recv() data will result in an ECONNRESET.
Case 3: linger->l_onoff is non-zero and linger->l_linger is non-zero:
A close() will either block (if a blocking socket) or fail with EWOULDBLOCK (if non-blocking) until a graceful shutdown completes or the time specified in linger->l_linger elapses (time-out). Upon time-out the stack behaves as in case 2 above.
You should set the SO_LINGER socket option with timeout 0 and call close(). If there's any pending data to send it is lost and an RST is sent instead of FIN.
Reference: http://alas.matf.bg.ac.rs/manuals/lspe/snode=105.html
I have a AF_INET/SOCK_STREAM server written in C running on Android/Linux which looks more ore less like this:
...
for (;;) {
client = accept(...);
read(client, &message, sizeof(message));
response = process(&message);
write(client, response, sizeof(*response));
close(client);
}
As far as I know, the call to close should not terminate the connection to the client immediately, but it apparently does: The client reports "Connection Reset by Peer" before it has had a chance to read the server's response.
If I insert a delay between write() and close() the client can read the response as expected.
I got a hint that it might have to do with the SO_LINGER option, but I checked it's value and both members of struct linger (l_onoff, l_linger) have a value of zero.
Any ideas?
Stevens describes a configuration in which this can happen, but it depends on the client sending more data after the server has called close() (after the client should “know” that the connection is being closed). UNP 2nd ed s5.12.
Try tcpdumping the conversation to find out what’s really going on. If there's any possibility that a “clever” gateway (e.g. NAT) is between the two endpoints, tcpdump both ends and look for discrepancies.
Connection gets reset when you call close() on connection with data being sent. Specially for this case the sequence of shutdown() with SHUT_WR flag and then blocking read() is used.
Shutting down the writing end of the socket sends FIN and returns immediately, and the said read() blocks and returns 0 as soon as your peer replies with FIN in due turn. Basically, this is what you need in place of the delay between write() and close() you are talking about.
You do not need do anything with linger options in this case, leave it all to default.
SO_LINGER should be set (i.e. set to 1 not 0) if you want queued data to be sent before a close is effected.
SO_LINGER
Lingers on a close() if data is present. This option controls the
action taken when unsent messages
queue on a socket and close() is
performed. If SO_LINGER is set, the
system shall block the calling thread
during close() until it can transmit
the data or until the time expires. If
SO_LINGER is not specified, and
close() is issued, the system handles
the call in a way that allows the
calling thread to continue as quickly
as possible. This option takes a
linger structure, as defined in the
header, to specify the
state of the option and linger
interval.