I am using a server that is crashing following a call to recv() returning -1 and errno set to ECONNRESET. I originally found this condition using nmap (I'm not a cracker, was just testing if the port was open at the time.) However, nmap uses raw sockets so I'm not too happy submitting this as a test case to the developers. I would rather write a client program in C that can cause the ECONNRESET.
So far I have tried two things: connect() to the server from my client and then shutdown() the socket immediately after connecting. recv() on the server still returned 1 (I have inserted debugging code so I can see the return value.) I also tried calling send() with some string and then immediately calling shutdown(). No dice, the string was transmitted fine.
So how would I cause this condition? Non portable is fine, I am using Linux.
The problem is that you are calling shutdown. Call close instead.
Take a look at a TCP state diagram.
http://tangentsoft.net/wskfaq/articles/debugging-tcp.html
Basically, shutdown closes a socket "politely" by sending a FIN and waiting for the peer to finish (FIN -> ACK/FIN -> ACK -> closed), at which point you call close and all is good. If you call close without calling shutdown first, it's the "impolite" version which sends a RST -- the equivalent of hanging up in the middle of a phone call, without waiting for the other person to finish what they're saying.
Think of "shutdown" as "say goodbye", and "close" as "hang up". You always have to hang up, but you don't have to say goodbye first.
About nmap: It is perfectly acceptable to give developers a test case with nmap. That's one of the main purposes of nmap anyway.
Your instincts were correct to use shutdown(), however you were not using it correctly for this.
Presumably you are trying shutdown() with SHUT_WR or SHUT_RDWR. When you close the writing direction, as these do, your side of the connection notifies the peer with a FIN - indicating that no more data will be forthcoming from your side. This will cause recv() on the other side to indicate a clean end-of-file on the connection, which isn't what you want in this case.
Instead, you want to use SHUT_RD to shutdown the reading direction of the socket only, and hold it open for writing. This will not notify the peer immediately - but if the peer sends any data after this point, your side will respond with a RST, to inform the peer that some data was lost - it wasn't seen by your client application.
(So, to ensure that you get a connection reset, you need to make sure that the server will be trying to send something to you - you might need to send something first, then perform the reading shutdown).
Related
I know that, here, on SO, are many questions themed like this. I've read through most of the similar questions and can not find an answer for my case.
I use kqueue for server/client socket echo application. The program uses exclusively BSD socket API. The program is work in progress. Now I am at the point of getting EOF from socket.
My setup follows.
Start server, that waits for connections, and accepts one socket.
Start client that connects.
No user data sent by this time. Close the client with SIGINT.
Server kqueue gets EOF flag with no errors.
read system call returns zero with no errors.
The problem is that I get no indication that connection was fully closed. I can not determine if I have to shutdown read end, or completely close a socket. I get no indication of EOF with the write end. And that is expected, since I did not register for the write event(no data were sent by now).
How to properly tell, if the socket was fully closed?
Update
I know that what follows may belong to other post. I think that this update is tightly connected with the question, and the question will benefit as a whole.
To the point. Since I get a read EOF, but not a write EOF(the socket is closed before any data comes in, or out), can I somehow query socket for its state?
What I learned from other network related questions, here, on SO, that network stack may get some packets on a socket. Like FIN, or RST. It will be a sure win for me to just get the socket state, in the particular case.
As a second option, will it help to add one-time write event after I got a read EOF, just to get a write EOF? Will the write EOF event trigger?
I know I will get write error eventually. But, until that time, the socket will be a dead weight.
It will be of a great convenience to getsockopt for the write end close. Or, at least, queuing an event for read endpoint shutdown, after the read returned EOF.
I did not found similar getsockopt options, and I am not sure about queue'ing write event. The source code for kevent, and a network stack in general, is too tough for me.
That is why I ask.
If read or recv returns 0 then that means the other end closed the connection. It's at least a half-close for writing (from the other peer), which means there's nothing more to be received from that connection.
Unless the protocol specifies that it's only a half-close and that you can continue to send data, it's generally best to simply do a full closing of the connection from your side.
I'm working with a fairly basic server/client setup, where both are located on the same network. They communicate via Winsock2 blocking sockets over TCP/IP, and are doing so perfectly fine.
However, for the scenario described below, the client sometimes sees an abortive connection termination (RST). It goes right roughly 99 out of 100 times, but that last time annoyingly fails some tests and therefore, my whole build. It is completely unpredictable when and where it happens, and so reproducing the problem has so far eluded me.
If I understand the the relevant MSDN page correctly, the nominal connection termination sequence for blocking sockets can be summarized as:
Client | Server
-----------------------------
shutdown(SD_SEND) |
| send() response data
i=recv() until i==0 | shutdown(SD_SEND)
closesocket() | closesocket()
In my setup it is necessary to
do a relatively expensive operation (let's call it expensive_operation()) depending on whether a portion of the received data (let's say, 512 bytes) contains a trigger value. The server is single-threaded, so expensive_operation() effectively stops recv()ing the data stream until expensive_operation() is complete
initiate a server shutdown sequence if the client sends a particular sentinel value, let's call it 0xDEADBEEF.
My client is implemented such that the sentinel value is always sent last, so after sending it, no other data is sent:
send( "data data data 0xDEADBEEF" ) to server
shutdown(SD_SEND) <------- FAILURE OCCURS HERE
recv() until 0 bytes received
closesocket()
Whenever the server receives 0xDEADBEEF, it confirms the shutdown request and continues termination:
recv() 512 bytes of data or until 0 bytes are returned
Check for trigger. If a trigger is found, perform expensive_operation() and go back to step 1, otherwise continue
Check for sentinel value. If sentinel is not found, go back to step 1.
If the sentinel is found:
send( confirmation ) to client
shutdown(SD_SEND)
closesocket()
all the normal server shutdown stuff
I can understand that if the client intends to send more data after the sentinel, this will result in abortive connection termination -- because the server actively terminates the connection. This is completely expected and by design, and I can indeed reliably reproduce this behavior.
However, in the nominal case, the sentinel is always last in the sequence, which indeed always happens as evidenced by the relevant log entries, and indeed graceful connection termination happens as expected most of the time. But not always...
As I said, it happens randomly and sporadically, so I can't produce a code snippet that reliably reproduces the problem. The only thing that's consistent is that the failure always occurs when calling shutdown() in the client...
I suspect it's more of a design flaw, or some synchronization issue I'm not handling yet, rather than a problem with the code (although I'd be happy to provide the relevant code snippets).
So is there anything obvious I'm overlooking here?
There are several ways you can provoke an RST to be sent apart from deliberately doing so at the sending end by means which I will not reveal here:
Write to a connection that had already been closed by the peer. After a few attempts this will cause ECONNRESET.
Close a connection without having read all the already-pending data. This will cause an immediate ECONNRESET.
Both of these indicate an application protocol error.
In your case I would get rid of the sentinel. It is redundant. Just shutdown the socket for output, or just close it if you know there is no more data coming in. That sends an entirely unambiguous indication to the peer that there is no more data, without any requirement that the peer be precisely sycnchronized byte-for-byte with the local application, which is a weakness and probable source of this bug in your current code.
You need to post some code to get any more concrete assistance.
I cannot reproduce, but I can imagine a use case where client sees abortive termination
client server
send sentinel
send confirmation
shutdown
close socket
shutdown => error writing on closedsocket !
if client process is preempted just after sending its sentinel, and if server is quick, you can fall in that scenario. This is caused by the fact that server side you immediately close the socket after shutdown without being sure client has done its shutdown. IMHO you should do
send( confirmation ) to client
shutdown(SD_SEND)
read until 0 or error
closesocket()
all the normal server shutdown stuff
The order is then deterministic for the upper part :
client server
send sentinel
send confirmation
shutdown shutdown
recv 0
close socket => cannot happen before client shutdown
recv 0 => socket may be closed server side but it would be harmless
I'm a newbie to socket programming, I know it's a bad habit to close socket using "control-c", but why socket on the receiving peer keeps receiving '' infinitely after I use "control-c" to close the sending process? shouldn't the socket on the sending peer be closed after "control-c" to exit the process? Thanks!
I know it's a bad habit to close socket using "control-c"
That closes the entire process, not just a socket.
why socket on the receiving peer keeps receiving '' infinitely after I use "control-c" to close the sending process?
At a guess, which is all that is possible without seeing the code you should have posted in your question, you are ignoring errors and end-of-stream when calling recv().
shouldn't the socket on the sending peer be closed after "control-c" to exit the process?
It is. The whole process is 'closed', including all its resources.
As regards the receiving socket, it is up to you to detect the conditions under which it should be close, and close it.
No code given, but here's an educated guess of what might be going on:
You have two separate bits of code running: Sending and receiving
You are in the process of transferring data when you use CTL+C to kill the sending socket.
You expect the receiving socket to stop, but it doesn't.
The issue could be one of "end of transmission" agreement. If the sending code fires off an End-of-File (EOF) (or abort or terminated) when you hit CTL+C, then the receiving socket should see that and quit receiving. However, you haven't stated what the sending code is doing the moment you hit CTL+C.
The receiving socket may just be waiting for more data; as far as the receiving code is concerned, it was to be told when transfer is done, and it is patiently waiting for more information.
There are far better socket programmers around than I am, but I think it's safe to say that, once you get down to that level, you should pay attention to the details of the transfer protocol. If CTL+C just terminates the server (sending) code, then the client has no idea if there is a real termination, an unexpected delay in the transmission, or the server process just had a brain-fart and will start sending again once things clear up.
If you have any means of monitoring the actual values going back an forth, take a look at what happens during a "normal" termination of data transfer and a CTL+C termination. This might help you zero in on the undesirable behavior.
In C, I understood that if we close a socket, it means the socket will be destroyed and can be re-used later.
How about shutdown? The description said it closes half of a duplex connection to that socket. But will that socket be destroyed like close system call?
This is explained in Beej's networking guide. shutdown is a flexible way to block communication in one or both directions. When the second parameter is SHUT_RDWR, it will block both sending and receiving (like close). However, close is the way to actually destroy a socket.
With shutdown, you will still be able to receive pending data the peer already sent (thanks to Joey Adams for noting this).
None of the existing answers tell people how shutdown and close works at the TCP protocol level, so it is worth to add this.
A standard TCP connection gets terminated by 4-way finalization:
Once a participant has no more data to send, it sends a FIN packet to the other
The other party returns an ACK for the FIN.
When the other party also finished data transfer, it sends another FIN packet
The initial participant returns an ACK and finalizes transfer.
However, there is another "emergent" way to close a TCP connection:
A participant sends an RST packet and abandons the connection
The other side receives an RST and then abandon the connection as well
In my test with Wireshark, with default socket options, shutdown sends a FIN packet to the other end but it is all it does. Until the other party send you the FIN packet you are still able to receive data. Once this happened, your Receive will get an 0 size result. So if you are the first one to shut down "send", you should close the socket once you finished receiving data.
On the other hand, if you call close whilst the connection is still active (the other side is still active and you may have unsent data in the system buffer as well), an RST packet will be sent to the other side. This is good for errors. For example, if you think the other party provided wrong data or it refused to provide data (DOS attack?), you can close the socket straight away.
My opinion of rules would be:
Consider shutdown before close when possible
If you finished receiving (0 size data received) before you decided to shutdown, close the connection after the last send (if any) finished.
If you want to close the connection normally, shutdown the connection (with SHUT_WR, and if you don't care about receiving data after this point, with SHUT_RD as well), and wait until you receive a 0 size data, and then close the socket.
In any case, if any other error occurred (timeout for example), simply close the socket.
Ideal implementations for SHUT_RD and SHUT_WR
The following haven't been tested, trust at your own risk. However, I believe this is a reasonable and practical way of doing things.
If the TCP stack receives a shutdown with SHUT_RD only, it shall mark this connection as no more data expected. Any pending and subsequent read requests (regardless whichever thread they are in) will then returned with zero sized result. However, the connection is still active and usable -- you can still receive OOB data, for example. Also, the OS will drop any data it receives for this connection. But that is all, no packages will be sent to the other side.
If the TCP stack receives a shutdown with SHUT_WR only, it shall mark this connection as no more data can be sent. All pending write requests will be finished, but subsequent write requests will fail. Furthermore, a FIN packet will be sent to another side to inform them we don't have more data to send.
There are some limitations with close() that can be avoided if one uses shutdown() instead.
close() will terminate both directions on a TCP connection. Sometimes you want to tell the other endpoint that you are finished with sending data, but still want to receive data.
close() decrements the descriptors reference count (maintained in file table entry and counts number of descriptors currently open that are referring to a file/socket) and does not close the socket/file if the descriptor is not 0. This means that if you are forking, the cleanup happens only after reference count drops to 0. With shutdown() one can initiate normal TCP close sequence ignoring the reference count.
Parameters are as follows:
int shutdown(int s, int how); // s is socket descriptor
int how can be:
SHUT_RD or 0
Further receives are disallowed
SHUT_WR or 1
Further sends are disallowed
SHUT_RDWR or 2
Further sends and receives are disallowed
This may be platform specific, I somehow doubt it, but anyway, the best explanation I've seen is here on this msdn page where they explain about shutdown, linger options, socket closure and general connection termination sequences.
In summary, use shutdown to send a shutdown sequence at the TCP level and use close to free up the resources used by the socket data structures in your process. If you haven't issued an explicit shutdown sequence by the time you call close then one is initiated for you.
I've also had success under linux using shutdown() from one pthread to force another pthread currently blocked in connect() to abort early.
Under other OSes (OSX at least), I found calling close() was enough to get connect() fail.
"shutdown() doesn't actually close the file descriptor—it just changes its usability. To free a socket descriptor, you need to use close()."1
Close
When you have finished using a socket, you can simply close its file descriptor with close; If there is still data waiting to be transmitted over the connection, normally close tries to complete this transmission. You can control this behavior using the SO_LINGER socket option to specify a timeout period; see Socket Options.
ShutDown
You can also shut down only reception or transmission on a connection by calling shutdown.
The shutdown function shuts down the connection of socket. Its argument how specifies what action to perform:
0
Stop receiving data for this socket. If further data arrives, reject it.
1
Stop trying to transmit data from this socket. Discard any data waiting to be sent. Stop looking for acknowledgement of data already sent; don’t retransmit it if it is lost.
2
Stop both reception and transmission.
The return value is 0 on success and -1 on failure.
in my test.
close will send fin packet and destroy fd immediately when socket is not shared with other processes
shutdown SHUT_RD, process can still recv data from the socket, but recv will return 0 if TCP buffer is empty.After peer send more data, recv will return data again.
shutdown SHUT_WR will send fin packet to indicate the Further sends are disallowed. the peer can recv data but it will recv 0 if its TCP buffer is empty
shutdown SHUT_RDWR (equal to use both SHUT_RD and SHUT_WR) will send rst packet if peer send more data.
linux: shutdown() causes listener thread select() to awake and produce error. shutdown(); close(); will lead to endless wait.
winsock: vice versa - shutdown() has no effect, while close() is successfully catched.
In the client, I have a
close(sockfd)
where sockfd is the socket that's connected to the server.
In the server I've got this:
if (sockfd.revents & POLLERR ||
desc_set[i].revents & POLLHUP || desc_set[i].revents & POLLNVAL) {
close(sockfd.fd);
printf("Goodbye (connection closed)\n");
}
Where sockfd is a struct pollfd, and sockfd.fd is the file descriptor of the client's socket.
When the client closes the socket like I put up there, the server doesn't seem to detect it with the second code (desc_set[i].revents & POLLHUP, etc.).
Does anyone know what's the problem?
Sounds like you've managed to half close the connection from the client side. In this state the connection can still send data in one direction, i.e. it operates in half-duplex mode. This is by design and would allow your server to finish replying to whatever the client sent. Typically this would mean completing a file transfer and calling close(), or answering all of the aspects of the query. In the half-closed state you can still quite sensibly send data to the side that has already called close(). In your server you will see eof if you try to read though. close() just means "I'm done sending, finish up whatever I asked for".
POLLHUP, POLLERR and POLLNVAL only checks the output side of the local connection, which is still valid here. There's a POLLRDHUP, which is a GNU extension that should detect the other side closing, but the tests you're doing are only checking if it's still writable, not if it's still readable.
See also this question, which is talking about java, but still very related.
A remote close or output shutdown is neither an error nor a hangup nor an invalid state. It is a read event such that read() will return zero. Just handle it as part of your normal read processing.
BTW your test condition above should read sockfd.revents & (POLLERR|POLLHUP|POLLNVAL).