I've a client and server running on the same server(Linux machine) and TCP connection in between them.
I've observed that when I kill client, Kernel/OS sends RST packet after exactly 2 seconds after the client is killed.
My question is which kernel parameter os Socket options govern this timer(2 secs)?
A RST isn't ordinarily sent between peers in a normal connection termination. A FIN is. When you kill the client, a FIN is sent on the connection to indicate to the server that the client won't be sending any more data.
But the server is apparently not paying attention to the FIN it receives when the client is killed (i.e. it would need to attempt a recv on the socket and react appropriately to the end-of-file indication it will get -- usually that means close its own socket). Subsequently, the server is attempting to send data to the client but the connection is closed. That is what results in a RST packet being sent.
RST means (roughly) "there is no active connection available to receive the data you're sending; it's pointless to send more."
And so the timing of that RST is likely based on when the server next attempts to send to the client, not on any kernel / OS configuration setting. If the server doesn't attempt to send and it doesn't close, the connection should just sit there idle forever, and no RST will be sent.
As mentioned in the UNIX Network programming Volume 1 Section Generic socket options, if a client is killed, TCP will send a FIN across the connection.
Related
I have a server written in C that closes the connection if the connection is sitting idle for a specific time. I have an issue (that rarely happens). Read is failing on the client side and it says Connection broken. I suspect the server is closing the connection and the client is sending some data at the same time.
Consider the following scenario (A is server, B is the client)
B initiates the connection and the connection between A and B is established.
B is sitting idle and the idle timeout is reached.
A initiates the close
Before B receives the FIN from A, it starts sending request to A
After B sends the request, it will read the response
Since A has already closed the connection, B is not able to read.
My questions are
Is this a possible situation ?
How to handle idle timeout for clients?
How to close the connection between A and B properly (avoid B sending request during the process). In short, how to close the connection atomically?
By my only little more than rudimentary network experience... and assuming that you are talking about a connection-oriented connection like TCP/IP in contrary to UDP/IP that is connection-less.
Yes, of course. You cannot avoid it.
There are multiple ways to do it, but all of them include: Send something from the client before the server's timeout elapses. If the client has no data to send, let it send something like a "life sign". This could be an empty data message, it all depends on your application protocol. Or make the timeout as long as necessary, including some margin. Some protocol timeout only after 3 times of allowed idle time.
You cannot close the connection atomically, because client and server are separated. Each packet on the network needs some time to be transmitted, and both can start sending at the very same moment, the server its closing message, and the client a new data message. There is nothing that you can do about this.
You need to make the client handle this situation properly. For example, it can accept such a broken connection and interpret it as closed. You should have already some reaction, if the server closes the connection while the client is idle.
How to close the connection between A and B properly (avoid B sending request during the process).
Server detects timeout
Server sends timeout detection message to the Client
Server waits for a reply (if timeout, assume Client dead)
if Client receives a timeout detection from the Server, it replies with ACK (or something like that)
if Server receives an ACK from the Client, then 'gracefully' closes the connection
from now on, neither the Server nor the Client should send/receive any messages (after sending the ACK, do not immediately close the connection from the client side, do linger for the agreed timeout - see setsockopt: SO_LINGER)
On top of that, like the other answers suggested, the Client should send a heartbeat if idle (to avoid timeout detections).
I implemented a hello-world like client & echo server on Linux, and used tcpdump to watch the packet exchanges between the two. The server forks a child process on each accepted connection, nothing fancy.
Once the child process serving the connection is killed, the server TCP socket goes into FIN-WAIT-2 (after sending a FIN and receiving the ACK), shown by command ss -tap. ss also shows it is orphaned since the Process column for this entry is empty.
Then I sent one more message from client, it triggers two more TCP messages:
client push the message to server
server respond with a RST
and then ss shows the server socket went missing, I assume it went back to CLOSED state, and got reclaimed.
My question is this:
I can understand for an un-orphaned socket, FIN_WAIT_2 can serve the purpose of half-closed connection. But for an orphaned socket, what is the point? Why not go back to CLOSED directly? I read from this post that FIN_WAIT_2 helps to prevent a future connection being mistakenly closed, but if that's the reason, then in my case, the server should NOT close the socket after receiving a regular message - it should wait forever until client sends a FIN, correct?
If I have a process with a connected socket, and I terminate this process, then Windows will cause an RST packet to be sent.
Is it guaranteed (is it documented somewhere) that an RST packet will always be sent when a process terminates, or could a FIN packet be sent instead?
TCP is not supposed to send an RST packet when a connection is closed. To close a connection, TCP goes through the following states on the client side:
Send a FIN packet. This action will change TCP state to FIN_WAIT_1.
In FIN_WAIT_1, TCP waits for an acknowledgement (ACK) from the server.
Once the acknowledgement is received, TCP enters FIN_WAIT_2.
In FIN_WAIT_2, TCP waits for a FIN packet from the server
Once FIN arrives, the client sends an ACK and enters TIME_WAIT
TIME_WAIT is exited after a while (typically, 30 secs. or 1 minute). The purpose of this state is to make it possible to resend the final ACK to the server in case it was lost.
There is no RST packet anywhere. RST is used to respond to unexpected traffic, not to close a connection.
For example, if you send a TCP packet to port 80, and the server is not running an HTTP server (and assuming the packet makes all the way to the server and is not blocked / ignored), then an RST reply is sent back to the client.
Well, i have to do a udp server in C on linux.
This is my issue, for example:
I have that each thread will manage a client. But recvfrom will accept packets from any source, so, how can do that each thread will receive the right client's packet and not others?
I hope I explained my issue well.
Thank you!
(I mean that my server will make a new thread for each client that want to be served. So with TCP i can use accept to have a "dedicated" socket for each thread-client connection, but with UDP i can't do the same.)
You can't have multiple threads reading from the same socket at the same time.
What you should do instead is either:
have a dedicated thread that receives all inbound packets, looks at the source IP/Port, and routes the data to the appropriate processing thread as needed.
give each processing thread its own socket that is bind()'ed to the same local IP/Port and connect()'ed to the particular source IP/Port it is interested in, then each thread can call recvfrom() independently and it will only return packets that match the source that the thread is expecting.
UDP is a message based connection-less protocol. Here, there is no definite connection / setup done between the client and server prior sending the data. It does not maintain any states for communication at either client or server. UDP uses datagram socket(SOCK_DGRAM) . This ensures each packet sent or received in any order but with preserved message boundaries.
TCP is a stream based connection oriented protocol. Here , a definite logical connection is setup between the client and server prior to the exchange of data. TCP states are maintained for communication. TCP uses stream socket(SOCK_STREAM). This provides a connection-oriented, sequenced flow of data with mechanisms for creating/releasing connection and for detecting errors.
In header of TCP and UDP , only port number will be present. It is the responsibility of the IP header to add the necessary destination IP address for routing. However the source ip address shall be obtained from the source host.
In both TCP server and UDP server, once the socket is created using the particular socket type, it is bound to the port using bind system call. This is done so that the client can address to this port of the server. The procedure of binding is not necessary in either the TCP client or UDP client, because it does not matter in whatever port the client communicates. Till this point, it is same for both TCP server and UDP server.
In the case of TCP server, the listen system call will be invoked to listen for connections on the particular socket. The listen indicates the limit on the number of connections that can be queued up and that the server process is ready to accept any new incoming connections on the socket.
The server calls the accept system call which is a blocking call that waits for incoming connections.
Once a new connection comes, a new socket descriptor is created and the connection gets established between the server and client. However, as server, it has to keep listening for new connections.
This is where the multiprocessing begins to get into picture. Since connections can be triggered by any client at any time by using connect system call, we need to have the accept call(blocking call)for the new connections. Further, to have these new connections processed in parallel, one option is to fork and create a child process which will in-turn take care of further processing of the new connections while the parent process waits for new connections on accept call. This is how individual client connections are identified and processed/handled in the server. In simple terms, in concurrent server architecture, there will be one listening socket and multiple connected socket end points at the server at any time.
Note : The connect system call from client triggers the TCP SYN from client for starting the 3 way handshake procedure. At this point of time, the client socket state is SYN_SENT and once the SYN reaches the server, the server socket state is SYN_RCVD. Then the server responds with SYN_ACK to client for which client will respond with ACK and this establishes the connection between client and server.
In the case of UDP server, there is no need for this initial connection establishment as it is a message based connection-less protocol using datagram socket(SOCK_DGRAM). So, UDP client will not need the connect call for connection establishment which turn strips off the need for listen and accept calls in the UDP server. So, the UDP client server architecture shall be a iterative architecture where every client request/response shall be sequentially processed as iterative server architecture are applicable for services like UDP(light-weight) that consume less processing time. Also note that since the time consumed for processing is less, iterative servers will be state less.
In the case of requirement of multiple threads listening on same socket, you can also go for select or epoll system call based software architecture, where every thread will be waiting on select call but in this scenario, it may not be of great help. An alternate design that can meet your requirement can be a combination of separate thread for receive,parse & dispatch that will direct multiple processing threads.
That's why you have a port number.
If your server is listening on say, port 8080, no other process will be allowed to bind a socket to that port.
All clients that want to connect to this server will need send their data to port 8080 on the server's IP.
The UDP datagram has a source IP and port - you know where it came from, so you can route it appropriately.
You may well need a dedicated thread to send to the port, maintain state and later match up replies with requests so that the originating thread can be signaled that it's own reply is available. This could be done by providing a callback in the request struct that the rx thread signals when the matching reply arrives. The callback could signal a synchro object that the originating thread is waiting on.
I have both TCP client and TCP server run on RHEL 5.3 on different machines.
I'm killing server and FIN is sent to the client. ACK is sent back by client's OS back immediately.
Client discovers the close (by read() returning zero) and perfroms close only after 90 sec.
At this stage I verified netstat on both sides and it's as expected (FIN_WAIT_2 on server and CLOSE_WAIT on client).
Due client close() after 90 sec, client's OS sends FIN to server, but in response we receive RST from server and not ACK as expected.
I also saw several times that due to "delayed" close(), client's OS sent RST instead of FIN.
Please note, that in both cases there's no pending reading packets on both sides and SO_LINGER option is not activated.
Any ideas?
The RST indicates that some "data" was lost. In this case, the "data" is the information that the client side closed the socket cleanly - the FIN from the client was not reported to the server side application (because it had been killed).
In other words, the RST tells the client that the server never saw end-of-stream from the client.