I have a AF_INET/SOCK_STREAM server written in C running on Android/Linux which looks more ore less like this:
...
for (;;) {
client = accept(...);
read(client, &message, sizeof(message));
response = process(&message);
write(client, response, sizeof(*response));
close(client);
}
As far as I know, the call to close should not terminate the connection to the client immediately, but it apparently does: The client reports "Connection Reset by Peer" before it has had a chance to read the server's response.
If I insert a delay between write() and close() the client can read the response as expected.
I got a hint that it might have to do with the SO_LINGER option, but I checked it's value and both members of struct linger (l_onoff, l_linger) have a value of zero.
Any ideas?
Stevens describes a configuration in which this can happen, but it depends on the client sending more data after the server has called close() (after the client should “know” that the connection is being closed). UNP 2nd ed s5.12.
Try tcpdumping the conversation to find out what’s really going on. If there's any possibility that a “clever” gateway (e.g. NAT) is between the two endpoints, tcpdump both ends and look for discrepancies.
Connection gets reset when you call close() on connection with data being sent. Specially for this case the sequence of shutdown() with SHUT_WR flag and then blocking read() is used.
Shutting down the writing end of the socket sends FIN and returns immediately, and the said read() blocks and returns 0 as soon as your peer replies with FIN in due turn. Basically, this is what you need in place of the delay between write() and close() you are talking about.
You do not need do anything with linger options in this case, leave it all to default.
SO_LINGER should be set (i.e. set to 1 not 0) if you want queued data to be sent before a close is effected.
SO_LINGER
Lingers on a close() if data is present. This option controls the
action taken when unsent messages
queue on a socket and close() is
performed. If SO_LINGER is set, the
system shall block the calling thread
during close() until it can transmit
the data or until the time expires. If
SO_LINGER is not specified, and
close() is issued, the system handles
the call in a way that allows the
calling thread to continue as quickly
as possible. This option takes a
linger structure, as defined in the
header, to specify the
state of the option and linger
interval.
Related
Here is the succinct code:
int main(){
/* here ellipsis socket connect malloc etc. */
send(socket,buffer, 1024*1024*1024,0);
return 1;
}
Question: If send() is blocked, no matter how large the buffer is, I think it will send all data before the main function returned. But actually, if the data large enough, the other socket end only receive a part of data which the send() function send. However, if I add this code before 'return 1;'
while(1){}
the other socket end receive intact data.
Does the send() is non_blocked?
or something wrong with send() function?
thanks advance.
When blocking socket is used, send() function blocks until last data is delivered to queue of local TCP-stack.
So send() may return when part of the data is still queued in local TCP-stack.
Because your process exits right after send() call, there can be undelivered data in local TCP stack during exit.
TCP stack may continue the data transfer after exit, if linger is enabled. Or TCP stack may reset the connection without any attempt to transfer undelivered data to the peer, if linger is disabled.
If you close the TCP connection gracefully when linger is enabled, then TCP-stack should (try to) deliver queued data to the peer.
Close the connection gracefully by adding close() call.
And make sure that SO_LINGER is enabled with reasonable timeout:
send(socket,buffer, 1024*1024*1024,0);
const struct linger linger_val = { 1, 600 };
setsockopt(socket, SOL_SOCKET, SO_LINGER, &linger_val, sizeof(linger_val));
close(socket);
return 1;
Usually there is no need to change SO_LINGER. More information about SO_LINGER in man page of socket(7):
SO_LINGER
When enabled, a close(2) or shutdown(2) will not return until
all queued messages for the socket have been successfully sent
or the linger timeout has been reached. Otherwise, the call
returns immediately and the closing is done in the background.
When the socket is closed as part of exit(2), it always
lingers in the background.
Whit 0 as flag, send() is like write() function:
https://linux.die.net/man/2/send
https://linux.die.net/man/2/write
and for write():
The number of bytes written may be less than count if, for example, there is insufficient space on the underlying physical medium, or the RLIMIT_FSIZE resource limit is encountered (see setrlimit(2)), or the call was interrupted by a signal handler after having written less than count bytes. (See also pipe(7).).
Check also this answer:
Blocking sockets: when, exactly, does "send()" return?
Hope this help
send() is blocking call, but it is blocked till all the data is pushed to sendbuffer. You can modify the program to exit when all the data is send from the socket. This is possible by reducing the sendbuffer size. you can use setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)send_buffer, send_buffer_sizeof);
I have a client application communicating with a QEMU process through a QMP Unix domain socket. Sometimes after the client calls close() on the socket connection, 'netstat -ap unix' still shows it in CONNECTED state. I do check the return value of the close() call and it returns successfully with a value of 0, but the connection still seems to be lingering.
Since QMP doesn't really support multiple connections on its socket, all the subsequent calls to connect to the socket fail since they wait indefinitely for the lingering connection to be closed.
Is there a way to make sure from the code that the socket is really closed, and is there a way to force the socket to close?
It could be that the file descriptor has been duped, forked, or leaked.
Call shutdown(sock, SHUT_RDWR) on it to close the connection for sure before closeing.
Have you tried closing the socket from the other end? It's asynchronous, but it gives both sides a chance to ensure socket closure.
You can send a close command through to the listener on the other end and have it recycle the socket. When the socket gets closed, you should end up getting a SIGPIPE. Catch the SIGPIPE and close your end of the socket. If you end up with an EPIPE doing that, then ignore it. That just means you were already notified of the socket closure.
You could just try SO_LINGER via setsockopt(2) option with a timeout of 0. This way, when you close the socket is forcibly closed, sending a RST instead of going into the FIN/ACK closing behavior.
The purpose of the SO_LINGER option is to control how the socket is shut down when the function close(2) is called. This option applies only to connection-oriented protocols such as TCP.
The default behavior of the kernel is to allow the close(2) function to return immediately to the caller. Any unsent TCP/IP data will be transmitted and delivered if possible, but no guarantee is made. Because the close(2) call returns control immediately to the caller, the application has no way of knowing whether the last bit of data was actually delivered.
The SO_LINGER option can be enabled on the socket, to cause the application to block in the close(2) call until all final data is delivered to the remote end. Furthermore, this assures the caller that both ends have acknowledged a normal socket shutdown. Failing this, the indicated option timeout occurs and an error is returned to the calling application.
One final scenario can be applied, by use of different SO_LINGER option values. If the calling application wants to abort communications immediately, appropriate values can be set in the linger structure. Then, a call to close(2) will initiate an abort of the communication link, discarding all pending data and immediately close the socket.
As the Title already says im looking for a way, to get notified when a client closes his Session unnormal.
I'm using the freeBSD OS.
The server is running with Xamount threads (depending on CPUcore amount). So I'm not forking, and there isn't a own process for each client.
That's why sending an deathpackage all time_t seconds, to recive a SIGPIPE isn't an option for me.
But i need to remove left clients from the kqueue, because otherwise after too many accept()'s my code will obviously run into memory troubles.
Is there a way, I can check without high performance loose per client, they are connected or not?
Or any event-notification, that would trigger if this happens? Or maybe is there a way of letting a programm send any signal to a port, even in abnormal termination case, before the Client process will exite?
Edit: that answer misses the question, because it's not about using kqueue. But if someone else finds the question by the title, it may be helpful anyway ...
I've often seen the following behaviour: if a client dies, and the server does a select() on the client's socket descriptor, select() returns with return code > 0 and FD_ISSET( fd ) will be true for that descriptor. But when you now try to read form the socket, read() (or recv()) return ERROR.
For a 'normal' connection using that to detect a client's death works fine for us, but there seems to be a different behaviour when the socket connection is tunneled but we haven't yet managed to figure that out completely.
According to the kqueue man page, kevent() should create an event when the socket has shutdown. From the description of th filter EVFILT_READ:
EVFILT_READ
Takes a descriptor as the identifier, and returns whenever there is data available to read. The behavior of the filter is slightly different depending on the descriptor type.
Sockets
Sockets which have previously been passed to listen() return when there is an incoming connection pending. data contains the size of the listen backlog.
Other socket descriptors return when there is data to be read, subject to the SO_RCVLOWAT value of the socket buffer. This may be overridden with a per-filter low water mark at the time the filter is added by setting the NOTE_LOWAT flag in fflags, and specifying the new low water mark in data. On return, data contains the number of bytes of protocol data available to read.
If the read direction of the socket has shutdown, then the filter also sets EV_EOF in flags, and returns the socket error (if any) in fflags. It is possible for EOF to be returned (indicating the connection is gone) while there is still data pending in the socket
buffer.
when a tcp client establishes a tcp connection with my TCP server, after it sends several packets, I want to make a hard/abortive close on this TCP connection, how to do it in linux C?
the hard/abortive close here means the tcp server will send a RST immediately to the client. No FIN/ACK at all.
thanks!
From Socket man page
SO_LINGER
Sets or gets the SO_LINGER option. The argument is a linger structure.
struct linger {
int l_onoff; /* linger active */
int l_linger; /* how many seconds to linger for */
};
When enabled, a close(2) or shutdown(2) will not return until all queued messages
for the socket have been successfully sent or the linger timeout has been reached. Otherwise, the call returns immediately and the closing is done in the background. When the socket is closed as part of exit(2), it always lingers in the background.
Another settings :
The effect of an setsockopt(..., SO_LINGER,...) depends on what the values in the linger structure (the third parameter passed to setsockopt()) are:
Case 1: linger->l_onoff is zero (linger->l_linger has no meaning):
This is the default.
On close(), the underlying stack attempts to gracefully shutdown the connection after ensuring all unsent data is sent. In the case of connection-oriented protocols such as TCP, the stack also ensures that sent data is acknowledged by the peer. The stack will perform the above-mentioned graceful shutdown in the background (after the call to close() returns), regardless of whether the socket is blocking or non-blocking.
Case 2: linger->l_onoff is non-zero and linger->l_linger is zero:
A close() returns immediately. The underlying stack discards any unsent data, and, in the case of connection-oriented protocols such as TCP, sends a RST (reset) to the peer (this is termed a hard or abortive close). All subsequent attempts by the peer's application to read()/recv() data will result in an ECONNRESET.
Case 3: linger->l_onoff is non-zero and linger->l_linger is non-zero:
A close() will either block (if a blocking socket) or fail with EWOULDBLOCK (if non-blocking) until a graceful shutdown completes or the time specified in linger->l_linger elapses (time-out). Upon time-out the stack behaves as in case 2 above.
You should set the SO_LINGER socket option with timeout 0 and call close(). If there's any pending data to send it is lost and an RST is sent instead of FIN.
Reference: http://alas.matf.bg.ac.rs/manuals/lspe/snode=105.html
In C, I understood that if we close a socket, it means the socket will be destroyed and can be re-used later.
How about shutdown? The description said it closes half of a duplex connection to that socket. But will that socket be destroyed like close system call?
This is explained in Beej's networking guide. shutdown is a flexible way to block communication in one or both directions. When the second parameter is SHUT_RDWR, it will block both sending and receiving (like close). However, close is the way to actually destroy a socket.
With shutdown, you will still be able to receive pending data the peer already sent (thanks to Joey Adams for noting this).
None of the existing answers tell people how shutdown and close works at the TCP protocol level, so it is worth to add this.
A standard TCP connection gets terminated by 4-way finalization:
Once a participant has no more data to send, it sends a FIN packet to the other
The other party returns an ACK for the FIN.
When the other party also finished data transfer, it sends another FIN packet
The initial participant returns an ACK and finalizes transfer.
However, there is another "emergent" way to close a TCP connection:
A participant sends an RST packet and abandons the connection
The other side receives an RST and then abandon the connection as well
In my test with Wireshark, with default socket options, shutdown sends a FIN packet to the other end but it is all it does. Until the other party send you the FIN packet you are still able to receive data. Once this happened, your Receive will get an 0 size result. So if you are the first one to shut down "send", you should close the socket once you finished receiving data.
On the other hand, if you call close whilst the connection is still active (the other side is still active and you may have unsent data in the system buffer as well), an RST packet will be sent to the other side. This is good for errors. For example, if you think the other party provided wrong data or it refused to provide data (DOS attack?), you can close the socket straight away.
My opinion of rules would be:
Consider shutdown before close when possible
If you finished receiving (0 size data received) before you decided to shutdown, close the connection after the last send (if any) finished.
If you want to close the connection normally, shutdown the connection (with SHUT_WR, and if you don't care about receiving data after this point, with SHUT_RD as well), and wait until you receive a 0 size data, and then close the socket.
In any case, if any other error occurred (timeout for example), simply close the socket.
Ideal implementations for SHUT_RD and SHUT_WR
The following haven't been tested, trust at your own risk. However, I believe this is a reasonable and practical way of doing things.
If the TCP stack receives a shutdown with SHUT_RD only, it shall mark this connection as no more data expected. Any pending and subsequent read requests (regardless whichever thread they are in) will then returned with zero sized result. However, the connection is still active and usable -- you can still receive OOB data, for example. Also, the OS will drop any data it receives for this connection. But that is all, no packages will be sent to the other side.
If the TCP stack receives a shutdown with SHUT_WR only, it shall mark this connection as no more data can be sent. All pending write requests will be finished, but subsequent write requests will fail. Furthermore, a FIN packet will be sent to another side to inform them we don't have more data to send.
There are some limitations with close() that can be avoided if one uses shutdown() instead.
close() will terminate both directions on a TCP connection. Sometimes you want to tell the other endpoint that you are finished with sending data, but still want to receive data.
close() decrements the descriptors reference count (maintained in file table entry and counts number of descriptors currently open that are referring to a file/socket) and does not close the socket/file if the descriptor is not 0. This means that if you are forking, the cleanup happens only after reference count drops to 0. With shutdown() one can initiate normal TCP close sequence ignoring the reference count.
Parameters are as follows:
int shutdown(int s, int how); // s is socket descriptor
int how can be:
SHUT_RD or 0
Further receives are disallowed
SHUT_WR or 1
Further sends are disallowed
SHUT_RDWR or 2
Further sends and receives are disallowed
This may be platform specific, I somehow doubt it, but anyway, the best explanation I've seen is here on this msdn page where they explain about shutdown, linger options, socket closure and general connection termination sequences.
In summary, use shutdown to send a shutdown sequence at the TCP level and use close to free up the resources used by the socket data structures in your process. If you haven't issued an explicit shutdown sequence by the time you call close then one is initiated for you.
I've also had success under linux using shutdown() from one pthread to force another pthread currently blocked in connect() to abort early.
Under other OSes (OSX at least), I found calling close() was enough to get connect() fail.
"shutdown() doesn't actually close the file descriptor—it just changes its usability. To free a socket descriptor, you need to use close()."1
Close
When you have finished using a socket, you can simply close its file descriptor with close; If there is still data waiting to be transmitted over the connection, normally close tries to complete this transmission. You can control this behavior using the SO_LINGER socket option to specify a timeout period; see Socket Options.
ShutDown
You can also shut down only reception or transmission on a connection by calling shutdown.
The shutdown function shuts down the connection of socket. Its argument how specifies what action to perform:
0
Stop receiving data for this socket. If further data arrives, reject it.
1
Stop trying to transmit data from this socket. Discard any data waiting to be sent. Stop looking for acknowledgement of data already sent; don’t retransmit it if it is lost.
2
Stop both reception and transmission.
The return value is 0 on success and -1 on failure.
in my test.
close will send fin packet and destroy fd immediately when socket is not shared with other processes
shutdown SHUT_RD, process can still recv data from the socket, but recv will return 0 if TCP buffer is empty.After peer send more data, recv will return data again.
shutdown SHUT_WR will send fin packet to indicate the Further sends are disallowed. the peer can recv data but it will recv 0 if its TCP buffer is empty
shutdown SHUT_RDWR (equal to use both SHUT_RD and SHUT_WR) will send rst packet if peer send more data.
linux: shutdown() causes listener thread select() to awake and produce error. shutdown(); close(); will lead to endless wait.
winsock: vice versa - shutdown() has no effect, while close() is successfully catched.