From one thread I got the following code:
int next_stuff(char **code){
...
len=read(file_desc,buffer+end_len,packet_size-end_len);
if(len<=0)
{
if(len==-1 && errno==EAGAIN) return(0);
else return(-1);
}
...
}
while (next_stuff(&buff) == 0)
{
...
}
On the other thread I'd like to finish that socket and exit this operation, but only doing a
close(file_desc);
does not cause read to return nonblocked. Am I missing something?
EDIT:
shutdown does not work as well. And I am trying that on Linux 2.6.23
shutdown(fd, SHUT_RD);
$ man -s 2 shutdown
NAME
shutdown -- shut down part of a full-duplex connection
SYNOPSIS
#include <sys/socket.h>
int shutdown(int socket, int how);
DESCRIPTION
The shutdown() call causes all or part of a full-duplex connection on the socket
associated with socket to be shut down.
If how is SHUT_RD, further receives will be disallowed. If how is
SHUT_WR, further sends will be
disallowed. If how is SHUT_RDWR, further sends and receives will be disallowed.
RETURN VALUES
The shutdown() function returns the value 0 if successful; otherwise the value -1 is
returned and the global variable errno is set to indicate the error.
In general, if you do not want blocking socket calls, you would use select() to see if the socket is ready to read, write, or is in the error state. In addition, pass a timeout value to select() so that this call isn't blocking forever. After the select() call returns, you can see if the application wants to quit and if so do the "right" thing (that's for you to decide).
If your read() call is nonblocking, it should return fairly fast, as all it will be doing is inserting memory.
To prevent doing any damage, you would use a mutex around your call to read() and close() such that they cant both run at the same time.
If your socket is blocking, i think you should make it nonblocking.
The other answers about non-blocking sockets are good, and I recommend recoding to use that approach.
As a direct answer to your question, though, try calling shutdown() and see if that will break the other thread out of read. I'm afraid close() is just decrementing the usage count, while shutdown() will actively tear down the socket.
I don't think you've provided enough information to answer this question, yet. For example if the socket that you've opened is UDP, then a close on the sending side will have no effect on the receiving side. If it is TCP, then something else is broken. I suggest that if you are really dealing with sockets, you use recv or recvfrom instead of read.
In the case of TCP, your read will return 0 bytes, an indication that the other side has closed the connection.
If you are really doing this between two threads instead of two processes, a pipe may be more appropriate. That's not to say that a pipe could not also be used between two separate processes, it just takes a bit more set up in that case.
Related
Two processes are communicating via sockets - Process A and Process B.
Process B is using select() call to check when the socket is the ready for I/O.
Process A is suddenly killed. What will happen to the B side socket. Will B side socket automatically detect that A's socket is no longer available and select() will return -1 with EABDF. OR select() call will remain blocked forever.
Select will unlock and either an error case or a read case will be returned.
select() returns and says that the socket is readable. When you read the socket, you will get -1 (and the corresponding error in errno) or 0 (EOF).
The tcp socket will remain half opened for some time if there's no heartbeat between two sides.
Finally tcp connection will time out, depends on the time out settings.
Refer to: http://en.wikipedia.org/wiki/Half-open_connection
I have a multithreaded server written in C, with each client thread looking something like this:
ssize_t n;
struct request request;
// Main loop: receive requests from the client and send responses.
while(running && (n = recv(sockfd, &request, sizeof(request), 0)) == sizeof(request)) {
// Process request and send response.
}
if(n == -1)
perror("Error receiving request from client");
else if(n != sizeof(act))
fprintf(stderr, "Error receiving request from client: Incomplete data\n");
// Clean-up code.
At some point, a client meets a certain criteria where it must be disconnected. If the client is regularly sending requests, this is fine because it can be informed of the disconnection in the responses; However sometimes the clients take a long time to send a request, so the client threads end up blocking in the recv call, and the client does not get disconnected until the next request/response.
Is there a clean way to disconnect the client from another thread while the client thread is blocking in the recv call? I tried close(sockfd) but that causes the error Error receiving request from client: Bad file descriptor to occur, which really isn't accurate.
Alternatively, is there a better way for me to be handling errors here?
So you have at least these possibilities:
(1) pthread_kill will blow the thread out of recv with errno == EINTR and you can clean up and exit the thread on your own. Some people think this is nasty. Depends, really.
(2) Make your client socket(s) non-blocking and use select to wait on input for a specific period of time before checking if a switch used between the threads has been set to indicated they should shut down.
(3) In combo with (2) have each thread share a pipe with the master thread. Add it to the select. If it becomes readable and contains a shutdonw request, the thread shuts itself down.
(4) Look into the pthread_cancel mechanism if none of the above (or variations thereof) do not meet your needs.
Shutdown the socket for input from another thread. That will cause the reading thread to receive an EOS, which should cause it to close the socket and terminate if it is correctly written.
To interrupt the thread, make the socket non-blocking (set O_NONBLOCK using fcntl) and then signal the thread with pthread_kill. This way, recv will fail with either EINTR if it was sleeping, or EAGAIN or EWOULDBLOCK if it wasn’t (also maybe if SA_RESTART is in effect, didn’t check). Note that the socket doesn’t need to, and actually should not, be non-blocking before that. (And of course the signal needs to be handled; empty handler is sufficient).
To be sure to catch the stop-signal but not anything else, use a flag; there are things that may go wrong. For example, recv may fail with EINTR on some spurious signal. Or it may succeed if there was some data available, effectively ignoring the stop request.
And what not to do:
Don’t use pthread_kill alone or with any plain check. It may arrive right before issuing the recv syscall, too early to interrupt it but after all the checks.
Don’t close the socket. That may not even work, and as #R.. pointer out, is dangerous as the socket file descriptor may be reused between close and recv (unless you’re sure nothing opens file descriptors).
This is a question similar to Proper way to close a blocking UDP socket. I have a thread in C which is reading from a UDP socket. The read is blocking. I would like to know if it is possible to be able to exit the thread, without relying on the recv() returning? For example can I close the socket from another thread and safely expect the socket read thread to exit? Didn't see any high voted answer on that thread, thats why I am asking it again.
This really depends on what system you're running under. For example, if you're running under a POSIX-compliant system and your thread is cancelable, the recv() call will be interrupted when you cancel the thread since it's a cancel point.
If you're using an older socket implementation, you could set a signal handler for your thread for something like SIGUSR1 and hope nobody else wanted it and signal, since recv() will interrupt on a signal. Your best option is not to block, if at all possible.
I don't think closing a socket involved in a blocking operation is a safe guaranteed way of terminating the operation. For instance, kernel.org warns darkly:
It is probably unwise to close file descriptors while they may be in
use by system calls in other threads in the same process. Since a
file descriptor may be reused, there are some obscure race conditions
that may cause unintended side effects.
Instead you could use a signal and make recv fail with EINTR
(make sure SA_RESTART is not enabled). You can send a signal to a
specific thread with pthread_kill
You could enable SO_RCVTIMEO on the socket before starting the recv
call
Personally I usually try to stay clear of all the signal nastiness but it's a viable option.
You've got a couple of options for that. A signal will interrupt the read operation, so all you need to do is make sure a signal goes off. The recv operation should fail with error number EINTR.
The simplest option is to set up a timer to interrupt your own process after some timeout e.g. 30 seconds:
itimerval timer
timeval time;
time.tv_sec = 30;
time.tv_usec = 0;
timer.it_value = time;
if( setitimer( ITIMER_REAL, &timer, NULL ) != 0 )
printf( "failed to start timer\n" );
You'll get a SIGALRM after the specified time, which will interrupt your blocking operation, and give you the chance to repeat the operation or quit.
You cannot deallocate a shared resource while another thread is or might be using it. In practice, you will find that you cannot even write code to do what you suggest.
Think about it. When you go to call close, how can you possibly know that the other thread is actually blocked in recv? What if it's about to call recv, but then another thread calls socket and gets the descriptor you just closed? Now, not only will that thread not detect any error, but it will be calling recv on the wrong socket!
There is probably a good way to solve your outer problem, the reason you need to exit from a blocking UDP socket read. There are also several ugly hacks available. The basic approach is to make the socket non-blocking and instead of making a blocking UDP socket read, fake a blocking read with select or poll. You can then abort this loop several ways:
One way is to have select time out and check an 'abort' flag when select returns.
Another way is to also select on the read end of a pipe. Send a single byte to the pipe to abort the select.
If posix complient system, you can try to monitor your thread:
pthread_create with a function that makes your recv and pthread_cond_signal just after, then returns.
The calling thread makes a pthread_cond_timedwait with the desired timeout and terminates the called thread if timed_out.
I am using c .
I have fd1 as a file descriptor, can I call like this twice?
main () {
....
shutdown(fd1, SHUT_WR);
....
shutdown(fd1, SHUT_WR);
....
}
I personally think it works because fd1 has not been really free yet. Just want somebody to confirm.
You should check the return value of the second call - shutdown(2) probably returns -1 - and check the value of errno(3).
On my version of Linux and glibc, I can call shutdown multiple times on the same open socket. It will happily return 0 until the socket is actually torn down in that direction and will then return -1 with errno == ENOTCONN. It will not return EBADF until you close the FD and then you shouldn't still be using that FD anyway.
This fact is actually pretty useful, since you can call shutdown in a loop in order to detect that the connection has been torn down one way or another. epolling for errors on the socket appears to wake it up at the right time.
Calling shutdown simply initiates a TCP level shutdown sequence. The socket descriptor is never released for reuse until you call close on it.
You can call shutdown as often as you like, though it's likely that subsequent calls will result in an error.
Call close when you are done with the socket.
You can call it once to shutdown the output and again to shutdown the input, or vice versa. Calling it twice to shutdown the output certainly won't send two FINs, whatever else it may do. Calling it twice to shutdown the input can't do anything twice either. So neither of those can possibly have any actual point.
NB You can't call it on a file descriptor at all. You can call it on a socket descriptor.
A situation I have under Windows XP (SP3) has been driving me nuts, and I'm reaching the end of my tether, so maybe someone can provide some inspiration.
I have a C++ networking program (non-GUI). This program is built to compile and run under Windows, MacOS/X, and Linux, so it uses select() and non-blocking I/O as the basis for its event loop.
In addition to its networking duties, this program needs to read text commands from stdin, and exit gracefully when stdin is closed. Under Linux and MacOS/X, that's easy enough -- I just include STDIN_FILENO in my read fd_set to select(), and select() returns when stdin is closed. I check to see that FD_ISSET(STDIN_FILENO, &readSet) is true, try to read some data from stdin, recv() returns 0/EOF, and so I exit the process.
Under Windows, on the other hand, you can't select on STDIN_FILE_HANDLE, because it's not a real socket. You can't do non-blocking reads on STDIN_FILE_HANDLE, either. That means there is no way to read stdin from the main thread, since ReadFile() might block indefinitely, causing the main thread to stop serving its network function.
No problem, says I, I'll just spawn a thread to handle stdin for me. This thread will run in an infinite loop, blocking in ReadFile(stdinHandle), and whenever ReadFile() returns data, the stdin-thread will write that data to a TCP socket. That socket's connection's other end will be select()'d on by the main thread, so the main thread will see the stdin data coming in over the connection, and handle "stdin" the same way it would under any other OS. And if ReadFile() returns false to indicate that stdin has closed, the stdin-thread just closes its end of the socket-pair so that the main thread will be notified via select(), as described above.
Of course, Windows doesn't have a nice socketpair() function, so I had to roll my own using listen(), connect(), and accept() (as seen in the CreateConnectedSocketPair() function here. But I did that, and it seems to work, in general.
The problem is that it doesn't work 100%. In particular, if stdin is closed within a few hundred milliseconds of when the program starts up, about half the time the main thread doesn't get any notification that the stdin-end of the socket-pair has been closed. What I mean by that is, I can see (by my printf()-debugging) that the stdin-thread has called closesocket() on its socket, and I can see that the main thread is select()-ing on the associated socket (i.e. the other end of the socket-pair), but select() never returns as it should... and if it does return, due to some other socket selecting ready-for-whatever, FD_ISSET(main_thread_socket_for_socket_pair, &readSet) returns 0, as if the connection wasn't closed.
At this point, the only hypothesis I have is that there is a bug in Windows' select() implementation that causes the main thread's select() not to notice that the other end of the socket-pair has closed by the stdin-thread. Is there another explanation? (Note that this problem has been reported under Windows 7 as well, although I haven't looked at it personally on that platform)
Just for the record, this problem turned out to be a different issue entirely, unrelated to threading, Windows, or stdin. The actual problem was an inter-process deadlock, where the parent process was blocked, waiting for the child processes to quit, but sometimes the child processes would be simultaneously blocked, waiting on the parent to supply them with some data, and so nothing would move forward.
Apologies to all for wasting your time on a red herring; if there's a standard way to close this case as unwarranted, let me know and I'll do it.
-Jeremy
Is it possible you have a race condition? Eg. Do you ensure that the CreateConnectedSocketPair() function has definitely returned before the stdin-thread has a chance to try closing its socket?
I am studying in your code. In the CreateConnectedSocketPair(), socket1 is used for listen(), and newfd is used for send/recv data. So, why does "socket1 = newfd"? How to close the listenfd then?
Not a solution, but as a workaround, couldn't you send some magic "stdin has closed" message across the TCP socket and have your receiving end disconnect its socket when it sees that and run whatever 'stdin has closed' handler?
Honestly your code is too long and I don't have time right now to spend on it.
Most likely the problem is in some cases closing the socket doesn't cause a graceful (FIN) shutdown.
Checking for exceptions returning from your select may catch the remainder of cases. There is also the (slim) possibility that no notification is actually being sent to the socket that the other end has closed. In that case, there is no way other than timeouts or "keep alive"/ping messages between the endpoints to know that the socket has closed.
If you want to figure out exactly what is happening, break out wireshark and look for FINs and RSTs (and the absence of anything). If you see the proper FIN sequence going across when your socket is closed, then the problem must be in your code. if you see RST, it may be caught by exceptions, and if you don't see anything you'll need to devise a way in your protocol to 'ping' each side of the connection to make sure they are still alive, or set a sufficiently short timeout for more data.
Rather than chasing perceived bugs in select(), I'm going to address your original fallacy that drove you away from simple, reliable, single-threaded design.
You said "You can't do non-blocking reads on STDIN_FILE_HANDLE, either. That means there is no way to read stdin from the main thread, since ReadFile() might block indefinitely" but this simply isn't the whole story. Look at ReadConsoleInput, WSAEventSelect, and WaitForMultipleObjects. The stdin handle will be signalled only when there is input and ReadConsoleInput will return immediately (pretty much the same idea behind select() in Unix).
Or, use ReadFileEx and WaitForMultipleObjectsEx to have the console reads fire off an APC (which isn't all that asynchronous, it runs on the main thread and only during WaitForMultipleObjectsEx or another explicit wait function).
If you want to stick with using a second thread to get async I/O on stdin, then you might try closing the handle being passed to select instead of doing a socket shutdown (via closesocket on the other end). In my experience select() tends to return really quickly when one of the fds it is waiting on gets closed.
Or, maybe your problem is the other way around. The select docs say "For connection-oriented sockets, readability can also indicate that a request to close the socket has been received from the peer". Typically you'd send that "request to close the socket" by calling shutdown(), not closesocket().