I am successfully reading from a pipe from another thread, and printing the output (in an ncurses window as it happens).
I need to do this one character at a time, for various reasons, and I'm using a select() on the FD for the read end of the pipe, along with a few other FDs (like stdin).
My idea is to attempt to read from the pipe only when it is imminently ready to be read, in preference to handling any input. This seems to be working - at least to start. select() sets up the fd_set and if FD_ISSET I do my read() of 1 byte from the FD. But select() says yes one time too many, and the read() blocks.
So my question is this - why would select() report that a fd is ready for reading, if a subsequent read() blocks?
(approximately) This same code worked fine when the other end of the pipe was connected to a forked process, if that helps.
I can post the code on request, but it's bog standard. Set up a fd_set, copy it, select the copy, if the FD is set call a function which reads a byte from the same FD... otherwise revert the fd_set copy
EDIT: by request, here's the code:
Setting up my fd_set:
fd_set fds;
FD_ZERO(&fds);
FD_SET(interp_output[0], &fds);
FD_SET(STDIN_FILENO, &fds);
struct timeval timeout, tvcopy; timeout.tv_sec=1;
int maxfd=interp_output[0]+1; //always >stdin+1
fd_set read_fds;
FD_COPY(&fds, &read_fds);
In a loop:
if (select(maxfd, &read_fds, NULL, NULL, &timeout)==-1) {perror("couldn't select"); return;}
if (FD_ISSET(interp_output[0], &read_fds)) {
handle_interp_out();
} else if (FD_ISSET(STDIN_FILENO, &read_fds)) {
//waddstr(cmdwin, "stdin!"); wrefresh(cmdwin);
handle_input();
}
FDCOPY(&fds, &read_fds);
handle_interp_out():
void handle_interp_out() {
int ch;
read(interp_output[0], &ch, 1);
if (ch>0) {
if (ch=='\n') { if (cmd_curline>=cmdheight) cmdscroll(); wmove(cmdwin, ++cmd_curline, 1); }
else waddch(cmdwin, ch);
wrefresh(cmdwin);
}
}
EDIT 2: The write code is just a fprintf on a FILE* opened with fdopen(interp_output[1], "w") - this is in a different thread. All I'm getting to is my "prompt> " - it prints all that properly, but does one more iteration that it shouldn't. I've turned off buffering, which was giving me other problems.
EDIT 3: This has become a problem with my invocation of select(). It appears that, right away, it's returning -1 and errno's being set to 'invalid argument'. The read() doesn't know that and just keeps going. What could be wrong with my select()? I've updated the code and changed the title to more accurately reflect the problem...
EDIT 4: So now I'm thoroughly confused. The timeout value of .tv_sec=1 was no good, somehow. By getting rid of it, the code works just fine. If anybody has any theories, I'm all ears. I would just leave it at NULL, except this thread needs to periodically do updates.
To absolutely guarantee that the read will never block, you must set O_NONBLOCK on the fd.
Your select error is almost certainly caused because you aren't setting the entire time struct. You're only setting the seconds. The other field will contain garbage data picked up from the stack.
Use struct initialization. That will guarantee the other fields are set to 0.
It would look like this:
struct timeval timeout = {1, 0};
Also, in your select loop you should be aware that Linux will write the time remaining into the timeout value. That means that it will not be 1 second the next time through the loop unless you reset the value to 1 second.
According to the manpage:
On error, -1 is returned, and errno is set appropriately; the sets and timeout become undefined, so do not rely on their contents after an error.
You are not checking the return code from select().
The most likely explanation is that select() is being interrupted (errno=EINTR) and so returning an error, and the FD bit is still set in the "read" fd_set giving you the behaviour you are seeing.
Incidentally it is a very bad idea to name variables after standard/system/common functions. "read_fds" would be a MUCH better name than "read".
It's correct. See the select() manpage in Linux for example: http://linux.die.net/man/2/select
"Under Linux, select() may report a socket file descriptor as "ready for reading", while nevertheless a subsequent read blocks"
The only solution is to use NON-BLOCKING socket.
The answer is "it wouldn't". The behaviour you describe should never happen. Something else is going wrong.
I have been in a similar situation many times, and usually it turned out that there was a silly typo or cut and paste error elsewhere that led to behaviour that I mis-diagnosed.
If you post your code maybe we can help better - post the write code as well as the read please.
Also consider using asynchronous IO, even if only for debug purposes. If what you suspect is happening really is happening, then read() will return EWOULDBLOCK.
Also, you say you are copying the fd_set. How? Can you post your code for that?
Related
With this operation I expected to get an error because I'm reading from nothing, but in fact the program seems to keep attempting to read until someone writes to it. If there are no writes it will be stuck in an indefinite loop trying to read and will not proceed.
What exactly happens behind the scene here, do the function kept looping, or is it waiting for a signal, or is something else going on? Is it still taking CPU resources?
Also, is it possible to make the program return an error code/print out something when trying to read without any writes? I don't really need to do it, just wondering if it's possible.
This is normal behavior. If nothing is available to read, the reading process will block until there is. It will not consume CPU time while blocking; the OS will put it to sleep until another process writes to the pipe.
Keep in mind that pipes were designed to be somewhat transparent; a simple filter-type program should not have to care whether the input is a file or a pipe. If every program that wanted to be able to read from a pipe (think grep) had to include special handling to wait until the writer was ready, it would be very tedious for those programmers. This behavior means that reading from pipes doesn't require doing anything special.
If you don't want to block if no data is available, you can set the O_NONBLOCK status flag on the file descriptor, either when you open(2) it, or with fcntl(fd, F_SETFL, ...). In this case, when no data is available, read(2) will return -1 and set errno to EAGAIN or EWOULDBLOCK. This means, of course, that every time you read from the file descriptor, you have to write code to handle such a case.
You can also use select(2) or poll(2) to wait until data is available, optionally with a timeout.
It is also possible to arrange it so that a signal arriving during the blocking will cause read(2) to return -1 and set errno to EINTR. This depends on system call restarting semantics and is a little bit complicated.
I am facing a problem with FD_SET. I am accessing CAN in Linux using socket CAN approach and using recvfrom. Since it is a blocking call, I want to use select system call. My program hangs # FD_SET macro. It works fine with FD_CLR macro.
code :
FD_ZERO(&readfd);
printf("\n 1.1");
FD_CLR(s, &readfd);
printf("\n 1.2");
FD_SET(sockaddr,&readfd); //hangs here
printf("\n 1.3");
Output:
1.1
1.2
Not getting anything after that...
Even the sockaddr value is 3 (less than FD_SETSIZE = 1024);
I can apply FD_SET to socket can approach sockets??
What can be the reason?
It's unlikely that it's hanging at the FD_SET(). What's more likely is that stdout is a terminal, in which case your C library does line buffering so that each line is sent to the terminal only after the terminating newline is seen.
Try doing e.g. printf("1.3\n") instead (or just puts("1.3"), which is the same thing and simpler). You could also use a debugger.
(If stdout is a regular file, do an fflush(stdout) on it after printing "1.3" like
5gon12eder suggested in the comments. The fflush() will work regardless of whether stdout is a terminal or a regular file (which will use block buffering), so it's a nicer solution in that way. Another alternative is to use stderr, which is unbuffered by default.)
FD_CLR(s, &readfd) after FD_ZERO(&readfd) is redundant by the way. FD_ZERO() already clears the fd_set readfd.
As for the real problem (the hang):
That you're doing FD_CLR(s, &readfd) after FD_ZERO(&readfd) might indicate that you're misunderstanding something about how select(2) works. If you follow those lines with a select(FD_SETSIZE, &readfd, NULL, NULL, NULL), then the result will be that you're only ever waiting for the sockaddr file descriptor to become readable (to not block when read).
Also make sure that readfd is an fd_set and that s and sockaddr are descriptors (as returned by e.g. socket(2) and open(2)). Otherwise, things don't make sense.
I have got signaled socket for read from select(), but then no data arrived by recv call(), instead it returns -1 with errno==EAGAIN.
I can grant that no other thread touch the socket.
I think that this behavior is not correct. If an subsequent close from other side occurs, I can expect return value 0 (graceful close) or other error code from recv, but not EAGAIN, because it means by my opinion that an data will arrive in the future.
I have found some previous thread about the problem here but without solution.
This behavior happens to me on Ubuntu Linux Oneric, or other last Linux distros, then info from link posted here
That it will be fixed in kernel is not true for 3.0.0 kernel or latest 2.6.x
Does anybody have an idea why it happens and how to avoid this unwanted behavior?
Select() reporting a socket as readable does not mean that there is something to read; it implies that a read will not block. The read could return -1 or 0, but it would not block.
UPDATE:
After select returns readable: if read() returns -1, check errno.
EAGAIN/EWOULDBLOCK and EINTR are the values to be treated specially: mostly by reissuing the read(), but you might trust on the select loop returning readable the next time around.
If there are multiple threads involved, things may get more difficult.
I'm getting the same problem but with epoll. I noticed, that it happens whenever the system is reusing the FD numbers of the sockets that are already closed.
After some research, I've noticed that this behavior is caused by closing the sockets while epolling on them. Try to avoid running select on a socket while closing it - that may help.
I am using c .
I have fd1 as a file descriptor, can I call like this twice?
main () {
....
shutdown(fd1, SHUT_WR);
....
shutdown(fd1, SHUT_WR);
....
}
I personally think it works because fd1 has not been really free yet. Just want somebody to confirm.
You should check the return value of the second call - shutdown(2) probably returns -1 - and check the value of errno(3).
On my version of Linux and glibc, I can call shutdown multiple times on the same open socket. It will happily return 0 until the socket is actually torn down in that direction and will then return -1 with errno == ENOTCONN. It will not return EBADF until you close the FD and then you shouldn't still be using that FD anyway.
This fact is actually pretty useful, since you can call shutdown in a loop in order to detect that the connection has been torn down one way or another. epolling for errors on the socket appears to wake it up at the right time.
Calling shutdown simply initiates a TCP level shutdown sequence. The socket descriptor is never released for reuse until you call close on it.
You can call shutdown as often as you like, though it's likely that subsequent calls will result in an error.
Call close when you are done with the socket.
You can call it once to shutdown the output and again to shutdown the input, or vice versa. Calling it twice to shutdown the output certainly won't send two FINs, whatever else it may do. Calling it twice to shutdown the input can't do anything twice either. So neither of those can possibly have any actual point.
NB You can't call it on a file descriptor at all. You can call it on a socket descriptor.
I'm currently working on a project which involves multiple clients connected to a server and waiting for data. I'm using select and monitoring the connection for incoming data. However, the client just continues to print nothing, acting as if select has discovered incoming data. Perhaps I'm attacking this wrong?
For the first piece of data the server does send, it is displayed correctly. However, the server then disconnects and the client continues to spew blank lines.
FD_ZERO(&readnet);
FD_SET(sockfd, &readnet);
while(1){
rv = select(socketdescrip, &readnet, NULL, NULL, &timeout);
if (rv == -1) {
perror("select"); // error occurred in select()
} else if (rv == 0) {
printf("Connection timeout! No data after 10 seconds.\n");
} else {
// one or both of the descriptors have data
if (FD_ISSET(sockfd, &readnet)) {
numbytes = recv(sockfd, buf, sizeof buf, 0);
printf("Data Received\n");
buf[numbytes] = '\0';
printf("client: received '%s'\n",buf);
sleep(10);
}
}
}
I think you need to check the result of recv. If it returns zero, I believe it means the server has closed the socket.
Also (depending on the implementation), you may need to pass socketdescrip+1 to select.
If I remember correctly, you need to initialise set of fds before each call to select() because select() corrupts it.
So move FD_ZERO() and FD_SET() inside the loop, just before select().
acting as if select has discovered
incoming data. Perhaps I'm attacking
this wrong?
In addition to what was said before, I'd like to note that select()/poll() do tell you not when "data are there" but rather that next corresponding system call will not block. That's it. As was said above, recv() doesn't block and properly returns 0, what means EOF, connection was closed by the other side.
Though on most *nix systems in the case only first call of recv() would return 0, following calls would return -1. When using async I/O rigorous error checking is a must!
And personally I would strongly suggest to use poll() instead. Unlike select(), it doesn't destroy its arguments and works fine with high numbered socket descriptors.
When server closes the connection, it will send a packet taking FIN flag to client side to announce that it no longer sends data. The packet is processed by TCP/IP stack at the client side and has no data for application level. The application level is notified to trigger select because something happened on the monitored file descriptor, and recv() return 0 bytes because no data sent by server.
Is this true when talking about your code?
select(highest_file_descriptor+1, &readnet, NULL, NULL, &timeout);
In your simple example (with FD_ZERO and FD_SET moved inside the while(1) loop as qrdl said) it should look like this:
select(sockfd+1, &readnet, NULL, NULL, &timeout);
Also - please note that when recv returns 0 bytes read it means that connection was closed - no more data! Your code is also buggy - when something bad happens on recv (it returns <0 when this happens) you will have serious trouble because something like buf[-1] may lead to unpredictable results. Please handle this case properly.
While I respect the fact that you try to use the low-level BSD sockets API I must say that I find it awfully inefficient. That's why I recommend to you if possible to use ACE which is a very efficient and productive framework which has a lot of things already implemented when it comes to network programming (ACE_Reactor for example is something that makes it easier to do what you're trying to achieve here).