I am facing a problem with FD_SET. I am accessing CAN in Linux using socket CAN approach and using recvfrom. Since it is a blocking call, I want to use select system call. My program hangs # FD_SET macro. It works fine with FD_CLR macro.
code :
FD_ZERO(&readfd);
printf("\n 1.1");
FD_CLR(s, &readfd);
printf("\n 1.2");
FD_SET(sockaddr,&readfd); //hangs here
printf("\n 1.3");
Output:
1.1
1.2
Not getting anything after that...
Even the sockaddr value is 3 (less than FD_SETSIZE = 1024);
I can apply FD_SET to socket can approach sockets??
What can be the reason?
It's unlikely that it's hanging at the FD_SET(). What's more likely is that stdout is a terminal, in which case your C library does line buffering so that each line is sent to the terminal only after the terminating newline is seen.
Try doing e.g. printf("1.3\n") instead (or just puts("1.3"), which is the same thing and simpler). You could also use a debugger.
(If stdout is a regular file, do an fflush(stdout) on it after printing "1.3" like
5gon12eder suggested in the comments. The fflush() will work regardless of whether stdout is a terminal or a regular file (which will use block buffering), so it's a nicer solution in that way. Another alternative is to use stderr, which is unbuffered by default.)
FD_CLR(s, &readfd) after FD_ZERO(&readfd) is redundant by the way. FD_ZERO() already clears the fd_set readfd.
As for the real problem (the hang):
That you're doing FD_CLR(s, &readfd) after FD_ZERO(&readfd) might indicate that you're misunderstanding something about how select(2) works. If you follow those lines with a select(FD_SETSIZE, &readfd, NULL, NULL, NULL), then the result will be that you're only ever waiting for the sockaddr file descriptor to become readable (to not block when read).
Also make sure that readfd is an fd_set and that s and sockaddr are descriptors (as returned by e.g. socket(2) and open(2)). Otherwise, things don't make sense.
Related
I have a C program that receives data from another program over a (Linux) pipe. I want the program to behave differently if the pipe was closed before writing any data.
The natural way to do this is to try to read from the pipe and check if I get EOF, but that consumes some data from the pipe if there is any available, and (as far as I know) there's no way to put data "back" in a pipe.
The part of the program where I want to check if the pipe is empty is pretty far away from where I process the data, so I'd rather not have to deal with saving the data from my first read until then.
Is there any way to check if a pipe is empty (read would return EOF) without consuming any data in the case it's not empty?
Note: I do want this to block if the pipe has not been written to or closed yet.
If you used Unix domain stream sockets instead of pipes – meaning you replace your pipe(fds) calls with socketpair(AF_UNIX, SOCK_STREAM, 0, fds) –, you could use recv(fd, dummybuffer, 1, MSG_PEEK) to read/receive one byte of data, without removing it from the receive buffer.
You can combine MSG_PEEK with MSG_DONTWAIT if you don't want to block, or with MSG_WAITALL if you want to block until the entire buffer can be filled.
The differences between an Unix domain stream socket and a pipe are minimal. The stream socket is bidirectional, but you can use shutdown(fd, SHUT_WR) (or SHUT_RD) to close the "write end" (resp. "read end"), meaning if the other end tries to read from the socket, they'll get an immediate end-of-stream (read(), recv() etc. return 0). (Closing the "read end" means that when the other end tries to write to the socket, they'll get EPIPE.)
Right now, I cannot even think of a reason why a program that works with a pipe would not work with an Unix domain stream socket pair.
If you use named pipes, you do need to change mkfifo() and open() to socket(AF_UNIX, SOCK_STREAM, 0) followed by a bind() to the socket address. read(), write(), and even the higher-level standard I/O facilities work just fine on top of an Unix domain stream socket (use fdopen() to convert the socket descriptor to a FILE handle).
If you cannot modify the readers, you can create a minimal dynamic library that interposes openat() (that's what current C library uses underneath fopen()), calling original openat() for all except the socket path, say named in an environment variable, and instead creates a socket and binds to the socket path for that one. When executing the reader binaries, you just set LD_PRELOAD to point to this interposing library.
In other words, I do believe there are no real obstacles for switching from pipes to Unix domain stream sockets.
You cannot use recv() with pipes, because pipes are implemented in Linux using a special filesystem, not sockets.
No, there is no way to do what you describe. The way to determine whether you have reached the end of a non-seekable file such as a pipe is to attempt to read from it. This is not just the natural way, it is the way.
but that consumes some data from the pipe if there is
any available,
Yes.
and (as far as I know) there's no way to put data
"back" in a pipe.
That depends. If you are reading with POSIX read(), then no. If you are wrapping the the pipe end in a FILE and using stdio functions to read it then there is ungetc().
Nevertheless, this:
The part of the program where I want to check if the
pipe is empty is pretty far away from where I process the data
seems like a design problem. You cannot know whether you will ever get data until you actually do get data or see EOF. The process(es) at the write end of the pipe can delay an arbitrary amount of time before doing anything with the pipe, and even if that process is provided by you, you cannot be fully in control of this aspect of its behavior. Thus, it doesn't make much sense to try to check for EOF before you're ready, in some sense, to consume data, because you cannot rely on getting an answer without blocking.
, so I'd
rather not have to deal with saving the data from my first read until
then.
I suppose you must want to avoid performing some kind of heavyweight initialization in the event that there is no data to process. Ok, but I don't see what the big deal is. You need to provide storage into which to read the data anyway. What's wrong with something like this:
void consume_pipe_data(int fd) {
char buffer[BUFFER_SIZE];
ssize_t count;
count = read(fd, buffer, BUFFER_SIZE);
if (count == 0) {
handle_no_data();
return;
} else if (count > 0) {
perform_expensive_initialization();
}
do {
if (count == -1) {
handle_error();
return;
}
consume_data(buffer);
count = read(fd, buffer, BUFFER_SIZE);
} while (count);
}
The point is not that that's necessarily an appropriate structure for your program, but rather that it is possible to structure the program so that storing the data, if any, from the initial read is pretty clean and natural.
You can do a dummy-write. If your stdin has reached eof, it is a non-blocking mechanism to determine if you've reached EOF, without any more sophisticated tools.
if( write( fileno(stdin), 0, 0 ) != 0 )
return 1; // Is end-of-file.
First I have following macro
#define MSG_UPDATE_DATA 70
Then open a pipe with popen
SensServer = popen("./SensServer", "w") ;
In the following code that uses the putc(...) function to write to pipe, the function makes the program block and the lines of code following do not execute
void requestTempAndPress(int pid) {
printf("Temp and presure requested. msg_type: %d\n", MSG_UPDATE_DATA);
int n = putc(MSG_UPDATE_DATA, SensServer);
printf("Data sent: %d\n", MSG_UPDATE_DATA);
}
It outputs Temp and presure requested. msg_type: 70 fine. But not the "Data sent..." line.
As per the man page,
pipe is a fd, type int.
putc() needs a FILE* stream as argument.
So, most possibly, in your code, you are supplying the wrong type of argument to putc(), creating the issue.
Given the information (and lack of a sample program), this sounds like a question asking how to make pipes non-blocking. This has been discussed before, usually for nonblocking reads, e.g.,
Non-blocking pipe using popen?
Correct Code - Non-blocking pipe with popen (C++)
The first link mentions fcntl and the O_NONBLOCK flag which the manual page says can be applied to both reads and writes.
However, using popen makes the pipe using buffered I/O, while the operations addressed by fcntl are non-buffered read and write (you really cannot mix the two). If the program were changed to use the low-level pipe (as in the example for the first link), and consistently used the non-buffered I/O, it would give the intended behavior.
Here are links to more general discussion on the topic:
Introduction to non-blocking I/O
Blocking and Non-Blocking I/0
On the other hand (noting comments), if the program fragment is for example part of some larger system doing handshaking (expecting a timely response back from the server), that will run into problems. The fragment is writing a single character across the pipe. However, popen opens a (block-)buffered stream. Nothing will be sent directly to the server as single-character writes unless some help is provided. For instance, one could flush the output stream after each putc, e.g.,
fflush(SensServer);
Alternatively, one could make the stream unbuffered by changing it immediately after the successful call to popen, e.g., using setvbuf:
setvbuf(SensServer, NULL, _IONBF, 0);
Here are links for further reading about buffering in pipes:
Turn off buffering in pipe
Unix buffering delays output to stdout, ruins your day
Force line-buffering of stdout when piping to tee
The problem was due to the fact I initialised the variable SensServer in the parent process but not the child. Which meant the pointer was 0 or I guess a random memory location.
This is the prototype of the select statement (acc to man pages):
int select(int n, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
struct timeval *timeout);
I know what to use the readfds parameter for: with this one you can see if data was written to one of your sockets. On the other hand, the writefds page that I found, states that this is to see "if any of the sockets is ready to send() data to". But what does this mean? In Windows Sockets Network Programming by Quin and Shute it says that this detects either the connected or the writable state. What is the point of this? Is it simply to check whether a socket still has a connection to a connected client and test whether is has any use writing something to that socket?
So: what does one normally use writefds for?
If you keep writing to a TCP socket and the other side doesn't receive as fast as you send, there comes a time when write blocks. You want to avoid that so you need to test that "you can write without blocking". Because this normally doesn't happen in test programs, it could come as a shock, but write(2) and send(2) can block.
So if select(2) says a fd is set in writefds then it means any write or send on it will actually write at least one byte without blocking.
EDIT
From the standard:
The pselect() function shall examine the file descriptor sets whose
addresses are passed in the readfds, writefds, and errorfds parameters
to see whether some of their descriptors are ready for reading, are
ready for writing, or have an exceptional condition pending,
respectively.
I am successfully reading from a pipe from another thread, and printing the output (in an ncurses window as it happens).
I need to do this one character at a time, for various reasons, and I'm using a select() on the FD for the read end of the pipe, along with a few other FDs (like stdin).
My idea is to attempt to read from the pipe only when it is imminently ready to be read, in preference to handling any input. This seems to be working - at least to start. select() sets up the fd_set and if FD_ISSET I do my read() of 1 byte from the FD. But select() says yes one time too many, and the read() blocks.
So my question is this - why would select() report that a fd is ready for reading, if a subsequent read() blocks?
(approximately) This same code worked fine when the other end of the pipe was connected to a forked process, if that helps.
I can post the code on request, but it's bog standard. Set up a fd_set, copy it, select the copy, if the FD is set call a function which reads a byte from the same FD... otherwise revert the fd_set copy
EDIT: by request, here's the code:
Setting up my fd_set:
fd_set fds;
FD_ZERO(&fds);
FD_SET(interp_output[0], &fds);
FD_SET(STDIN_FILENO, &fds);
struct timeval timeout, tvcopy; timeout.tv_sec=1;
int maxfd=interp_output[0]+1; //always >stdin+1
fd_set read_fds;
FD_COPY(&fds, &read_fds);
In a loop:
if (select(maxfd, &read_fds, NULL, NULL, &timeout)==-1) {perror("couldn't select"); return;}
if (FD_ISSET(interp_output[0], &read_fds)) {
handle_interp_out();
} else if (FD_ISSET(STDIN_FILENO, &read_fds)) {
//waddstr(cmdwin, "stdin!"); wrefresh(cmdwin);
handle_input();
}
FDCOPY(&fds, &read_fds);
handle_interp_out():
void handle_interp_out() {
int ch;
read(interp_output[0], &ch, 1);
if (ch>0) {
if (ch=='\n') { if (cmd_curline>=cmdheight) cmdscroll(); wmove(cmdwin, ++cmd_curline, 1); }
else waddch(cmdwin, ch);
wrefresh(cmdwin);
}
}
EDIT 2: The write code is just a fprintf on a FILE* opened with fdopen(interp_output[1], "w") - this is in a different thread. All I'm getting to is my "prompt> " - it prints all that properly, but does one more iteration that it shouldn't. I've turned off buffering, which was giving me other problems.
EDIT 3: This has become a problem with my invocation of select(). It appears that, right away, it's returning -1 and errno's being set to 'invalid argument'. The read() doesn't know that and just keeps going. What could be wrong with my select()? I've updated the code and changed the title to more accurately reflect the problem...
EDIT 4: So now I'm thoroughly confused. The timeout value of .tv_sec=1 was no good, somehow. By getting rid of it, the code works just fine. If anybody has any theories, I'm all ears. I would just leave it at NULL, except this thread needs to periodically do updates.
To absolutely guarantee that the read will never block, you must set O_NONBLOCK on the fd.
Your select error is almost certainly caused because you aren't setting the entire time struct. You're only setting the seconds. The other field will contain garbage data picked up from the stack.
Use struct initialization. That will guarantee the other fields are set to 0.
It would look like this:
struct timeval timeout = {1, 0};
Also, in your select loop you should be aware that Linux will write the time remaining into the timeout value. That means that it will not be 1 second the next time through the loop unless you reset the value to 1 second.
According to the manpage:
On error, -1 is returned, and errno is set appropriately; the sets and timeout become undefined, so do not rely on their contents after an error.
You are not checking the return code from select().
The most likely explanation is that select() is being interrupted (errno=EINTR) and so returning an error, and the FD bit is still set in the "read" fd_set giving you the behaviour you are seeing.
Incidentally it is a very bad idea to name variables after standard/system/common functions. "read_fds" would be a MUCH better name than "read".
It's correct. See the select() manpage in Linux for example: http://linux.die.net/man/2/select
"Under Linux, select() may report a socket file descriptor as "ready for reading", while nevertheless a subsequent read blocks"
The only solution is to use NON-BLOCKING socket.
The answer is "it wouldn't". The behaviour you describe should never happen. Something else is going wrong.
I have been in a similar situation many times, and usually it turned out that there was a silly typo or cut and paste error elsewhere that led to behaviour that I mis-diagnosed.
If you post your code maybe we can help better - post the write code as well as the read please.
Also consider using asynchronous IO, even if only for debug purposes. If what you suspect is happening really is happening, then read() will return EWOULDBLOCK.
Also, you say you are copying the fd_set. How? Can you post your code for that?
I am having trouble understanding what it means to add a descriptor to writefds set for select() in linux. I wrote some simple code that adds the descriptor for stdout to the writefds set and uses a timeout of NULL. Now, my code just infinite loops checking if this descriptor is set, and if it does, it prints "WRITING". When I run my code it just keeps printing "WRITING" to infinity. The same thing happens when I do this for stdin. Again, there is no other code in the loop. Are stdin/stdout always just ready for writing?
It means you can call write on that fd and the kernel promises to not-block and consume at least 1 byte.
More details. If your socket is not in non-blocking mode and the kernel buffers associated with the socket are full, the kernel will put your thread to sleep until it can empty some of the buffer and be able to consume part of your write.
If your socket is in non-blocking mode and the kernel buffers are full, the write will return immediately without consuming any bytes.
The answer to the question "Is stdout always ready for writing" is "It depends."
stdout can be connected to anything that can be opened as a file descriptor - like a disk file, a network socket, or a pipe. The usual case is that it's connected to a terminal device.
Most of these types of file descriptors can block on writing (which means they might not be marked writeable after select() returns), but usually only if you're just written a very large amount of data to them (and so filled some kind of buffer). "Large amount" varies between the device types - if your stdout terminal is a 9600 baud serial device, then you could fill the write buffer pretty easily; an xterm, not so much.
Some device will never block - like disk files, or /dev/null, for example. (write() to a disk file might not complete immediately, but this isn't considered "blocking" - it's a "disk wait").
Yes, a truthy return from FD_ISSET(fd, &writefds) means fd is writeable. If you call select() with that FD set in the writefds after you get EWOULDBLOCK or EAGAIN (equivalent on Linux, at least) it blocks until the FD is again writeable.
There's more to it than that. For instance, an FD is also considered writeable if you've done a non-blocking connect() on it, you got EAGAIN, and call select() to wait for the connection to be established. That establishment is signalled in the writefds.