How to detect empty epoll set - c

I'm learning to use epoll, and I wrote the following example
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/epoll.h>
#include <unistd.h>
int main() {
int epfd;
struct epoll_event ev;
struct epoll_event ret;
char buf[200];
int n,k,t;
epfd = epoll_create(100);
assert(0 ==
fcntl(0, F_SETFL, fcntl(0, F_GETFL) | O_NONBLOCK)
);
ev.data.fd = 0;
ev.events = EPOLLIN | EPOLLET;
if(epoll_ctl(epfd, EPOLL_CTL_ADD, 0, &ev) != 0)
perror("epoll_ctl");
while((n = epoll_wait(epfd, &ret, 1, -1)) > 0) {
printf("tick!\n");
if(ret.data.fd == 0) {
k=0;
while((t=read(0, buf, 100)) > 0) {
k+=t;
}
if(k == 0) {
close(0);
printf("stdin done\n");
}
}
}
perror("epoll");
return 0;
}
If you try running it in the terminal it won't work properly since fds 0, 1 and 2 all point to same open file, so close(0) won't remove stdin from the epoll set. You can get around this by doing "cat | ./a.out". Dirty trick, I know, but setting up a small example with named pipes or sockets would be more complicated.
Now, everything works and the file is removed from the epoll set, but then the next epoll_wait call blocks permanently since it's on an empty set! So I would need to detect if the epoll file descriptor (epfd) is an empty epoll set.
How can I get around this? (in a general manner, not just calling exit when stdin is done)
Thanks!

Basically, if you're using epoll "correctly", then you should never have a situation of an unexpectedly empty epoll set. You should know when there is more to do or not. Well, or that's the theory at least. Let me review it:
You are using EPOLLET here (which is, imho, the right think in general). It means that the file descriptor 0 is removed from the epoll when it is returned in &ret. At this point you should handle it by reading some amount of data from 0, as you do, but then "re-arming" it by adding again file descriptor 0 into the epoll (unless it was closed of course). For an example of how this is supposed to work, remove the inner loop and just do:
k = read(0, buf, 100);
reading a maximum of 100 bytes. The idea is that if you pipe a file bigger than that, it should go several times through the whole loop. In order to make this work, if k > 0, after you handle the k bytes, you need to call epoll_ctl(..EPOLL_CTL_ADD..) again.
Note an annoying detail: it's possible occasionally that the read() returns 0 bytes without meaning the file or socket is at the end. Check if errno == EAGAIN || errno == EWOULDBLOCK. To detect that case, and then epoll_ctl(..EPOLL_CTL_ADD..) again.

The epoll set will be empty when you've removed everything that was added. As far as I know, you can't introspect the epoll set to find out whether there are any file descriptors present. So, it's up to you to determine when the epoll set becomes empty as outlined in Armin's answer.
Since you haven't explained what you expect from your program, I'll take a guess that you expect it exit when stdin is closed, because doing a close(0) will potentially cause file descriptor 0 to be removed from the epoll set. However, the code as listed is flawed. If you continue to wait on an epoll set that doesn't contain any file descriptors (whether removed automatically or by using EPOLL_CTL_DEL), the epoll_wait will wait forever.
The following code shows this nicely.
#include <errno.h>
#include <stdio.h>
#include <sys/epoll.h>
int main() {
int epfd;
int n;
struct epoll_event ret;
epfd = epoll_create(100);
while((n = epoll_wait(epfd, &ret, 1, -1)) > 0) {
/* Never gets here. */
printf("tick!\n");
}
return 0;
}
The epoll set doesn't contain any file descriptors, so the epoll_wait wait forever. If you happened to have a file connected to stdin in your program and no other file descriptor in your program was connected to stdin, the close(0) would have removed fd 0 from the set, the epoll set becomes empty, and the next epoll_wait waits forever.
In general, you manage file descriptors in the epoll set yourself, not rely on close calls to automatically remove your file descriptor from the set. It's up to you to decide whether to continue waiting on the epoll set after you've done the close(0).
I'd also suggest that you change the structure of your program to epoll_wait after the read. This guarantees that you'll obtain any data that may have arrived on stdin before your first call to epoll_wait.
Also, be careful with code like this:
k=0;
while((t=read(0, buf, 100)) > 0) {
k+=t;
}
if(k == 0) {
close(0);
printf("stdin done\n");
}
If you assume that the read in the loop consecutively returns 100 followed by 0 indicating some data plus an end of file, the close(0) will not be called. The program will loop and wait forever again on epoll_wait. Best to check the result of each read specifically for end-of-file and errors.

Related

Linux select() not blocking

I'm trying to understand the difference between select() and poll() better. For this I tried to implement a simple program that will open a file as write-only, add its file descriptor to the read set and than execute select in hopes that the function will block until the read permission is granted.
As this didnt work (and as far as I understood, this is intended behaviour) I tried to block access to the file using flock before the select() executen. Still, the program did not block its execution.
My sample code is as follows:
#include <stdio.h>
#include <poll.h>
#include <sys/file.h>
#include <errno.h>
#include <sys/select.h>
int main(int argc, char **argv)
{
printf("[+] Select minimal example\n");
int max_number_fds = FOPEN_MAX;
int select_return;
int cnt_pollfds;
struct pollfd pfds_array[max_number_fds];
struct pollfd *pfds = pfds_array;
fd_set fds;
int fd_file = open("./poll_text.txt", O_WRONLY);
struct timeval tv;
tv.tv_sec = 10;
tv.tv_usec = 0;
printf("\t[+] Textfile fd: %d\n", fd_file);
//create and set fds set
FD_ZERO(&fds);
FD_SET(fd_file, &fds);
printf("[+] Locking file descriptor!\n");
if(flock(fd_file,LOCK_EX) == -1)
{
int error_nr = errno;
printf("\t[+] Errno: %d\n", error_nr);
}
printf("[+] Executing select()\n");
select_return = select(fd_file+1, &fds, NULL, NULL, &tv);
if(select_return == -1){
int error_nr = errno;
printf("[+] Select Errno: %d\n", error_nr);
}
printf("[+] Select return: %d\n", select_return);
}
Can anybody see my error in this code? Also: I first tried to execute this code with two FDs added to the read list. When trying to lock them I had to use flock(fd_file,LOCK_SH) as I cannot exclusively lock two FDs with LOCK_EX. Is there a difference on how to lock two FDs of the same file (compared to only one fd)
I'm also not sure why select will not block when a file, that is added to the Read-set is opened as Write-Only. The program can never (without a permission change) read data from the fd, so in my understanding select should block the execution, right?
As a clarification: My "problem" I want to solve is that I have to check if I'm able to replace existing select() calls with poll() (existing in terms of: i will not re-write the select() call code, but will have access to the arguments of select.). To check this, I wanted to implement a test that will force select to block its execution, so I can later check if poll will act the same way (when given similar instructions, i.e. the same FDs to check).
So my "workflow" would be: write tests for different select behaviors (i.e. block and not block), write similar tests for poll (also block, not block) and check if/how poll can be forced do exactly what select is doing.
Thank you for any hints!
When select tells you that a file descriptor is ready for reading, this doesn't necessarily mean that you can read data. It only means that a read call will not block. A read call will also not block when it returns an EOF or error condition.
In your case I expect that read will immediately return -1 and set errno to EBADF (fd is not a valid file descriptor or is not open for reading) or maybe EINVAL (fd is attached to an object which is unsuitable for reading...)
Edit: Additional information as requested in a comment:
A file can be in a blocking state if a physical operation is needed that will take some time, e.g. if the read buffer is empty and (new) data has to be read from the disk, if the file is connected to a terminal and the user has not yet entered any (more) data or if the file is a socket or a pipe and a read would have to wait for (new) data to arrive...
The same applies for write: If the send buffer is full, a write will block. If the remaining space in the send buffer is smaller than your amount of data, it may write only the part that currently fits into the buffer.
If you set a file to non-blocking mode, a read or write will not block but tell you that it would block.
If you want to have a blocking situation for testing purposes, you need control over the process or hardware that provides or consumes the data. I suggest to use read from a terminal (stdin) when you don't enter any data or from a pipe where the writing process does not write any data. You can also fill the write buffer on a pipe when the reading process does not read from it.

why non-blocking write to disk doesn't return EAGAIN or EWOULDBLOCK?

I modified a program from APUE, the program first open a file, then mark the fd as non-blocking, then continue write to the fd until write return -1.
I think since disk I/O is slow, when write buffers in OS is nearly full, the write system call will return -1, and the errno should be EAGAIN or EWOULDBLOCK.
But I ran the program for about several minutes and I repeated running the program serveral times, the write system call didn't returned -1 even once! Why?
Here's the code:
#include "apue.h"
#include <errno.h>
#include <fcntl.h>
char buf[4096];
int
main(void)
{
int nwrite;
int fd = open("a.txt", O_RDWR);
if(fd<0){
printf("fd<0\n");
return 0;
}
int i;
for(i = 0; i<sizeof(buf); i++)
buf[i] = i*2;
set_fl(fd, O_NONBLOCK); /* set nonblocking */
while (1) {
nwrite = write(fd, buf, sizeof(buf));
if (nwrite < 0) {
printf("write returned:%d, errno=%d\n", nwrite, errno);
return 0;
}
}
clr_fl(STDOUT_FILENO, O_NONBLOCK); /* clear nonblocking */
exit(0);
}
The O_NONBLOCK flag is primarily meaningful for file descriptors representing streams (e.g, pipes, sockets, and character devices), where it prevents read and write operations from blocking when there is no data waiting to read, or buffers are too full to write anything more at the moment. It has no effect on file descriptors opened to regular files; disk I/O delays are essentially ignored by the system.
If you want to do asynchronous I/O to files, you may want to take a look at the POSIX AIO interface. Be warned that it's rather hairy and infrequently used, though.

FIFO pipe is always readable in select()

In C pseudo-code:
while (1) {
fifo = open("fifo", O_RDONLY | O_NONBLOCK);
fd_set read;
FD_SET(fifo, &read);
select(nfds, &read, NULL, NULL, NULL);
}
The process sleeps as triggered by select() until another process writes into fifo. Afterwards it will always find fifo as a readable file descriptor.
How to avoid this behavior (that is, after fifo has been read once, how to make it be found as unreadable until it gets another write?)
You opened that FIFO as read only (O_RDONLY), whenever there is no writer to the FIFO, the read end will receive an EOF.
Select system call will return on EOF and for every EOF you handle there will be a new EOF. This is the reason for the observed behavior.
To avoid this open that FIFO for both reading and writing (O_RDWR). This ensures that you have at least one writer on the FIFO thus there wont be an EOF and as a result select won't return unless someone writes to that FIFO.
The simple answer is to read until read() returns EWOULDBLOCK (or EAGAIN), or craps out with an error.
What you are saying simply cannot be happening unless the operating system (or runtime) that you are using is buggy. Otherwise you must be doing something wrong. For example, select() is using level-triggered I/O. I'd think that, most likely, you are not draining the socket completely, and so select() always indicates that you have something left in there (this does not happen with edge-triggered event notifications).
Below is a simple example that shows how one should read until the read() returns EWOULDBLOCK in order to avoid leaving descriptor in readable state (I've compiled and tested this on OS X, and there is also mostly no error checking, but you should get the idea):
/*
* FIFO example using select.
*
* $ mkfifo /tmp/fifo
* $ clang -Wall -o test ./test.c
* $ ./test &
* $ echo 'hello' > /tmp/fifo
* $ echo 'hello world' > /tmp/fifo
* $ killall test
*/
#include <sys/types.h>
#include <sys/select.h>
#include <errno.h>
#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
int main()
{
int fd;
int n;
fd_set set;
ssize_t bytes;
size_t total_bytes;
char buf[1024];
fd = open("/tmp/fifo", O_RDWR | O_NONBLOCK);
if (fd == -1) {
perror("open");
return EXIT_FAILURE;
}
FD_ZERO(&set);
FD_SET(fd, &set);
for (;;) {
n = select(fd+1, &set, NULL, NULL, NULL);
if (!n)
continue;
if (n == -1) {
perror("select");
return EXIT_FAILURE;
}
if (FD_ISSET(fd, &set)) {
printf("Descriptor %d is ready.\n", fd);
total_bytes = 0;
for (;;) {
bytes = read(fd, buf, sizeof(buf));
if (bytes > 0) {
total_bytes += (size_t)bytes;
} else {
if (errno == EWOULDBLOCK) {
/* Done reading */
printf("done reading (%lu bytes)\n", total_bytes);
break;
} else {
perror("read");
return EXIT_FAILURE;
}
}
}
}
}
return EXIT_SUCCESS;
}
Basically, level-triggered I/O means that you get notified all the time if there is something to read, even though you might have been notified of this before. On a contrary, edge-triggered I/O means that you are getting notified only once every time new data arrives and it doesn't matter whether you read it or not. select() is a level-triggered I/O interface.
Hope it helps. Good Luck!

How to see if a pipe is empty

Assuming a pipe,
int pipe_fd[2];
pipe(pipe_fd);
We fork, and expect that one process will write into the pipe at an arbitrary time. In one of the processes, we want to be able to check the contents of the pipe without blocking.
i.e. While a typical read will block if nothing is present and the write end remains open. I want to go do other stuff and potentially even read a bit at a time, do some stuff, and then check back to see if there's more, a la:
close(pipe_fd[1]);
while(1){
if(/**Check pipe contents**/){
int present_chars = 0;
while( read(pipe_fd[0],&buffer[present_chars],1) != 0)
++present_chars;
//do something
}
else
//do something else
}
Your logic is wrong in that read will not return 0 when it runs out of characters; instead, it will block until it receives more, unless you put the file in non-blocking mode, but then it will return -1 and set errno to EWOULDBLOCK or EAGAIN rather than returning 0. The only time read can ever return 0 is when the size argument was 0 or end-of-file has been reached. And, for pipes, end-of-file means the writing end of the pipe has been closed; end-of-file status does not occur just because there's not any input available yet.
With that said, the simplest way to check is:
if (poll(&(struct pollfd){ .fd = fd, .events = POLLIN }, 1, 0)==1) {
/* data available */
}
but unless you're using nonblocking mode, you'll need to make this check before every single read operation. Passing a larger buffer to read rather than doing it a byte-at-a-time would eliminate most of the cost of checking.
You can check if there is data to be read with the read() function. From read(3):
When attempting to read from an empty pipe or FIFO:
* If some process has the pipe open for writing and
O_NONBLOCK is set, read() shall return -1 and set
errno to [EAGAIN].
* If some process has the pipe open for writing and
O_NONBLOCK is clear, read() shall block the calling
thread until some data is written or the pipe is
closed by all processes that had the pipe open for
writing.
The read() function shall fail if:
EAGAIN or EWOULDBLOCK
The file descriptor is for a socket, is marked
O_NONBLOCK, and no data is waiting to be received.
So if you set O_NONBLOCK, you will be able to tell if something is to be read on the pipe, by simply calling read().
As a reminder, from open(3):
SYNOPSIS
int open(const char *path, int oflag, ... );
DESCRIPTION
Values for oflag are constructed by a
bitwise-inclusive OR of flags from the following
list, defined in <fcntl.h>. Applications shall
specify exactly one of the first three values
(file access modes) below in the value of oflag:
O_NONBLOCK [...]
I hope it helps.
R..'s answer is good however poll returns the number of file descriptor structs that have flags set in "revents". This will be 1 if you can read from fd but will also be 1 if any of the error flags are set. This means R..'s answer will say the pipe is readable if it ever enters an error state. A more robust check could be something like this:
bool canReadFromPipe(){
//file descriptor struct to check if POLLIN bit will be set
//fd is the file descriptor of the pipe
struct pollfd fds{ .fd = fd, .events = POLLIN };
//poll with no wait time
int res = poll(&fds, 1, 0);
//if res < 0 then an error occurred with poll
//POLLERR is set for some other errors
//POLLNVAL is set if the pipe is closed
if(res < 0||fds.revents&(POLLERR|POLLNVAL))
{
//an error occurred, check errno
}
return fds.revents&POLLIN;
}

How to terminate read() when EOF is not encountered?

I am building a client/server model but using sockets, using named pipes, with mkfifo().
A client writes output into the name pipe, and I read the input in my server using:
while ((n = read(fd_in, &newChar, 1)) == 1) { /* ... */ }
I am reading one character at a time, until I encounter the two characters: <'CR'><'LF'>. I would like to make my code in such a way that if a client does not terminate with <'CR'><'LF'> after some time maybe, I can discard it and proceed to another client, otherwise the next client will have to wait, maybe infinitely.
Is there a way please to terminate the execution of read()? If it has not returned in 2 seconds, I could say interrupt read and discard the previously read characters, and start reading again please?
Thank you for your help,
Jary
#include <stdbool.h>
#include <poll.h>
do {
ssize_t ret;
struct pollfd ps = {.fd = fd_in, .events = POLLIN};
if (poll(&ps, 1, 2000) < 0)
break; /* kick client */
ret = read(in_fd, ...);
if (ret != 1)
break;
/* process read data */
} while (true);
This checks for whether there is data to be read; if there is not within 2000 msec, do whatever it is you want (e.g. disconnect).
Try passing the O_NONBLOCK flag when you open the read-end of the FIFO. That should change the behavior so that read returns right away even if the number of requested characters is not in the pipe.
To handle multiple clients simultaneously, you should set the file descriptors non-blocking with fcntl(), and then use select() or poll() to block until input appears on at least one of them.

Resources