I'm trying to write a simple daemon in Linux, which will create a FIFO, then collect anything written to the FIFO and write that data to a file at a later time.
My expectations are that once my daemon has created the FIFO, I can do "echo text > /myfifo" repeatedly. When I'm done, I can do "echo quit > /myfifo" and my program will exit and write all data to disk.
I'm currently using poll() to know when there's more data on the FIFO. This works fine until after the first time I echo data to the FIFO. The data is echoed fine, but my poll continuously returns SIGHUP after that.
Do I need to reset (or close & reopen) the FIFO after each process writes to it?
Pseudo-code of my code looks like this:
ret = fifo(my_fifo, mode);
fd = open(my_fifo, O_RDONLY | O_NONBLOCK);
polling.fd = fd;
polling.events = POLLIN | POLLPRI;
do {
ret = poll(&polling, 1, -1);
amt = read(fd, buf, bufsize);
// do stuff
} while (!done);
You have to keep reopening the FIFO, I think. I have a program that monitors a FIFO, and the monitor loop is:
/* Implement monitor mode */
void sql_monitor(char *fifo)
{
if (chk_fifo(fifo) != 0)
cmd_error(E_NOTFIFO, fifo);
/* Monitor -- device is assumed to be a FIFO */
while (1)
{
ctxt_newcontext();
if (ctxt_setinput(fifo) != 0)
sql_exit(1);
sql_file();
ctxt_endcontext();
}
}
The ctxt_newcontext() function stashes the current I/O state; the ctxt_setinput() function sets the input file to the named file - a FIFO in this case. The sql_file() function reads from the file (FIFO) until the end is reached - the file is closed. The ctxt_endcontext() undoes what ctxt_newcontext() does. The process repeats... This code has been around since about 1990.
So, YES, you will need to keep closing and reopening the FIFO after reading the end of the file (after each process such as echo finishes writing to the FIFO).
(You should also notice that there really isn't a need to poll the FIFO unless there's something else for the process to be doing when there is no data. The read() call will wait until there is data before returning.)
Related
I'm trying to understand the difference between select() and poll() better. For this I tried to implement a simple program that will open a file as write-only, add its file descriptor to the read set and than execute select in hopes that the function will block until the read permission is granted.
As this didnt work (and as far as I understood, this is intended behaviour) I tried to block access to the file using flock before the select() executen. Still, the program did not block its execution.
My sample code is as follows:
#include <stdio.h>
#include <poll.h>
#include <sys/file.h>
#include <errno.h>
#include <sys/select.h>
int main(int argc, char **argv)
{
printf("[+] Select minimal example\n");
int max_number_fds = FOPEN_MAX;
int select_return;
int cnt_pollfds;
struct pollfd pfds_array[max_number_fds];
struct pollfd *pfds = pfds_array;
fd_set fds;
int fd_file = open("./poll_text.txt", O_WRONLY);
struct timeval tv;
tv.tv_sec = 10;
tv.tv_usec = 0;
printf("\t[+] Textfile fd: %d\n", fd_file);
//create and set fds set
FD_ZERO(&fds);
FD_SET(fd_file, &fds);
printf("[+] Locking file descriptor!\n");
if(flock(fd_file,LOCK_EX) == -1)
{
int error_nr = errno;
printf("\t[+] Errno: %d\n", error_nr);
}
printf("[+] Executing select()\n");
select_return = select(fd_file+1, &fds, NULL, NULL, &tv);
if(select_return == -1){
int error_nr = errno;
printf("[+] Select Errno: %d\n", error_nr);
}
printf("[+] Select return: %d\n", select_return);
}
Can anybody see my error in this code? Also: I first tried to execute this code with two FDs added to the read list. When trying to lock them I had to use flock(fd_file,LOCK_SH) as I cannot exclusively lock two FDs with LOCK_EX. Is there a difference on how to lock two FDs of the same file (compared to only one fd)
I'm also not sure why select will not block when a file, that is added to the Read-set is opened as Write-Only. The program can never (without a permission change) read data from the fd, so in my understanding select should block the execution, right?
As a clarification: My "problem" I want to solve is that I have to check if I'm able to replace existing select() calls with poll() (existing in terms of: i will not re-write the select() call code, but will have access to the arguments of select.). To check this, I wanted to implement a test that will force select to block its execution, so I can later check if poll will act the same way (when given similar instructions, i.e. the same FDs to check).
So my "workflow" would be: write tests for different select behaviors (i.e. block and not block), write similar tests for poll (also block, not block) and check if/how poll can be forced do exactly what select is doing.
Thank you for any hints!
When select tells you that a file descriptor is ready for reading, this doesn't necessarily mean that you can read data. It only means that a read call will not block. A read call will also not block when it returns an EOF or error condition.
In your case I expect that read will immediately return -1 and set errno to EBADF (fd is not a valid file descriptor or is not open for reading) or maybe EINVAL (fd is attached to an object which is unsuitable for reading...)
Edit: Additional information as requested in a comment:
A file can be in a blocking state if a physical operation is needed that will take some time, e.g. if the read buffer is empty and (new) data has to be read from the disk, if the file is connected to a terminal and the user has not yet entered any (more) data or if the file is a socket or a pipe and a read would have to wait for (new) data to arrive...
The same applies for write: If the send buffer is full, a write will block. If the remaining space in the send buffer is smaller than your amount of data, it may write only the part that currently fits into the buffer.
If you set a file to non-blocking mode, a read or write will not block but tell you that it would block.
If you want to have a blocking situation for testing purposes, you need control over the process or hardware that provides or consumes the data. I suggest to use read from a terminal (stdin) when you don't enter any data or from a pipe where the writing process does not write any data. You can also fill the write buffer on a pipe when the reading process does not read from it.
Similar to the problem asked a while ago on kernel 3.x, but I'm seeing it on 4.9.37.
The named fifo is created with mkfifo -m 0666. On the read side it is opened with
int fd = open(FIFO_NAME, O_RDONLY | O_NONBLOCK);
The resulting fd is passed into a call to select(). Everything works ok, till I run echo >> <fifo-name>.
Now the fd appears in the read_fds after the select() returns. A read() on the fd will return one byte of data. So far so good.
The next time when select() is called and it returns, the fd still appears in the read_fds, but read() will always return zero meaning with no data. Effectively the read side would consume 100% of the processor capacity. This is exactly the same problem as observed by the referenced question.
Has anybody seen the same issue? And how can it be resolved or worked-around properly?
I've figured out if I close the read end of the fifo, and re-open it again, it will work properly. This probably is ok because we are not sending a lot of data. Though this is not a nice or general work-around.
This is expected behaviour, because the end-of-input case causes a read() to not block; it returns 0 immediately.
If you look at man 2 select, it says clearly that a descriptor in readfds is set if a read() on that descriptor would not block (at the time of the select() call).
If you used poll(), it too would immediately return with POLLHUP in revents.
As OP notes, the correct workaround is to reopen the FIFO.
Because the Linux kernel maintains exactly one internal pipe object to represent each open FIFO (see man 7 fifo and man 7 pipe), the robust approach in Linux is to open another descriptor to the FIFO whenever an end of input is encountered (read() returning 0), and close the original. During the time when both descriptors are open, they refer to the same kernel pipe object, so there is no race window or risk of data loss.
In pseudo-C:
fifoflags = O_RDONLY | O_NONBLOCK;
fifofd = open(fifoname, fifoflags);
if (fifofd == -1) {
/* Error checking */
}
/* ... */
/* select() readfds contains fifofd, or
poll() returns POLLIN for fifofd: */
n = read(fifofd, buffer, sizeof buffer)
if (!n) {
int tempfd;
tempfd = open(fifopath, fifoflags);
if (tempfd == -1) {
const int cause = errno;
close(fifofd);
/* Error handling */
}
close(fifofd);
fifofd = tempfd;
/* A writer has closed the FIFO. */
} else
/* Handling for the other read() result cases */
The file descriptor allocation policy in Linux is such that tempfd will be the lowest-numbered free descriptor.
On my system (Core i5-7200U laptop), reopening a FIFO in this way takes less than 1.5 µs. That is, it can be done about 680,000 times a second. I do not think this reopening is a bottleneck for any sensible scenario, even on low-powered embedded Linux machines.
I am working with pipes and one pipe won't open, even though mkfifo() was successful.
I have this:
/* create the FIFO (named pipe) */
int ret_mk = mkfifo(out_myfifo, 0666);
if(ret_mk < 0) {
perror(out_myfifo);
unlink(out_myfifo);
return -1;
}
printf("ret_mk = %d\n", ret_mk);
/* write to the FIFO */
out_fd = open(out_myfifo, O_WRONLY);
printf("out_fd = %d\n", out_fd);
but nothing gets printed after open(), even a print of random text won't show up.
From here we have:
The open() function returns an integer value, which is used to refer to the file. If unsuccessful, it returns -1, and sets the global variable errno to indicate the error type.
What can I do to see why it won't open?
Read fifo(7). For FIFOs, an open call may block. To make open(2) non-blocking, use O_NONBLOCK in the flag argument:
out_fd = open(out_myfifo, O_WRONLY|O_NONBLOCK);
if (out_fd<0) perror(out_myfifo);
printf("%d\n", out_fd);
But usually you want a blocking open for write on a FIFO, because some other process should open the same FIFO for reading (and you want your writing process to wait that to happen).
Notice that there is no way to poll(2) the event that someone else has opened the other end of your FIFO (because poll wants an opened file descriptor). See also inotify(7); you could also want to use unix(7) sockets.
BTW, you could also use strace(1) for debugging purposes.
See also intro(2) and Advanced Linux Programming.
I'm creating a quite-big project as an homework where I need to create a server program which listen to 2 fifos, where clients will write.
Everything works, but there is something that is making me angry: whenever I do an operation, which is composed from some write/reads between client and server, when I close fifos on client, it looks like server "think" that there is still someone keeping those fifos opened.
Due to this, the server tries to read 64 byte after each operation, obviusly failing (reading 0 bytes). Only one time per operation this thing happens, it doesn't keep trying to read 64 byte
It doesn't create any problem to clients but it's really strange and I hate those type of bugs
I think it's a problem connected to open/close and to the fact that clients use a lock.
Note, flags used on the open operation are specified in this pseudocode text
Server behaviour:
Open Fifo(1) for READING (O_RDONLY)
Open Fifo(2) for WRITING (O_WRONLY)
Do some operations
Close Fifo(1)
Close Fifo(2)
Client behaviour:
Set a lock on Fifo(1) (waiting if there is already one)
Set a lock on Fifo(2) (same as before)
Open Fifo(1) for WRITING (O_WRONLY)
Open Fifo(2) for READING (O_RDONLY)
Do some operations
Close Fifo(1)
Close Fifo(2)
Get lock from Fifo(1)
Get lock from Fifo(2)
I can't post directly the code, except from the functions used for networking because the project is quite big and I don't use syscalls directly. Here you are:
int Network_Open(const char* path,int oflag)
{
return open(path,oflag);
}
ssize_t Network_IO(int fifo,NetworkOpCodes opcode,void* data,size_t dataSize)
{
ssize_t retsize = 0;
errno = 0;
if (dataSize == 0) return 0;
while ((retsize = (opcode == NetworkOpCode_Write? write(fifo,data,dataSize) : read(fifo,data,dataSize))) < 0)
{
if (errno != EINTR) break;
}
return retsize;
}
Boolean Network_Send(int fifo,const void* data,size_t dataSize)
{
return ((ssize_t)dataSize) == Network_IO(fifo,NetworkOpCode_Write,(void*)data,dataSize);
}
Boolean Network_Receive(int fifo,void* data,size_t dataSize)
{
return ((ssize_t)dataSize) == Network_IO(fifo,NetworkOpCode_Read,data,dataSize);
}
Boolean Network_Close(int fifo)
{
if (fifo >= 0)
return close(fifo) == 0;
}
Any help will be appreciated, thanks.
EDIT 1:
Client output: http://pastie.org/2523854
Server output (strace): http://pastie.org/2523858
Zero bytes returned from (blocking) read() indicates an end of file, i.e., that the other end has closed the FIFO. Read the manpage for read.
The zero bytes result from read() means that the other process has finished. Now your server must close the original file descriptor and reopen the FIFO to serve the next client. The blocking operations will resume once you start working with the new file descriptor.
That's the way it is supposed to work.
AFAIK, after you get the zero bytes, further attempts to read on the file descriptor will also return 0 bytes, in perpetuity (or until you close the file descriptor). Even if another process opens the FIFO, the original file descriptor will continue to indicate EOF (the other client process will be hung waiting for a server process to open the FIFO for reading).
I am building a client/server model but using sockets, using named pipes, with mkfifo().
A client writes output into the name pipe, and I read the input in my server using:
while ((n = read(fd_in, &newChar, 1)) == 1) { /* ... */ }
I am reading one character at a time, until I encounter the two characters: <'CR'><'LF'>. I would like to make my code in such a way that if a client does not terminate with <'CR'><'LF'> after some time maybe, I can discard it and proceed to another client, otherwise the next client will have to wait, maybe infinitely.
Is there a way please to terminate the execution of read()? If it has not returned in 2 seconds, I could say interrupt read and discard the previously read characters, and start reading again please?
Thank you for your help,
Jary
#include <stdbool.h>
#include <poll.h>
do {
ssize_t ret;
struct pollfd ps = {.fd = fd_in, .events = POLLIN};
if (poll(&ps, 1, 2000) < 0)
break; /* kick client */
ret = read(in_fd, ...);
if (ret != 1)
break;
/* process read data */
} while (true);
This checks for whether there is data to be read; if there is not within 2000 msec, do whatever it is you want (e.g. disconnect).
Try passing the O_NONBLOCK flag when you open the read-end of the FIFO. That should change the behavior so that read returns right away even if the number of requested characters is not in the pipe.
To handle multiple clients simultaneously, you should set the file descriptors non-blocking with fcntl(), and then use select() or poll() to block until input appears on at least one of them.