I have created a FIFO where I can do non-blocking writes into in this way:
// others, searching for a non-blocking FIFO-writer may copy this ;-)
mkfifo("/tmp/myfifo", S_IRWXU);
int fifo_fd = open("/tmp/myfifo", O_RDWR);
fcntl(fifo_fd, F_SETFL, fcntl(fifo_fd, F_GETFL) | O_NONBLOCK);
// and then in a loop:
LOGI("Writing into fifo.");
if (write(fifo_fd, data, count) < 0) {
LOGE("Failed to write into fifo: %s", strerror(errno));
}
The non-blocking write works perfect, but due to my logging I can see:
(..)
Writing into fifo.
Writing into fifo.
Writing into fifo.
Failed to write into fifo: Try again
Writing into fifo.
Failed to write into fifo: Try again
Writing into fifo.
Failed to write into fifo: Try again
After I created a reader like cat /tmp/myfifo > foo.out, the error goes away.
I thought the FIFO works like a ring-buffer and drops the first-written bytes when the buffer is full. But now I have learned that it blocks/prevents new bytes until the first bytes are read.
Does anyone know a simple other way or additional operation I can make so that the FIFO behaves like a ring-buffer?
Related
In this code my program crashes in when I am opening the pipe for writing.
char pipe[30];
int fd, tmp = 2;
sprintf(pipe, "root_%d", getpid());
ret_val = mkfifo(pipe, 0666);
fd = open(pipe, O_WRONLY); //HERE IS CRASHING - SUDDENLY FREEZES
write(fd, &tmp, sizeof(int));
close(fd)
All seems good, but where is my mistake;
It is an expected behavior. From man 7 fifo:
Normally, opening the FIFO blocks until the other end is opened also.
So your open does not return until somebody opens the same pipe for reading. You may want to add O_NONBLOCK flag (and likely get SIGPIPE on writing), or revisit the design.
Similar to the problem asked a while ago on kernel 3.x, but I'm seeing it on 4.9.37.
The named fifo is created with mkfifo -m 0666. On the read side it is opened with
int fd = open(FIFO_NAME, O_RDONLY | O_NONBLOCK);
The resulting fd is passed into a call to select(). Everything works ok, till I run echo >> <fifo-name>.
Now the fd appears in the read_fds after the select() returns. A read() on the fd will return one byte of data. So far so good.
The next time when select() is called and it returns, the fd still appears in the read_fds, but read() will always return zero meaning with no data. Effectively the read side would consume 100% of the processor capacity. This is exactly the same problem as observed by the referenced question.
Has anybody seen the same issue? And how can it be resolved or worked-around properly?
I've figured out if I close the read end of the fifo, and re-open it again, it will work properly. This probably is ok because we are not sending a lot of data. Though this is not a nice or general work-around.
This is expected behaviour, because the end-of-input case causes a read() to not block; it returns 0 immediately.
If you look at man 2 select, it says clearly that a descriptor in readfds is set if a read() on that descriptor would not block (at the time of the select() call).
If you used poll(), it too would immediately return with POLLHUP in revents.
As OP notes, the correct workaround is to reopen the FIFO.
Because the Linux kernel maintains exactly one internal pipe object to represent each open FIFO (see man 7 fifo and man 7 pipe), the robust approach in Linux is to open another descriptor to the FIFO whenever an end of input is encountered (read() returning 0), and close the original. During the time when both descriptors are open, they refer to the same kernel pipe object, so there is no race window or risk of data loss.
In pseudo-C:
fifoflags = O_RDONLY | O_NONBLOCK;
fifofd = open(fifoname, fifoflags);
if (fifofd == -1) {
/* Error checking */
}
/* ... */
/* select() readfds contains fifofd, or
poll() returns POLLIN for fifofd: */
n = read(fifofd, buffer, sizeof buffer)
if (!n) {
int tempfd;
tempfd = open(fifopath, fifoflags);
if (tempfd == -1) {
const int cause = errno;
close(fifofd);
/* Error handling */
}
close(fifofd);
fifofd = tempfd;
/* A writer has closed the FIFO. */
} else
/* Handling for the other read() result cases */
The file descriptor allocation policy in Linux is such that tempfd will be the lowest-numbered free descriptor.
On my system (Core i5-7200U laptop), reopening a FIFO in this way takes less than 1.5 µs. That is, it can be done about 680,000 times a second. I do not think this reopening is a bottleneck for any sensible scenario, even on low-powered embedded Linux machines.
I am trying to send a string to a pipe in unix. When I go through a line-by-line debugging process, the call mkfifo() creates the file in the same directory as the source code. However, when I reach the open() call, the debugger is no longer able to proceed. I'm not sure why it is unable to access the pipe file.
Here's the code in question:
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
int main()
{
int fd;
char * myfifo = "myfifo";
/* create the FIFO (named pipe) */
mkfifo(myfifo, 0666);
/* write "Hi" to the FIFO */
fd = open(myfifo, O_WRONLY);
write(fd, "Hi", sizeof("Hi"));
close(fd);
/* remove the FIFO */
unlink(myfifo);
return 0;
}
Any suggestions are appreciated. Thank you.
Normally a FIFO has to be open at both ends simultaneously before either side can proceed. Since you didn't mention anything about a reader, the most likely answer is that you haven't got one, or you haven't set it up yet. Once you do, the open will be allowed to proceed.
mkfifo(3) routes to fifo(7) which reads:
The kernel maintains exactly one pipe object for each FIFO special file that is opened by at least one process. The FIFO must be opened on both ends (reading and writing) before data can be passed. Normally, opening the FIFO blocks until the other end is opened also.
There is a solution for non-blocking read:
A process can open a FIFO in nonblocking mode. In this case, opening for read only will succeed even if no-one has opened on the write side yet, opening for write only will fail with ENXIO (no such device or address) unless the other end has already been opened.
So you could fork another process for reading:
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
long strlen(char * c){
return c[0] == 0 ? 0 : 1 + strlen(++c);
}
int main()
{
int fd;
int fr;
char buf[3];
char * MESSAGE = "Hi\n";
char * myfifo = "myfifo";
/* create the FIFO (named pipe) */
mkfifo(myfifo, 0666);
int msglen = strlen(MESSAGE);
int child = fork();
if (child == 0){
/* read "Hi" from the FIFO (CHILD)*/
fr = open(myfifo, O_RDONLY);
read(fr, buf, msglen);
write(1, buf, msglen);
close(fr);
} else {
/* write "Hi" to the FIFO (PARENT)*/
fd = open(myfifo, O_WRONLY);
write(fd, MESSAGE, sizeof(char) * msglen);
close(fd);
/* remove the FIFO */
wait(child);
unlink(myfifo);
}
return 0;
}
I guess you have to open both ends before you write.
here is an excerpt from the man page for mkfifo() See my notes at the end
mkfifo() makes a FIFO special file with name pathname. mode
specifies the FIFO's permissions. It is modified by the process's
umask in the usual way: the permissions of the created file are (mode
& ~umask).
A FIFO special file is similar to a pipe, except that it is created
in a different way. Instead of being an anonymous communications
channel, a FIFO special file is entered into the filesystem by
calling mkfifo().
Once you have created a FIFO special file in this way, any process
can open it for reading or writing, in the same way as an ordinary
file. However, it has to be open at both ends simultaneously before
you can proceed to do any input or output operations on it. Opening
a FIFO for reading normally blocks until some other process opens the
same FIFO for writing, and vice versa. See fifo(7) for nonblocking
handling of FIFO special files.
Notes: There are two details of importance:
1) the 'mode' parameter is modified by the value of 'umask'
2) both ends of the fifo must be open at the same time
before any I/O operations can be performed.
PRETEND THEY'RE NOT PARENT AND CHILD PROCESSES EVEN THOUGH THEY ARE.
MAKE FIFO:
/* Create response FIFO. */
if (mkfifo(RESP_FIFO_NAME, FIFO_MODE) == -1) {
if (errno != EEXIST) {
fprintf(stderr, "Server: Couldn’t create %s FIFO.\n", RESP_FIFO_NAME);
exit(1);
}
}
Fork:
/* 3. Fork the client process. */
switch (fork()) {
/* Fork failed. */
case (pid_t) -1:
fprintf(stderr, "Call to fork failed.\n");
exit(1);
/* Client (child) process. */
case 0:
execlp(CLIENT_NAME, CLIENT_NAME, argv[SERVER_CMD_FILE_ARG], argv[SERVER_LOG_FILE_ARG], NULL);
/* Server (parent) Process */
default:
sleep(1);
server(infd, outfd, argv[INIT_DB_ARG], argv[FINAL_DB_ARG]);
} /* End of switch. */
server function:
int server(int infd, int outfd, char *init_db_name, char *final_db_name) {
...
if ((outfd = open(RESP_FIFO_NAME, O_WRONLY)) == -1) {
fprintf(stderr, "Server: Failed to open %s FIFO.\n", RESP_FIFO_NAME);
perror(RESP_FIFO_NAME);
exit(1);
}
...
}
client program:
printf("RESP_FIFO FILE DESCRIPTOR: %d\n", infd);
/* Open the response FIFO for reading. */
if ((infd = open(RESP_FIFO_NAME, O_RDONLY)) == -1) {
fprintf(stderr, "Client: Failed to open %s FIFO.\n", RESP_FIFO_NAME);
exit(1);
}
else printf("RESP_FIFO FILE DESCRIPTOR: %d\n", infd);
TL;DR The open for reading call in client program is not being executed before the open for writing call in the server program.
Are you opening the response fifo for writing before its other end is open for reading? See fex. Having a trouble with opening FIFO in C
Either wait until you know the FIFO is open for reading or make the open blocking, to wait for the client. Also make sure the server has write permission for the FIFO file.
What about pipe() for this
From pipe(2):
Create descriptor pair for interprocess communication.
The pipe() function creates a pipe (an object that allows unidirectional
data flow) and allocates a pair of file descriptors. The first descrip-
tor connects to the read end of the pipe; the second connects to the
write end.
Data written to fildes1 appears on (i.e., can be read from) fildes[0].
You can look how memcached use it
Yes, without a reader, an open() of a filesystem FIFO for writing will either block or, in the nonblocking case, fail with ENXIO.
You have at least two easy options.
First, you could open the "command" FIFO, nonblocking, for reading in addition to writing, either O_RDWR or with two separate file descriptors, one O_RDONLY and one O_WRONLY.
Second, you could use a filesystem socket instead, and have the server listen there. That gives you a bi-directional communication channel over one ofile.
UNIX gives you other options, too — message queues or files or shared memory segments, perhaps using signals for one interlocutor to prod the other, come to mind — but the above are quite straightforward.
I wrote into a FIFO on one end without reading out of the other end. Just having the files open for reading and writing is not enough, you have to actually read the text out of the FIFO or the program will incur an error (ENXIO if you have O_NONBLOCK flag set).
I'm trying to write a simple daemon in Linux, which will create a FIFO, then collect anything written to the FIFO and write that data to a file at a later time.
My expectations are that once my daemon has created the FIFO, I can do "echo text > /myfifo" repeatedly. When I'm done, I can do "echo quit > /myfifo" and my program will exit and write all data to disk.
I'm currently using poll() to know when there's more data on the FIFO. This works fine until after the first time I echo data to the FIFO. The data is echoed fine, but my poll continuously returns SIGHUP after that.
Do I need to reset (or close & reopen) the FIFO after each process writes to it?
Pseudo-code of my code looks like this:
ret = fifo(my_fifo, mode);
fd = open(my_fifo, O_RDONLY | O_NONBLOCK);
polling.fd = fd;
polling.events = POLLIN | POLLPRI;
do {
ret = poll(&polling, 1, -1);
amt = read(fd, buf, bufsize);
// do stuff
} while (!done);
You have to keep reopening the FIFO, I think. I have a program that monitors a FIFO, and the monitor loop is:
/* Implement monitor mode */
void sql_monitor(char *fifo)
{
if (chk_fifo(fifo) != 0)
cmd_error(E_NOTFIFO, fifo);
/* Monitor -- device is assumed to be a FIFO */
while (1)
{
ctxt_newcontext();
if (ctxt_setinput(fifo) != 0)
sql_exit(1);
sql_file();
ctxt_endcontext();
}
}
The ctxt_newcontext() function stashes the current I/O state; the ctxt_setinput() function sets the input file to the named file - a FIFO in this case. The sql_file() function reads from the file (FIFO) until the end is reached - the file is closed. The ctxt_endcontext() undoes what ctxt_newcontext() does. The process repeats... This code has been around since about 1990.
So, YES, you will need to keep closing and reopening the FIFO after reading the end of the file (after each process such as echo finishes writing to the FIFO).
(You should also notice that there really isn't a need to poll the FIFO unless there's something else for the process to be doing when there is no data. The read() call will wait until there is data before returning.)