By default a POSIX FIFO is opened in blocking mode. My problem is that when I open it in blocking mode it just freezes (blocks) and nothing else happens.
Originally I opened both sides with the RDWR flag and I had no problems because RDWR makes it nonblocking because "Under Linux, opening a FIFO for read and write will succeed both in blocking and nonblocking mode" (https://linux.die.net/man/7/fifo). But in nonblocking mode I sometimes get dropped records, so I need to open it in blocking mode.
Here's how I call mkfifo:
int64_t fifo_setup(char * fifo_name)
{
if (mkfifo(fifo_name, S_IWUSR | S_IRUSR | S_IRGRP | S_IROTH) == -1) {
perror("mkfifo");
exit(EXIT_FAILURE); }
return 0; }
My original call to open:
int64_t fifo_open(char * fifo_name, int64_t read_write) {
int c;
if (read_write == 1) {
c = open(fifo_name, O_RDWR);}
if (read_write == 2) {
c = open(fifo_name, O_RDWR);}
perror("open");
return (c);
}
Then I changed it to:
int64_t fifo_open(char * fifo_name, int64_t read_write) {
int c;
if (read_write == 1) {
c = open(fifo_name, O_WRONLY);}
if (read_write == 2) {
c = open(fifo_name, O_RDONLY);}
perror("open");
return (c);
}
My program reaches the open call for the writer threads (this is a NASM call to C):
Open_FIFO_Write:
lea rdi,[fifo_name]
mov rsi,1
call fifo_open wrt ..plt
mov [fifo_write_fd],rax
But then the Putty terminal freezes (blocks) and nothing else happens. I never get to the block that opens the reading side. Specifically it blocks just before it reaches perror("open") in the fifo_open code above:
According to the Linux man page at https://www.man7.org/linux/man-pages/man7/fifo.7.html, "Normally, opening the FIFO blocks until the other end is opened also." That's great but I can't get to the code that opens the writer because the reader blocks all further progress. It's like a Mexican standoff.
Of course there must be a way but my research hasn't found it yet.
Thanks for any help on this.
Three options:
FIFOs themselves don't drop messages in nonblocking mode. If your code is dropping messages, fix your code to stop doing that, and then just use nonblocking mode.
Open the FIFO in nonblocking mode, then once you get both ends open, use fcntl to change it to blocking mode.
Use a separate thread or process to open each half of the FIFO.
Related
I am creating three FIFO pipes for a single process where multiple writer threads write to a single reader thread. The goal is a separate file descriptor for each writer thread with a corresponding reader file descriptor for each writer file descriptor so that I know which writer thread belongs to each record. My setup works fine with one writer file descriptor shared among all writer threads and a single reader file descriptor.
The multiple reader/writer setup works successfully to set up three writer fds (6, 7, 8) and three reader fds (9, 10 and 11). All writer threads successfully send data, BUT the reader fds do not successfully read the data (return zero bytes read).
When I open the fd on the writer side, I get the error message "open: Bad file descriptor" but it still returns a unique fd, and that fd is used successfully to write, but not to read.
I call the C programs from NASM in a loop, one iteration for each writer thread:
xor r12,r12
mov r13,[Number_Of_Cores_Calc]
Setup_pipe:
lea rdi,[fifo_base_name]
mov rax,r12
add rax,48
mov byte[rdi+11],al
call fifo_delete wrt ..plt
call fifo_setup wrt ..plt
push rdi
call fifo_close wrt ..plt
pop rdi
Open_FIFO_Write:
mov rsi,1
call fifo_open wrt ..plt
lea rbp,[fifo_write_fds]
mov [rbp+r12*8],rax
add r12,1
cmp r12,r13
jl Setup_pipe
The corresponding C programs:
int64_t fifo_setup(char * fifo_name)
{
remove(fifo_name);
if (mkfifo(fifo_name, S_IWUSR | S_IRUSR | S_IRGRP | S_IROTH) == -1) {
perror("mkfifo");
exit(EXIT_FAILURE); }
return 0;
}
int64_t fifo_open(char * fifo_name, int64_t read_write) {
int c;
if (read_write == 1) {
c = open(fifo_name, O_CREAT | O_RDWR);} //O_WRONLY
if (read_write == 2) {
c = open(fifo_name, O_CREAT | O_RDWR);} //O_RDONLY
perror("open");
return (c);
}
int64_t fifo_read(int fd, void *buf, size_t count) {
int status_read = read(fd, buf, count);
return status_read;
}
int64_t fifo_write(int fd, const void *buf, size_t count) {
int status_write = write(fd, buf, count);
return status_write; }
int64_t fifo_close(int fifo_fdes) {
close(fifo_fdes);
return 0; }
// Delete pre-existing file
int64_t fifo_delete(char * fifo_name) {
if( access( fifo_name, F_OK ) != -1 ) {
if (remove(fifo_name) == 0)
printf("File deleted successfully");
else
printf("Unable to delete the file");
}
return 0; }
So my questions are:
Can I write from multiple threads to a single reader thread using separate fds for each thread on both the write and read sides?
If I can, what did I do wrong above -- especially why am I getting "open: Bad file descriptor" but I still get what looks like a valid file descriptor.
If I can't use POSIX FIFOs, what IPC methods can I use so each writer thread has its own unique fd on both writer and reader sides? Datagram sockets?
Thanks for any help on this.
perror("open");
You cannot simply call perror() as it can return the previous error.
You have to check whether your syscall indeed generated an error. In case of open() you should do:
if (c < 0) {
perror("open");
}
If all the communication is going to be between threads in a single process, you can use pipe(2) system call and give fd[0] to the reader thread while giving fd[1] to the writer. You don't even need to open() or have a name in the filesytem associated to the fifo. You still will need to close fd[1] if you want to signal end of file to the reader.
I have a USB serial port module and I want to use it in my C application. The problem is the open system call doesn't return!
int serialDevice = open("/dev/ttyUSB0", O_RDWR | O_NOCTTY); // doesn't return
I added my user to dialout group and I can access the port without root access using minicom in terminal. So it's not permission problem. Even if it is, it should return instantly and give permission error. But open function doesn't return at all.
What should I do? Do you have any idea?
Thanks
I believe this can happen if the device is busy, etc..
You need to add the flag O_NONBLOCK:
O_NONBLOCK
When opening a FIFO with O_RDONLY or O_WRONLY set:
*
If O_NONBLOCK is set, an open() for reading-only shall return without delay. An open() for writing-only shall return an error if no
process currently has the file open for reading.
*
If O_NONBLOCK is clear, an open() for reading-only shall block the calling thread until a thread opens the file for writing. An open()
for writing-only shall block the calling thread until a thread opens
the file for reading.
When opening a block special or character special file that supports
non-blocking opens:
*
If O_NONBLOCK is set, the open() function shall return without blocking for the device to be ready or available. Subsequent behavior
of the device is device-specific.
*
If O_NONBLOCK is clear, the open() function shall block the calling thread until the device is ready or available before
returning.
Otherwise, the behavior of O_NONBLOCK is unspecified.
int serialDevice = open("/dev/ttyUSB0", O_RDWR | O_NOCTTY);
In addition to #darune's answer and O_NONBLOCK, you might consider O_SYNC too. Also see How to open, read, and write from serial port in C?
You might also consider making the file descriptor exclusive so another program like Modem Manager does not open the device and muck with your state. Exclusive is fine because of O_RDWR. Also see How to make /dev/ttyACM0 (and friends) exclusive? on the Kernel Newbies mailing list.
To make the file descriptor exclusive you need to use ioctl and TIOCEXCL. O_EXCL does not work as expected because it is not honored for character devices (and the kernel folks say -ENOPATCH).
int term_config(int fd, int speed)
{
struct termios tty;
memset(&tty, 0, sizeof(tty));
if (tcgetattr(fd, &tty) != 0) {
log_error("term_config: tcgetattr: %s\n", strerror(errno));
return -1;
}
cfmakeraw(&tty);
tty.c_cflag |= CLOCAL; /* ignore status lines */
tty.c_cflag |= CRTSCTS; /* hardware flow control */
cfsetospeed(&tty,(speed_t)speed);
cfsetispeed(&tty,(speed_t)speed);
if (tcsetattr(fd, TCSANOW, &tty) != 0) {
log_error("term_config: tcsetattr: %s\n", strerror(errno));
return -1;
}
if (ioctl(fd, TIOCEXCL, NULL) != 0) {
log_error("term_config: ioctl_tty: %s\n", strerror(errno));
return -1;
}
return 0;
}
You would call term_config something like:
int serialDevice = open("/dev/ttyUSB0", ...);
if (serialDevice == -1) { /* error */ }
int result = term_config(serialDevice, B115200);
if (result != 0) { /* error */ }
Similar to the problem asked a while ago on kernel 3.x, but I'm seeing it on 4.9.37.
The named fifo is created with mkfifo -m 0666. On the read side it is opened with
int fd = open(FIFO_NAME, O_RDONLY | O_NONBLOCK);
The resulting fd is passed into a call to select(). Everything works ok, till I run echo >> <fifo-name>.
Now the fd appears in the read_fds after the select() returns. A read() on the fd will return one byte of data. So far so good.
The next time when select() is called and it returns, the fd still appears in the read_fds, but read() will always return zero meaning with no data. Effectively the read side would consume 100% of the processor capacity. This is exactly the same problem as observed by the referenced question.
Has anybody seen the same issue? And how can it be resolved or worked-around properly?
I've figured out if I close the read end of the fifo, and re-open it again, it will work properly. This probably is ok because we are not sending a lot of data. Though this is not a nice or general work-around.
This is expected behaviour, because the end-of-input case causes a read() to not block; it returns 0 immediately.
If you look at man 2 select, it says clearly that a descriptor in readfds is set if a read() on that descriptor would not block (at the time of the select() call).
If you used poll(), it too would immediately return with POLLHUP in revents.
As OP notes, the correct workaround is to reopen the FIFO.
Because the Linux kernel maintains exactly one internal pipe object to represent each open FIFO (see man 7 fifo and man 7 pipe), the robust approach in Linux is to open another descriptor to the FIFO whenever an end of input is encountered (read() returning 0), and close the original. During the time when both descriptors are open, they refer to the same kernel pipe object, so there is no race window or risk of data loss.
In pseudo-C:
fifoflags = O_RDONLY | O_NONBLOCK;
fifofd = open(fifoname, fifoflags);
if (fifofd == -1) {
/* Error checking */
}
/* ... */
/* select() readfds contains fifofd, or
poll() returns POLLIN for fifofd: */
n = read(fifofd, buffer, sizeof buffer)
if (!n) {
int tempfd;
tempfd = open(fifopath, fifoflags);
if (tempfd == -1) {
const int cause = errno;
close(fifofd);
/* Error handling */
}
close(fifofd);
fifofd = tempfd;
/* A writer has closed the FIFO. */
} else
/* Handling for the other read() result cases */
The file descriptor allocation policy in Linux is such that tempfd will be the lowest-numbered free descriptor.
On my system (Core i5-7200U laptop), reopening a FIFO in this way takes less than 1.5 µs. That is, it can be done about 680,000 times a second. I do not think this reopening is a bottleneck for any sensible scenario, even on low-powered embedded Linux machines.
I am working with pipes and one pipe won't open, even though mkfifo() was successful.
I have this:
/* create the FIFO (named pipe) */
int ret_mk = mkfifo(out_myfifo, 0666);
if(ret_mk < 0) {
perror(out_myfifo);
unlink(out_myfifo);
return -1;
}
printf("ret_mk = %d\n", ret_mk);
/* write to the FIFO */
out_fd = open(out_myfifo, O_WRONLY);
printf("out_fd = %d\n", out_fd);
but nothing gets printed after open(), even a print of random text won't show up.
From here we have:
The open() function returns an integer value, which is used to refer to the file. If unsuccessful, it returns -1, and sets the global variable errno to indicate the error type.
What can I do to see why it won't open?
Read fifo(7). For FIFOs, an open call may block. To make open(2) non-blocking, use O_NONBLOCK in the flag argument:
out_fd = open(out_myfifo, O_WRONLY|O_NONBLOCK);
if (out_fd<0) perror(out_myfifo);
printf("%d\n", out_fd);
But usually you want a blocking open for write on a FIFO, because some other process should open the same FIFO for reading (and you want your writing process to wait that to happen).
Notice that there is no way to poll(2) the event that someone else has opened the other end of your FIFO (because poll wants an opened file descriptor). See also inotify(7); you could also want to use unix(7) sockets.
BTW, you could also use strace(1) for debugging purposes.
See also intro(2) and Advanced Linux Programming.
I'm trying to write a simple daemon in Linux, which will create a FIFO, then collect anything written to the FIFO and write that data to a file at a later time.
My expectations are that once my daemon has created the FIFO, I can do "echo text > /myfifo" repeatedly. When I'm done, I can do "echo quit > /myfifo" and my program will exit and write all data to disk.
I'm currently using poll() to know when there's more data on the FIFO. This works fine until after the first time I echo data to the FIFO. The data is echoed fine, but my poll continuously returns SIGHUP after that.
Do I need to reset (or close & reopen) the FIFO after each process writes to it?
Pseudo-code of my code looks like this:
ret = fifo(my_fifo, mode);
fd = open(my_fifo, O_RDONLY | O_NONBLOCK);
polling.fd = fd;
polling.events = POLLIN | POLLPRI;
do {
ret = poll(&polling, 1, -1);
amt = read(fd, buf, bufsize);
// do stuff
} while (!done);
You have to keep reopening the FIFO, I think. I have a program that monitors a FIFO, and the monitor loop is:
/* Implement monitor mode */
void sql_monitor(char *fifo)
{
if (chk_fifo(fifo) != 0)
cmd_error(E_NOTFIFO, fifo);
/* Monitor -- device is assumed to be a FIFO */
while (1)
{
ctxt_newcontext();
if (ctxt_setinput(fifo) != 0)
sql_exit(1);
sql_file();
ctxt_endcontext();
}
}
The ctxt_newcontext() function stashes the current I/O state; the ctxt_setinput() function sets the input file to the named file - a FIFO in this case. The sql_file() function reads from the file (FIFO) until the end is reached - the file is closed. The ctxt_endcontext() undoes what ctxt_newcontext() does. The process repeats... This code has been around since about 1990.
So, YES, you will need to keep closing and reopening the FIFO after reading the end of the file (after each process such as echo finishes writing to the FIFO).
(You should also notice that there really isn't a need to poll the FIFO unless there's something else for the process to be doing when there is no data. The read() call will wait until there is data before returning.)