I am working with pipes and one pipe won't open, even though mkfifo() was successful.
I have this:
/* create the FIFO (named pipe) */
int ret_mk = mkfifo(out_myfifo, 0666);
if(ret_mk < 0) {
perror(out_myfifo);
unlink(out_myfifo);
return -1;
}
printf("ret_mk = %d\n", ret_mk);
/* write to the FIFO */
out_fd = open(out_myfifo, O_WRONLY);
printf("out_fd = %d\n", out_fd);
but nothing gets printed after open(), even a print of random text won't show up.
From here we have:
The open() function returns an integer value, which is used to refer to the file. If unsuccessful, it returns -1, and sets the global variable errno to indicate the error type.
What can I do to see why it won't open?
Read fifo(7). For FIFOs, an open call may block. To make open(2) non-blocking, use O_NONBLOCK in the flag argument:
out_fd = open(out_myfifo, O_WRONLY|O_NONBLOCK);
if (out_fd<0) perror(out_myfifo);
printf("%d\n", out_fd);
But usually you want a blocking open for write on a FIFO, because some other process should open the same FIFO for reading (and you want your writing process to wait that to happen).
Notice that there is no way to poll(2) the event that someone else has opened the other end of your FIFO (because poll wants an opened file descriptor). See also inotify(7); you could also want to use unix(7) sockets.
BTW, you could also use strace(1) for debugging purposes.
See also intro(2) and Advanced Linux Programming.
Related
I'm trying to understand the difference between select() and poll() better. For this I tried to implement a simple program that will open a file as write-only, add its file descriptor to the read set and than execute select in hopes that the function will block until the read permission is granted.
As this didnt work (and as far as I understood, this is intended behaviour) I tried to block access to the file using flock before the select() executen. Still, the program did not block its execution.
My sample code is as follows:
#include <stdio.h>
#include <poll.h>
#include <sys/file.h>
#include <errno.h>
#include <sys/select.h>
int main(int argc, char **argv)
{
printf("[+] Select minimal example\n");
int max_number_fds = FOPEN_MAX;
int select_return;
int cnt_pollfds;
struct pollfd pfds_array[max_number_fds];
struct pollfd *pfds = pfds_array;
fd_set fds;
int fd_file = open("./poll_text.txt", O_WRONLY);
struct timeval tv;
tv.tv_sec = 10;
tv.tv_usec = 0;
printf("\t[+] Textfile fd: %d\n", fd_file);
//create and set fds set
FD_ZERO(&fds);
FD_SET(fd_file, &fds);
printf("[+] Locking file descriptor!\n");
if(flock(fd_file,LOCK_EX) == -1)
{
int error_nr = errno;
printf("\t[+] Errno: %d\n", error_nr);
}
printf("[+] Executing select()\n");
select_return = select(fd_file+1, &fds, NULL, NULL, &tv);
if(select_return == -1){
int error_nr = errno;
printf("[+] Select Errno: %d\n", error_nr);
}
printf("[+] Select return: %d\n", select_return);
}
Can anybody see my error in this code? Also: I first tried to execute this code with two FDs added to the read list. When trying to lock them I had to use flock(fd_file,LOCK_SH) as I cannot exclusively lock two FDs with LOCK_EX. Is there a difference on how to lock two FDs of the same file (compared to only one fd)
I'm also not sure why select will not block when a file, that is added to the Read-set is opened as Write-Only. The program can never (without a permission change) read data from the fd, so in my understanding select should block the execution, right?
As a clarification: My "problem" I want to solve is that I have to check if I'm able to replace existing select() calls with poll() (existing in terms of: i will not re-write the select() call code, but will have access to the arguments of select.). To check this, I wanted to implement a test that will force select to block its execution, so I can later check if poll will act the same way (when given similar instructions, i.e. the same FDs to check).
So my "workflow" would be: write tests for different select behaviors (i.e. block and not block), write similar tests for poll (also block, not block) and check if/how poll can be forced do exactly what select is doing.
Thank you for any hints!
When select tells you that a file descriptor is ready for reading, this doesn't necessarily mean that you can read data. It only means that a read call will not block. A read call will also not block when it returns an EOF or error condition.
In your case I expect that read will immediately return -1 and set errno to EBADF (fd is not a valid file descriptor or is not open for reading) or maybe EINVAL (fd is attached to an object which is unsuitable for reading...)
Edit: Additional information as requested in a comment:
A file can be in a blocking state if a physical operation is needed that will take some time, e.g. if the read buffer is empty and (new) data has to be read from the disk, if the file is connected to a terminal and the user has not yet entered any (more) data or if the file is a socket or a pipe and a read would have to wait for (new) data to arrive...
The same applies for write: If the send buffer is full, a write will block. If the remaining space in the send buffer is smaller than your amount of data, it may write only the part that currently fits into the buffer.
If you set a file to non-blocking mode, a read or write will not block but tell you that it would block.
If you want to have a blocking situation for testing purposes, you need control over the process or hardware that provides or consumes the data. I suggest to use read from a terminal (stdin) when you don't enter any data or from a pipe where the writing process does not write any data. You can also fill the write buffer on a pipe when the reading process does not read from it.
Similar to the problem asked a while ago on kernel 3.x, but I'm seeing it on 4.9.37.
The named fifo is created with mkfifo -m 0666. On the read side it is opened with
int fd = open(FIFO_NAME, O_RDONLY | O_NONBLOCK);
The resulting fd is passed into a call to select(). Everything works ok, till I run echo >> <fifo-name>.
Now the fd appears in the read_fds after the select() returns. A read() on the fd will return one byte of data. So far so good.
The next time when select() is called and it returns, the fd still appears in the read_fds, but read() will always return zero meaning with no data. Effectively the read side would consume 100% of the processor capacity. This is exactly the same problem as observed by the referenced question.
Has anybody seen the same issue? And how can it be resolved or worked-around properly?
I've figured out if I close the read end of the fifo, and re-open it again, it will work properly. This probably is ok because we are not sending a lot of data. Though this is not a nice or general work-around.
This is expected behaviour, because the end-of-input case causes a read() to not block; it returns 0 immediately.
If you look at man 2 select, it says clearly that a descriptor in readfds is set if a read() on that descriptor would not block (at the time of the select() call).
If you used poll(), it too would immediately return with POLLHUP in revents.
As OP notes, the correct workaround is to reopen the FIFO.
Because the Linux kernel maintains exactly one internal pipe object to represent each open FIFO (see man 7 fifo and man 7 pipe), the robust approach in Linux is to open another descriptor to the FIFO whenever an end of input is encountered (read() returning 0), and close the original. During the time when both descriptors are open, they refer to the same kernel pipe object, so there is no race window or risk of data loss.
In pseudo-C:
fifoflags = O_RDONLY | O_NONBLOCK;
fifofd = open(fifoname, fifoflags);
if (fifofd == -1) {
/* Error checking */
}
/* ... */
/* select() readfds contains fifofd, or
poll() returns POLLIN for fifofd: */
n = read(fifofd, buffer, sizeof buffer)
if (!n) {
int tempfd;
tempfd = open(fifopath, fifoflags);
if (tempfd == -1) {
const int cause = errno;
close(fifofd);
/* Error handling */
}
close(fifofd);
fifofd = tempfd;
/* A writer has closed the FIFO. */
} else
/* Handling for the other read() result cases */
The file descriptor allocation policy in Linux is such that tempfd will be the lowest-numbered free descriptor.
On my system (Core i5-7200U laptop), reopening a FIFO in this way takes less than 1.5 µs. That is, it can be done about 680,000 times a second. I do not think this reopening is a bottleneck for any sensible scenario, even on low-powered embedded Linux machines.
Assuming a pipe,
int pipe_fd[2];
pipe(pipe_fd);
We fork, and expect that one process will write into the pipe at an arbitrary time. In one of the processes, we want to be able to check the contents of the pipe without blocking.
i.e. While a typical read will block if nothing is present and the write end remains open. I want to go do other stuff and potentially even read a bit at a time, do some stuff, and then check back to see if there's more, a la:
close(pipe_fd[1]);
while(1){
if(/**Check pipe contents**/){
int present_chars = 0;
while( read(pipe_fd[0],&buffer[present_chars],1) != 0)
++present_chars;
//do something
}
else
//do something else
}
Your logic is wrong in that read will not return 0 when it runs out of characters; instead, it will block until it receives more, unless you put the file in non-blocking mode, but then it will return -1 and set errno to EWOULDBLOCK or EAGAIN rather than returning 0. The only time read can ever return 0 is when the size argument was 0 or end-of-file has been reached. And, for pipes, end-of-file means the writing end of the pipe has been closed; end-of-file status does not occur just because there's not any input available yet.
With that said, the simplest way to check is:
if (poll(&(struct pollfd){ .fd = fd, .events = POLLIN }, 1, 0)==1) {
/* data available */
}
but unless you're using nonblocking mode, you'll need to make this check before every single read operation. Passing a larger buffer to read rather than doing it a byte-at-a-time would eliminate most of the cost of checking.
You can check if there is data to be read with the read() function. From read(3):
When attempting to read from an empty pipe or FIFO:
* If some process has the pipe open for writing and
O_NONBLOCK is set, read() shall return -1 and set
errno to [EAGAIN].
* If some process has the pipe open for writing and
O_NONBLOCK is clear, read() shall block the calling
thread until some data is written or the pipe is
closed by all processes that had the pipe open for
writing.
The read() function shall fail if:
EAGAIN or EWOULDBLOCK
The file descriptor is for a socket, is marked
O_NONBLOCK, and no data is waiting to be received.
So if you set O_NONBLOCK, you will be able to tell if something is to be read on the pipe, by simply calling read().
As a reminder, from open(3):
SYNOPSIS
int open(const char *path, int oflag, ... );
DESCRIPTION
Values for oflag are constructed by a
bitwise-inclusive OR of flags from the following
list, defined in <fcntl.h>. Applications shall
specify exactly one of the first three values
(file access modes) below in the value of oflag:
O_NONBLOCK [...]
I hope it helps.
R..'s answer is good however poll returns the number of file descriptor structs that have flags set in "revents". This will be 1 if you can read from fd but will also be 1 if any of the error flags are set. This means R..'s answer will say the pipe is readable if it ever enters an error state. A more robust check could be something like this:
bool canReadFromPipe(){
//file descriptor struct to check if POLLIN bit will be set
//fd is the file descriptor of the pipe
struct pollfd fds{ .fd = fd, .events = POLLIN };
//poll with no wait time
int res = poll(&fds, 1, 0);
//if res < 0 then an error occurred with poll
//POLLERR is set for some other errors
//POLLNVAL is set if the pipe is closed
if(res < 0||fds.revents&(POLLERR|POLLNVAL))
{
//an error occurred, check errno
}
return fds.revents&POLLIN;
}
I just want to know what is the behavior of having two consecutive close on a fd.
e.g.-
close(fd);
close(fd);
[fd is an int]
The first call should return 0; the second call should return -1, and set errno to EBADF.
You should prevent the second call from happening by setting fd to a known bad number, e.g. a -1 immediately after the first call of close, and subsequently checking fd before making the second call (and not making the call if fd is -1):
close(fd);
fd = -1;
...
// More code
...
if (fd != -1) {
close(fd)
fd = -1;
}
This code pattern will help when you need to make calls to close from multiple places, but you are not sure if the file is open, or if it has been closed already. Passing -1 to close is harmless (you would get an EBADF, of course).
It should be harmless unless you're threaded or doing something between the two calls to close. Then you might end up closing an fd that something else in your program has opened.
The way threading is relevant is that libraries almost always do weird things behind your back. Libc will open files for looking up error messages or other locale dependent stuff, the resolver can open configuration files, etc. If you close a file descriptor and close it again, in a threaded environment you can easily end up in a situation where the file descriptor has been reused by a library and you close it behind its back.
Closing the same fd twice should be non-fatal as others have pointed out, but beware of code like this
close(fd);
/* ... */
newfd = open(....);
/* ... */
close(fd);
By the second close, you can't be sure if fd is actually the same as newfd! This would lead to crashes whenever you tried to use newfd.
So (if there's code between both close calls) it's not safe to do this. Always closefile descriptors exactly once. Always free buffers exactly once.
Second call will fail with Errno: EBADF when
because by then, fd is not an active file descriptor.
It should have no effect at all on execution. However, if any error number was set by the first close, that will be lost, so you shouldn't close the file-descriptor twice.
If the value of fd remains the same the second call will return an error that the fd is not valid (EBADF - as dasblinkenlight pointed out)
Think of doing somthing likg
if fd != -1 )
{
close (fd );
fd = -1;
}
I have created a following program in which I wish to poll on the file descriptor of the file that I am opening in the program.
#define FILE "help"
int main()
{
int ret1;
struct pollfd fds[1];
ret1 = open(FILE, O_CREAT);
fds[0].fd = ret1;
fds[0].events = POLLIN;
while(1)
{
poll(fds,1,-1);
if (fds[0].revents & POLLIN)
printf("POLLING");
}
return 0;
}
It is going in infinite loop. I am expecting to run the loop when some operation happen to the file. (Its a ASCII file)
plz help
poll() actually doesn't work on opened files. Since a read() on a file will never block, poll() will always return that you can read non-blocking from the file.
This would (almost) work on character devices*, named pipes** or sockets, though, since those block when you read() from them when there is no data available. (you also need to actually read that data then, or else poll will tell again and again that data is available)
To "poll" a growing/shrinking file, see man inotify or implement your own polling using fstat() in a loop.
* block devices are a story apart; while technically a read from a harddisk can block for 10 ms or longer, this is not perceived as blocking I/O in linux.
** see also how to flush a named pipe using bash
No idea if this is the cause of your problems (probably not), but it is a particularly bad idea to redefine the standard macro FILE.
Didn't your compiler complain about this?