I have a file descriptor that is open for reading which may be non-blocking. What I need to do is simply read all data until reaching EOF and write that data to a writable file descriptor. As the process that performs this copying is not "aware" of anything that is going on around it, I don't think that I can do something useful while waiting for data, and I don't want to use a while loop (while errno is not EAGAIN) because I think that it would be wasteful. Is there a way to block or otherwise suspend execution of the copying process until data becomes available?
Your other answer simply unsets O_NONBLOCK, which sets the file back to blocking. That's perfectly fine, if that works in your situation; but if not, you can use select() to block until your non-blocking file descriptor is readable.
Chapter 7 of the Linux SCSI Generic (sg) HOWTO gives an example of how to do this:
int flags = fcntl(fd, F_GETFL);
fcntl(fd, F_SETFL, flags & (~O_NONBLOCK));
Related
I would like to know what happens with the O_NONBLOCK flag when I use the given file_des in a function. Does it keep the set flag, or not?
If not, should I reset it inside function? Is there any other way?
main()
{
int file_des;
fcntl(file_des, F_SETFD, O_NONBLOCK);
function(file_des);
}
function(int file_des)
{
//do something with file_des
//What happens with the O_NONBLOCK flag?
}
File descriptors are process-wide. When used in a function, or a thread, they always work the same way. That way is controlled by status flags. In Linux, there are five status flags:
O_APPEND: Causes all writes to occur at the end of the file, ignoring file position.
O_ASYNC: A signal is generated when read or write is possible; only available for terminals, pseudoterminals, sockets, pipes, and FIFOs. (I do seem to recall it is available also for some character devices, but I have not verified which ones, if any; the man pages do not say.)
O_DIRECT: Skip page cache for I/O. Complicated, with many constraints; do not use except in very limited special circumstances.
O_NOATIME: Do not update last access time.
O_NONBLOCK: Non-blocking I/O. Instead of waiting (blocking) when data is not immediately available, or cannot be immediately sent, return a short count. If nothing can be sent or received, read()/write() etc. return -1 with errno == EWOULDBLOCK.
O_NONBLOCK has no effect on normal files or block devices.
You modify these by setting the new set of status flags using fcntl(fd, F_SETFL, flags), with zero or more flags OR'ed together. (To disable all, use zero.)
The fcntl(fd, F_SETFD, dflags) sets the set of file descriptor flags. Currently, there is only one such flag, O_CLOEXEC, which causes the descriptor to be automatically closed when an execve() or other exec family of functions succeeds (including popen() and all others that fork and execute a new process). O_CLOEXEC is normally used as a flag for the open() call, though, to avoid the race window against another thread doing a fork() in between.
When you use open(filename, flags) or open(filename, flags, mode), the flags argument is a combination of access mode (O_RDONLY, O_WRONLY, or O_RDWR; exactly one must be used), file creation flags (including file descriptor flags), and file status flags, OR'ed together. (Except for O_ASYNC, which cannot be specified at open() time, and must be set afterwards using fcntl(fd, F_SETFL, flags | O_ASYNC).)
I have read the Linux manpage of pipe2, that states that you can use the O_DIRECT flag to create a pipe that performs I/O in packet mode.
I haven't found anything similar for fifos (named pipes) although I have heard that pipes and fifos share a lot of code in Linux, and it could be useful to me in a project (we already pass messages in fifos, but we have to seek for an especial terminator, reading one byte at a time).
Is there anything equivalent to perform fifo I/O in packet mode?
The O_* flags on a file descriptor can usually be changed by fcntl(fd, F_SETFL, ...) but in this case I don't think it will work, because of this:
https://lkml.org/lkml/2015/12/15/480
It's a patch that was submitted just 2 weeks ago to support exactly this use case, and the only replies to it have been a couple of build failures from an automated tester.
So you can try fixing that patch and applying it (should be easy - looks like a typo, the f should be filp)...
or (this is the option I'd prefer) see if your needs can be met by an AF_UNIX, SOCK_SEQPACKET socket instead of a pipe with this new flag. They're more powerful and more portable.
(I assume you already tried passing the O_DIRECT to open when you open the named pipe, since that's the most obvious thing.)
Both pipe & fifo are byte stream, it open a file descriptor, then use read() and write() to do communication.
If u want a packet (I am not sure what u mean, I assume you want to read a block of data without determine the boundary by yourself), POSIX - message queue might be a good choice. It send / receive data in unit of a message instead of byte by byte.
And you can also set priority on the messages which change the order of receiving, if all messages have the same priority (e.g 0), then the send & receive order is the same as FIFO.
If that's what u want, then check man mq_overview for details.
I'm not able to switch to a socket or message queue, and am stuck with using pipe. But good news, the patch refereed to in user2404501's answer eventually got accepted and is in linux v4.5 and newer. So using fnctl() to set O_DIRECT on a named pipe is valid.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0dbf5f20652108106cb822ad7662c786baaa03ff
I want to use named fifo channel and I want to implement a timeout when I write in this fifo.
fd = open(pipe, O_WRONLY);
write(fd, msg, len);
Program is blocked by function open, so using the function select will not work.
Thanks.
use select() and its timeout argument.
Read pipe(7), fifo(7), poll(2)
You might setup a timer or or alarm with a signal handler (see time(7) & signal(7)) before your call to open(2) - but I won't do that - or you could use the O_NONBLOCK flag, since fifo(7) says:
A process can open a FIFO in nonblocking mode. In this case, opening
for read-only will succeed even if no-one has opened on the write
side yet, opening for write-only will fail with ENXIO (no such device
or address) unless the other end has already been opened.
However, you need something (some other process reading) on the other side of the FIFO or pipe.
Perhaps you should consider using unix(7) sockets, i.e. the AF_UNIX address family. It looks more relevant to your case: change your code above (trying to open for writing a FIFO) to a AF_UNIX socket on the client side (with a connect), and change the other process to become an AF_UNIX socket server.
As 5gon12eder commented, you might also look into inotify(7). Or even perhaps D-bus !
I'm guessing that FIFOs or pipes are not the right solution in your situation. You should explain more and give a broader picture of your concerns and goals.
I would like to create blocking and non-blocking file in Unix's C. First, blocking:
fd = open("file.txt", O_CREAT | O_WRONLY | O_EXCL);
is that right? Shouldnt I add some mode options, like 0666 for example?
How about non-blocking file? I have no idea for this.
I would like to achieve something like:
when I open it to write in it, and it's opened for writing, it's ok; if not it blocks.
when I open it to read from it, and it's opened for reading, it's ok; if not it blocks.
File descriptors are blocking or non-blocking; files are not. Add O_NBLOCK to the options in the open() call if you want a non-blocking file descriptor.
Note that opening a FIFO for reading or writing will block unless there's a process with the FIFO open for the other operation, or you specify O_NBLOCK. If you open it for read and write, the open() is non-blocking (will return promptly); I/O operations are still controlled by whether you set O_NBLOCK or not.
The updated question is not clear. However, if you're looking for 'exclusive access to the file' (so that no-one else has it open), then neither O_EXCL nor O_NBLOCK is the answer. O_EXCL affects what happens when you create the file; the create will fail if the file already exists. O_NBLOCK affects whether a read() operation will block when there's no data available to read. If you read the POSIX open() description, there is nothing there that allows you to request 'exclusive access' to a file.
To answer the question about file mode: if you include O_CREAT, you need the third argument to open(). If you omit O_CREAT, you don't need the third argument to open(). It is a varargs function:
int open(const char *filename, int options, ...);
I don't know what you are calling a blocking file (blocking IO in Unix means that the IO operations wait for the data to be available or for a sure failure, they are opposed to non-blocking IO which returns immediately if there is no available data).
You always need to specify a mode when opening with O_CREAT.
The open you show will fails if the file already exists (when fixed for the above point).
Unix has no standard way to lock file for exclusive access excepted that. There are advisory locks (but all programs must respect the protocol). Some have mandatory lock extension. The received wisdom is not to rely on either kind of locking when accessing network file system.
Shouldn't I add some mode options?
You should, if the file is write-only and to be created if nonexistent. In this case, open() expects a third argument as well, so omitting it results in undefined behavior.
Edit:
The updated question is even more confusing...
when I open it to write in it, and it's opened for writing, it's ok; if not it blocks.
Why would you need that? See, if you try to write to a file/file descriptor not opened for writing, write() will return -1 and you can check the error code stored in errno. Tell us what you're trying to achieve by this bizarre thing you want instead of overcomplicating and messing up your code.
(Remarks in parentheses:
I would like to create blocking and non-blocking file
What's that?
in unix's C
Again, there's no such thing. There is the C language, which is platform-independent.)
I have a popen() function which executes tail -f sometextfile. Aslong as there is data in the filestream obviously I can get the data through fgets(). Now, if no new data comes from tail, fgets() hangs. I tried ferror() and feof() to no avail. How can I make sure fgets() doesn't try to read data when nothing new is in the file stream?
One of the suggestion was select(). Since this is for Windows Platform select doesn't seem to work as anonymous pipes do not seem to work for it (see this post).
In Linux (or any Unix-y OS), you can mark the underlying file descriptor used by popen() to be non-blocking.
#include <fcntl.h>
FILE *proc = popen("tail -f /tmp/test.txt", "r");
int fd = fileno(proc);
int flags;
flags = fcntl(fd, F_GETFL, 0);
flags |= O_NONBLOCK;
fcntl(fd, F_SETFL, flags);
If there is no input available, fgets will return NULL with errno set to EWOULDBLOCK.
fgets() is a blocking read, it is supposed to wait until data is available if there is no data.
You'll want to perform asynchronous I/O using select(), poll(), or epoll(). And then perform a read from the file descriptor when there is data available.
These functions use the file descriptor of the FILE* handle, retrieved by: int fd = fileno(f);
i solved my problems by using threads , specifically _beginthread , _beginthreadex.
You can instead try reading sometextfile using low-level IO functions (open(), read(), etc.), like tail itself does. When there's nothing more to read, read() returns zero, but will still try to read more the next time, unlike FILE* functions.
I you would use POSIX functions for IO instead of those of C library, you could use select or poll.