I would like to know what happens with the O_NONBLOCK flag when I use the given file_des in a function. Does it keep the set flag, or not?
If not, should I reset it inside function? Is there any other way?
main()
{
int file_des;
fcntl(file_des, F_SETFD, O_NONBLOCK);
function(file_des);
}
function(int file_des)
{
//do something with file_des
//What happens with the O_NONBLOCK flag?
}
File descriptors are process-wide. When used in a function, or a thread, they always work the same way. That way is controlled by status flags. In Linux, there are five status flags:
O_APPEND: Causes all writes to occur at the end of the file, ignoring file position.
O_ASYNC: A signal is generated when read or write is possible; only available for terminals, pseudoterminals, sockets, pipes, and FIFOs. (I do seem to recall it is available also for some character devices, but I have not verified which ones, if any; the man pages do not say.)
O_DIRECT: Skip page cache for I/O. Complicated, with many constraints; do not use except in very limited special circumstances.
O_NOATIME: Do not update last access time.
O_NONBLOCK: Non-blocking I/O. Instead of waiting (blocking) when data is not immediately available, or cannot be immediately sent, return a short count. If nothing can be sent or received, read()/write() etc. return -1 with errno == EWOULDBLOCK.
O_NONBLOCK has no effect on normal files or block devices.
You modify these by setting the new set of status flags using fcntl(fd, F_SETFL, flags), with zero or more flags OR'ed together. (To disable all, use zero.)
The fcntl(fd, F_SETFD, dflags) sets the set of file descriptor flags. Currently, there is only one such flag, O_CLOEXEC, which causes the descriptor to be automatically closed when an execve() or other exec family of functions succeeds (including popen() and all others that fork and execute a new process). O_CLOEXEC is normally used as a flag for the open() call, though, to avoid the race window against another thread doing a fork() in between.
When you use open(filename, flags) or open(filename, flags, mode), the flags argument is a combination of access mode (O_RDONLY, O_WRONLY, or O_RDWR; exactly one must be used), file creation flags (including file descriptor flags), and file status flags, OR'ed together. (Except for O_ASYNC, which cannot be specified at open() time, and must be set afterwards using fcntl(fd, F_SETFL, flags | O_ASYNC).)
Related
I would like to create blocking and non-blocking file in Unix's C. First, blocking:
fd = open("file.txt", O_CREAT | O_WRONLY | O_EXCL);
is that right? Shouldnt I add some mode options, like 0666 for example?
How about non-blocking file? I have no idea for this.
I would like to achieve something like:
when I open it to write in it, and it's opened for writing, it's ok; if not it blocks.
when I open it to read from it, and it's opened for reading, it's ok; if not it blocks.
File descriptors are blocking or non-blocking; files are not. Add O_NBLOCK to the options in the open() call if you want a non-blocking file descriptor.
Note that opening a FIFO for reading or writing will block unless there's a process with the FIFO open for the other operation, or you specify O_NBLOCK. If you open it for read and write, the open() is non-blocking (will return promptly); I/O operations are still controlled by whether you set O_NBLOCK or not.
The updated question is not clear. However, if you're looking for 'exclusive access to the file' (so that no-one else has it open), then neither O_EXCL nor O_NBLOCK is the answer. O_EXCL affects what happens when you create the file; the create will fail if the file already exists. O_NBLOCK affects whether a read() operation will block when there's no data available to read. If you read the POSIX open() description, there is nothing there that allows you to request 'exclusive access' to a file.
To answer the question about file mode: if you include O_CREAT, you need the third argument to open(). If you omit O_CREAT, you don't need the third argument to open(). It is a varargs function:
int open(const char *filename, int options, ...);
I don't know what you are calling a blocking file (blocking IO in Unix means that the IO operations wait for the data to be available or for a sure failure, they are opposed to non-blocking IO which returns immediately if there is no available data).
You always need to specify a mode when opening with O_CREAT.
The open you show will fails if the file already exists (when fixed for the above point).
Unix has no standard way to lock file for exclusive access excepted that. There are advisory locks (but all programs must respect the protocol). Some have mandatory lock extension. The received wisdom is not to rely on either kind of locking when accessing network file system.
Shouldn't I add some mode options?
You should, if the file is write-only and to be created if nonexistent. In this case, open() expects a third argument as well, so omitting it results in undefined behavior.
Edit:
The updated question is even more confusing...
when I open it to write in it, and it's opened for writing, it's ok; if not it blocks.
Why would you need that? See, if you try to write to a file/file descriptor not opened for writing, write() will return -1 and you can check the error code stored in errno. Tell us what you're trying to achieve by this bizarre thing you want instead of overcomplicating and messing up your code.
(Remarks in parentheses:
I would like to create blocking and non-blocking file
What's that?
in unix's C
Again, there's no such thing. There is the C language, which is platform-independent.)
I have a sample application of a SIP server listening on both tcp and udp ports 5060.
At some point in the code, I do a system("pppd file /etc/ppp/myoptions &");
After this if I do a netstat -apn, It shows me that ports 5060 are also opened for pppd!
Is there any method to avoid this? Is this standard behaviour of the system function in Linux?
Thanks,
Elison
Yes, by default whenever you fork a process (which system does), the child inherits all the parent's file descriptors. If the child doesn't need those descriptors, it SHOULD close them. The way to do this with system (or any other method that does a fork+exec) is to set the FD_CLOEXEC flag on all file descriptors that shouldn't be used by the children of you process. This will cause them to be closed automatically whenever any child execs some other program.
In general, ANY TIME your program opens ANY KIND of file descriptor that will live for an extended period of time (such as a listen socket in your example), and which should not be shared with children, you should do
fcntl(fd, F_SETFD, fcntl(fd, F_GETFD) | FD_CLOEXEC);
on the file descriptor.
As of the 2016? revision of POSIX.1, you can use the SOCK_CLOEXEC flag or'd into the type of the socket to get this behavior automatically when you create the socket:
listenfd = socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, 0);
bind(listenfd, ...
listen(listemfd, ...
which guarentees it will be closed properly even if some other simultaneously running thread does a system or fork+exec call. Fortunately, this flag has been supported for awhile on Linux and BSD unixes (but not OSX, unfortunately).
You should probably avoid the system() function altogether. It's inherently dangerous, in that it invokes the shell, which can be tampered with and rather non-portable, even between Unicies.
What you should do is the fork()/exec() dance. It goes something like this
if(!fork()){
//close file descriptors
...
execlp("pppd", "pppd", "file", "/etc/ppp/myoptions", NULL);
perror("exec");
exit(-1);
}
Yes, this is standard behavior of fork() in Linux, from which system() is implemented.
The identifier returned from the socket() call is a valid file descriptor. This value is usable with file-oriented functions such as read(), write(), ioctl(), and close().
The converse, that every file descriptor is a socket, is not true. One cannot open a regular file with open() and pass that descriptor to, e.g., bind() or listen().
When you call system() the child process inherits the same file descriptors as the parent. This is how stdout (0), stdin (1), and stderr (2) are inherited by child processes. If you arrange to open a socket with a file descriptor of 0, 1 or 2, the child process will inherit that socket as one of the standard I/O file descriptors.
Your child process is inheriting every open file descriptor from the parent, including the socket you opened.
As others have stated, this is standard behavior that programs depend on.
When it comes to preventing it you have a few options. Firstly is closing all file descriptors after the fork(), as Dave suggests. Second, there is the POSIX support for using fcntl with FD_CLOEXEC to set a 'close on exec' bit on a per-fd basis.
Finally, though, since you mention you are running on Linux, there are a set of changes designed to let you set the bit right at the point of opening things. Naturally, this is platform dependent. An overview can be found at http://udrepper.livejournal.com/20407.html
What this means is that you can use a bitwise or with the 'type' in your socket creation call to set the SOCK_CLOEXEC flag. Provided you're running kernel 2.6.27 or later, that is.
system() copies current process and then launch a child on top of it. (current process is no more there. that is probably why pppd uses 5060. You can try fork()/exec() to create a child process and keep parent alive.
I'm seeing such code in nginx:
if(fcntl(ngx_processes[s].channel[0], F_SETFL, fcntl(s, F_GETFL) | O_NONBLOCK) == -1) {
...
if (ioctl(ngx_processes[s].channel[0], FIOASYNC, &on) == -1) {
...
Anyone can tell me what's the difference between fcntl(s, F_SETFL, fcntl(s, F_GETFL) | O_NONBLOCK) and ioctl(s, FIOASYNC, &on) ,aren't async and nonblocking the same thing??
FIOASYNC toggles the O_ASYNC flag (which is usually set in open(2) or fcntl(2)) for a file descriptor, which will ask the kernel to send SIGIO or SIGPOLL to the process when the file descriptor is ready for IO.
O_ASYNC is not used often:
it is extremely difficult to properly handle IO in signal handlers; they are best left as tiny as possible
because signals interrupt the control flow of the program, they 'cost more' to run than standard system calls, such as select(2) or poll(2)
signals provide less information than other calls: they only report one fd ready vs many fds that might be ready.
The O_NONBLOCK doesn't provide any notification to the user process that a fd is ready for read(2) or write(2) -- instead, it changes the behavior of read(2) and write(2) and similar calls to return immediately if the file descriptor isn't ready for reading or writing. O_NONBLOCK is typically used in conjunction with select(2) or poll(2) or similar calls to guarantee that the main loop of a client or server won't block on one specific peer, and thus starve all its peers.
Like so:
if (fcntl(fd, F_SETFD, FD_CLOEXEC) == -1) {
...
Though I've read man fcntl, I can't figure out what it does.
It sets the close-on-exec flag for the file descriptor, which causes the file descriptor to be automatically (and atomically) closed when any of the exec-family functions succeed.
It also tests the return value to see if the operation failed, which is rather useless if the file descriptor is valid, since there is no condition under which this operation should fail on a valid file descriptor.
It marks the file descriptor so that it will be close()d automatically when the process or any children it fork()s calls one of the exec*() family of functions. This is useful to keep from leaking your file descriptors to random programs run by e.g. system().
Note that the use of this flag is essential in some multithreaded programs, because using a separate fcntl(2) F_SETFD operation to set the FD_CLOEXEC flag does not suffice to avoid race conditions where one thread opens a file descriptor and attempts to set its close-on-exec flag using fcntl(2) at the same time as another thread does a fork(2) plus execve(2). Depending on the order of execution, the race may lead to the file descriptor returned by open() being unintentionally leaked to the program executed by the child process created by fork(2).
(This kind of race is, in principle, possible for any system call that creates a file descriptor whose close-on-exec flag should be set, and various other Linux system calls provide an equivalent of the O_CLOEXEC flag to deal with this problem.)
I have a file descriptor that is open for reading which may be non-blocking. What I need to do is simply read all data until reaching EOF and write that data to a writable file descriptor. As the process that performs this copying is not "aware" of anything that is going on around it, I don't think that I can do something useful while waiting for data, and I don't want to use a while loop (while errno is not EAGAIN) because I think that it would be wasteful. Is there a way to block or otherwise suspend execution of the copying process until data becomes available?
Your other answer simply unsets O_NONBLOCK, which sets the file back to blocking. That's perfectly fine, if that works in your situation; but if not, you can use select() to block until your non-blocking file descriptor is readable.
Chapter 7 of the Linux SCSI Generic (sg) HOWTO gives an example of how to do this:
int flags = fcntl(fd, F_GETFL);
fcntl(fd, F_SETFL, flags & (~O_NONBLOCK));