After ignoring SIGPIPE with a call to signal(), there is a possibility that a write() call will fail with errno set equal to EPIPE. Do I still have to close the file descriptor like in the example below?
if (write(sock_fd, buf, len) < 0) {
if (errno == EPIPE) {
close(sock_fd);
}
}
Remember that when you create a pipe, using the system call of the same name, you get two file descriptors: one for reading and one for writing. You get EPIPE from a write operation, on the write fd, when the read fd has been closed. The write fd is still open. If you try to write to it again, you'll get EPIPE again.
(Often, when this happens, the pipe was set up by a shell or some other parent process, and the read fd was never available to your program, but that doesn't matter to the kernel. It was open in some process(es) and now it isn't.)
Since it's still open, you do need to close it. However, exiting automatically closes all fds that are still open. So if the very next thing you'd do after closing the pipe is to exit, then you don't need to bother closing it first. Since it is a pipe, and you already got EPIPE, there can't be any delayed write errors that close might report.
You always have to close file descriptors. No ifs, no buts.
Related
fd = open("file", O_RDONLY);
if (fd < 0) exit(1);
while((res = read(fd, buf, sizeof(buf)))){
if (res < 0){
close(fd);
fprintf(stderr, "Read error!\n");
break;
} else {
printf("Read %zd bytes\n", res);
}
}
close (fd);
In the single-threaded program this is an obvious bug: 'fd' is being closed twice. What could be the impact of this bug in a multithreaded program?
On the one hand, closing an already closed fd will just make your second close() return EBADF (fd isn't a valid open file descriptor). This is harmless.
On the other hand, in a multithreaded program, nothing guarantees you that the fd will not be reused, e.g. another thread is calling open() in the meantime between the two calls to close(). In that situation, your are unexpectedly closing the recently opened file, and this is harmful.
It could be anything. There's no reliable way to predict the effect of "contaminating" shared resources in a multi-threaded program.
The most obvious problem would be that you might close a socket that another thread had opened for some important purpose after your first close but before your second close. But there's no way you can predict what some piece code will do under those circumstances without understanding that code.
As just one example, you might prematurely normally terminate a connection to a remote server, causing the server to process a partial request instead of a complete one. There's no way to know what that partial request might do without understanding the protocol being used.
Here's an even worse scenario:
This thread closes descriptor 4.
A logging thread opens a local file and gets descriptor 4.
This thread closes descriptor 4 again. (The bug.)
Another thread opens a socket connection to an untrusted server and gets descriptor 4.
The logging thread runs again and writes sensitive information to descriptor 4.
Oops. We just sent highly-sensitive information to a random remote connection.
What could be the impact of this bug in a multithreaded program?
99.999% of the time, nothing; the second call to close() will fail with errno set to EBADF and life will go on.
The other .001% of the time, another thread will run and call open() in that exact instant between the executions of the two close() calls in your thread, and the other thread's open() call will return the same file-descriptor value that your fd local variable is set to (because the runtime is allowed to a re-use file-descriptor value after it has been closed)... then your thread will run again, and your second call to close(fd) will close the other thread's socket(!)
At that point, the other thread's networking calls that use that file-descriptor will start erroring out with EBADF (since it is trying to use a now-closed file-descriptor), and you will spend several unhappy hours/days/weeks trying to track down why your program's networking seems to randomly fail once in a blue moon.
When using a pipe for process-process communication, what is the purpose of closing one end of the pipe?
For example: How to send a simple string between two programs using pipes?
Notice that one side of the pipe is closed in the child and parent processes. Why is this required?
If you connect two processes - parent and child - using a pipe, you create the pipe before the fork.
The fork makes the both processes have access to both ends of the pipe. This is not desirable.
The reading side is supposed to learn that the writer has finished if it notices an EOF condition. This can only happen if all writing sides are closed. So it is best if it closes its writing FD ASAP.
The writer should close its reading FD just in order not to have too many FDs open and thus reaching a maybe existing limit of open FDs. Besides, if the then only reader dies, the writer gets notified about this by getting a SIGPIPE or at least an EPIPE error (depending on how signals are defined). If there are several readers, the writer cannot detect that "the real one" went away, goes on writing and gets stuck as the writing FD blocks in the hope, the "unused" reader will read something.
So here in detail what happens:
parent process calls pipe() and gets 2 file descriptors: let's call it rd and wr.
parent process calls fork(). Now both processes have a rd and a wr.
Suppose the child process is supposed to be the reader.
Then
the parent should close its reading end (for not wasting FDs and for proper detection of dying reader) and
the child must close its writing end (in order to be possible to detect the EOF condition).
The number of file descriptors that can be open at a given time is limited. If you keep opening pipes and not closing them pretty soon you'll run out of FDs and can't open anything anymore: not pipes, not files, not sockets, ...
Another reason why it can be important to close the pipe is when the closing itself has a meaning to the application. For example, a common use of pipes is to send the errno from a child process to the parent when using fork and exec to launch an external program:
The parent creates the pipe, calls fork to create a child process, closes its writing end, and tries to read from the pipe.
The child process attempts to use exec to run a different program:
If exec fails, for example because the program does not exist, the child writes errno to the pipe, and the parent reads it and knows what went wrong, and can tell the user.
If exec is successful the pipe is closed without anything being written. The read function in the parent returns 0 indicating the pipe was closed and knows the program was successfully started.
If the parent did not close its writing end of the pipe before trying to read from the pipe this would not work because the read function would never return when exec is successful.
Closing unused pipe file descriptor is more than a matter of ensuring that a process doesn't exhaust its limited set of file descriptor-it is essential to the correct use of pipes. We now consider why the unused file descriptors for both the read and write ends of the pipe must be closed.
The process reading from the pipe closes its write descriptor for the pipe, so that, when the other process completes its output and closes its write descriptor, the read sees end-of-file (once it has ready any outstanding data in the pipe).
If the reading process doesn't close the write end of the pipe, then after the other process closes its write descriptor, the reader won't see end-of-file, even after it has read all data from the pipe. Instead, a read() would block waiting for data, because the kernel knows that there is still at least one write descriptor open for the pipe.That this descriptor is held open by the reading process itself is irrelevant; In theory, that process could still write to the pipe, even if it is blocked trying to read.
For example, the read() might be interrupted by a signal handler that writes data to the pipe.
The writing process closes its read descriptor for the pipe for a different reason.
When a process tries to write to a pipe for which no process has an open read descriptor, the kernel sends the SIGPIPE signal to the writing process. By default, this signal kills a process. A process can instead arrange to catch or ignore this signal, in which case the write() on the pipe fails with the error EPIPE (broken pipe). Receiving the SIGPIPE signal or getting the EPIPE error is useful indication about the status of the pipe, and this is why unused read descriptors for the pipe should be closed.
If the writing process doesn't close the read end of the pipe, then even after the other process closes the read end of the pipe, the writing process will fill the pipe, and a further attempt to write will block indefinitely.
One final reason for closing unused file descriptor is that only after it all file descriptor are closed that the pipe is destroyed and its resources released for reuse by other processes. At this point, any unread data in the pipe is lost.
~ Micheal Kerrisk , the Linux programming interface
I saw this code snippet from APUE
dup2(fd,0);
dup2(fd,1);
dup2(fd, 2);
if (fd > 2)
close(fd);
In my understanding, it makes stdin, stdout and stderr all point to fd. It says that lots program contain this code, why? What's it functionality?
I'm going to add to the comments and answer here because even though they're correct, I would still have a hard time understanding exactly when and why this sequence of calls were needed.
This sequence of function calls is typically used when a process will run as a daemon. In that case, among other things, the daemon doesn't want to have the standard I/O file descriptors attached to the terminal (or other resources). To 'detach' those descriptors, something like the following might occur:
int fd;
fd = open("/dev/null",O_RDWR); // missing from APUE exercise 3.4 example
if (fd != -1)
{
dup2 (fd, 0); // stdin
dup2 (fd, 1); // stdout
dup2 (fd, 2); // stderr
if (fd > 2) close (fd);
}
What this does is bind /dev/null' to each of the standard I/O descriptors and closes the temporary descriptor used to open/dev/null` in the first place (as long as that open didn't end up using one of the descriptors usually used for the standard I/O descriptors for some reason).
Now the daemon has valid stdin/stdout/stderr descriptors, but they aren't referring to a file or device that might interfere with another process.
This is mostly used in daemon programs because the daemon not connected with the terminal or tty. so for that we need maintain the error or printed statements in one file. for that only we were using this statements. In our system File descriptor 0,1,2 is already allocated for the standard buffers like stdin,etc...
Dup2 function is something different from dup function.
In dup2 function we no need to close already using file descriptor.
In this dup2 function itself if the second argument file descriptors is already using means
without close() function dup2 is closed the second argument fd and allocated a dup of first argument fd.
Then first argument fd is connected to second fd and do the first fd works
For example dup2(fd,1) means the file descriptor works are copied to the stdout.
fd is contains any the statements is print the stdout.
When we use dup to redirect STDOUT to a pipe we do:
close(1); dup(fd[1]);
close(fd[0]);
close(fd[1]);
execlp("ls","-al",(char *) NULL);
but we are closing both ends end of the pipe. then how the STDOUT can be written to the pipe?
You're not closing both ends of the pipe, in the example code. You're closing fd[0] and fd[1]. Initially, closing those would have been enough to close both ends of the pipe, but it's not after you duplicated fd[0]. You'd have to close the duplicated fd also to close all your references to the pipe. That would be silly though: you're keeping an end open precisely so that ls can write to.
Perhaps your confusion is about close() is closing? It closes the fd, the reference to one of the ends of the pipe. It doesn't close the pipe itself: that's what shutdown() would do. (If you don't call shutdown, the pipe is automatically closed when every fd referring to it has been closed.) So, because the duplicated fd is still open, the process can write to the pipe (which isn't closed, because only two of the three references to it were closed).
Because once the file descriptor number 1 (e.g. standard out) is closed, that number is available for further dup or open calls.
You should check the result of your close and dup syscalls.
Of course closing both ends of a pipe is non-sense, unless you do something useful before (i.e. reading or writing on appropriate ends).
See open(2), dup(2), pipe(2), close(2) man pages. Read the Advanced Linux Programming book.
The following code reads messages from other processes through a pipe. All processes correctly print out all the messages, but then they will never proceed past the while loop. Tried debugging in Eclipse, after reading reading all the messages, it will just stop at that while loop.
The index is a number assigned to each process. The first process would have index == 0.
The message itself is simply the index of the process sending the message.
while((n = read(fd[index][0], &mymsg, sizeof(int))) == sizeof(int))
printf("process%d has received a message from process%d\n", index, mymsg);
Any ideas why this would happen?
Here is how each process writes to another:
// Write to other process
if(write(fd[index2][1], &index, sizeof(int)) != sizeof(int))
sys_error(2);
This is done five times. fd is a table of read-and-write ends for each process.
The call to read() is blocking until more data shows up. From the man page for pipe
If a process attempts to read from an empty pipe, then read(2) will
block until data is available. If a process attempts to write to a
full pipe (see below), then write(2) blocks until sufficient data has
been read from the pipe to allow the write to complete. Nonblocking
I/O is possible by using the fcntl(2) F_SETFL operation to enable the
O_NONBLOCK open file status flag.
After you open each file descriptor before you enter that while loop do this to each one:
fcntl(fd, F_SETFL, O_NONBLOCK);
However, you really should read up on blocking vs. non-blocking I/O, including reading the man pages for pipe, read, fcntl, etc.