In "Advanced Programming in the Unix Environment", 2nd edition, By W. Richard Stevens. Section 8.3 fork function.
Here's the description:
It is important that the parent and the child share the same file offset.
Consider a process that forks a child, then waits for the child to complete. Assume that both processes write to standard output as part of their normal processing. If the parent has its standard output redirected (by a shell, perhaps) it is essential that the parent's file offset be updated by the child when the child writes to standard output.
My responses:
{1} What does it mean? if parent's std output is redirected to a 'file1' for example, then what should child update after child writes? parent's original std output offset or redirected ouput(i.e file1) offset? Can't be the later, right?
{2} How is the update done? by child explicitly, by OS implicitly, by files descriptor itself? After fork, I thought parent and child went their own ways and has their own COPY of file descriptor. So how does child update offset to parent side?
In this case, the child can write to standard output while the parent is waiting for it; on completion of the child, the parent can continue writing to standard output, knowing that its output will be appended to whatever the child wrote. If the parent and the child did not share the same file offset, this type of interaction would be more difficult to accomplish and would require explicit actions by the parent.
If both parent and child write to the same descriptor, without any form of synchronization, such as having the parent wait for the child, their output will be intermixed (assuming it's a descriptor that was open before the fork). Although this is possible, it's not the normal mode of operation.
There are two normal cases for handling the descriptors after a fork.
The parent waits for the child to complete. In this case, the parent does not need to do anything with its descriptors. When the child terminates, any of the shared descriptors that the child read from or wrote to will have their file offsets updated accordingly.
Both the parent and the child go their own ways. Here, after the fork, the parent closes the descriptors that it doesn't need, and the child does the same thing. This way, neither interferes with the other's open descriptors. This scenario is often the case with network servers.
My response:
{3} When fork() is invoked, all i understand is that child get a COPY of what parent has, file descriptor in this case, and does its thing. If any offset changes to file descriptor that parent and child share, it can only be because the descriptor remember the offset itself. Am I right?
I am kind of new to the concepts.
It's important to distinguish between the file descriptor, which is a small integer that the process uses in its read and write calls to identify the file, and the file description, which is a structure in the kernel. The file offset is part of the file description. It lives in the kernel.
As an example, let's use this program:
#include <unistd.h>
#include <fcntl.h>
#include <sys/wait.h>
int main(void)
{
int fd;
fd = open("output", O_CREAT|O_TRUNC|O_WRONLY, 0666);
if(!fork()) {
/* child */
write(fd, "hello ", 6);
_exit(0);
} else {
/* parent */
int status;
wait(&status);
write(fd, "world\n", 6);
}
}
(All error checking has been omitted)
If we compile this program, call it hello, and run it like this:
./hello
here's what happens:
The program opens the output file, creating it if it didn't exist already or truncating it to zero size if it did exist. The kernel creates a file description (in the Linux kernel this is a struct file) and associates it with a file descriptor for the calling process (the lowest non-negative integer not already in use in that process's file descriptor table). The file descriptor is returned and assigned to fd in the program. For the sake of argument suppose that fd is 3.
The program does a fork(). The new child process gets a copy of its parent's file descriptor table, but the file description is not copied. Entry number 3 in both processes' file tables points to the same struct file.
The parent process waits while the child process writes. The child's write causes the first half of "hello world\n" to be stored in the file, and advances the file offset by 6. The file offset is in the struct file!
The child exits, the parent's wait() finishes, and the parent writes, using fd 3 which is still associated with the same file description that had its file offset updated by the child's write(). So the second half of the message is stored after the first part, not overwriting it as it would have done if the parent had a file offset of zero, which would be the case if the file description was not shared.
Finally the parent exits, and the kernel sees that the struct file is no longer in use and frees it.
In the same section, of book, there is a diagram showing three tables which are there when a file is opened.
The user filedescriptor table(part of the process table entry), filetable and inode table (v-node table).
Now a filedescriptor (which is an index to the file descriptor table) entry points to a file table entry,which points to a inode table entry.
Now the file offset (the position from where the next read/write happens) is there in the File table.
So say you have a file opened in parent, that means it is having a descriptor, a file
table entry and an inode reference too.
Now when you are creating a child the file descriptor table is copied for the child.
So the reference count in the file table entry (for that open descriptor) is increased that means now there are two references for the same file table entry.
This descriptor is now available in both parent and child, pointing to the same File Table entry, hence sharing the offset.
Now having this background let us see your questions,
What does it mean? if parent's std output is redirected to a 'file1' for example, then what should child update after child writes? parent's original std output offset or
redirected ouput(i.e file1) offset? Can't be the later, right?]
The child explicitly need not update anything. The author of the book is trying to
tell that, suppose parents's standard out put is redirected to a file and the a fork call is made. After that the parent is wating.So the descriptor is now duplicated, that is the file offset is also shared. Now whenever now the child writes anything to standard out, the written data is saved in the redirected file.
The offset automatically is incremented by the write call.
Now say the child exits.
So the parent come out of waiting and writes something on the standard out(which is
redirected). Now where the parent 's write call's output will be placed -> after the data, which was written by the child. Why -> since the offset's current value is now changed after the child has written.
Parent ( )
{
open a file for writing, that is get the
descriptor( say fd);
close(1);//Closing stdout
dup(fd); //Now writing to stdout means writing to the file
close(fd)
//Create a child that is do a fork call.
ret = fork();
if ( 0 == ret )
{
write(1, "Child", strlen("Child");
exit ..
}
wait(); //Parent waits till child exit.
write(1, "Parent", strlen("Parent");
exit ..
}
Pl. see the above pseudo code, the final data that the opened file contains will be ChildParent. So you see that the file offset got changed when the child has written and this was avilable to the parent's write call, since the offest is shared.
2.How is the update done? by child explicitly, by OS implicitly, by files descriptor itself? After fork, i thought parent and child went their own ways and
has their own COPY of file descriptor. So how does child update offset to parent side?]
Now I think the answer is clear-> by the system call that is by the OS.
[3. When fork() is invoked, all i understand is that child get a COPY of what parent has,
file descriptor in this case, and does its thing. If any offset changes to file descriptor that parent and child share, it can only be because the descriptor remember the offset itself. Am i right?]
This should also be clear. The entry of the user file table points to the file table
table entry (which contains the offset).
In other words, the system calls can fetch the offset from the descriptor.
Related
Within a parent process when a child
closes a file descriptor inherited across a fork. In other words, does the file remain open in the
parent, or is it closed there as well?
Every process has its own set of file descriptors. If you close one in one process, it is not closed in another process.
These file descriptors may point to the same file description. If you fork a process, then read from the FD in one process, then read from the FD in the other process, it will count as two different reads - not the same read twice. The first read will get the first piece of data and the second read will get the second piece of data. This is also how FDs created with dup or dup2 work.
File descriptors which point to the same file description don't have to have the same number.
This question already has answers here:
Seeking a simple description regarding 'file descriptor' after fork()
(2 answers)
Closed 6 years ago.
so I have a question regarding what a child process has access to.
If a parent open()s a file, and then fork()s a child, does the child process have access to the open file, or would it need to open the file itself? Would it be possible to have access to the already open file through shared memory between the processes? (C programming)
Short answer yes.
You can read more about it here:
http://man7.org/linux/man-pages/man2/fork.2.html
As it is said there, the child process is an exact duplicate of the parent process except
for the following points:
* The child has its own unique process ID, and this PID does not
match the ID of any existing process group (setpgid(2)).
* The child's parent process ID is the same as the parent's process
ID.
* The child does not inherit its parent's memory locks (mlock(2),
mlockall(2)).
* Process resource utilizations (getrusage(2)) and CPU time counters
(times(2)) are reset to zero in the child.
* The child's set of pending signals is initially empty
(sigpending(2)).
* The child does not inherit semaphore adjustments from its parent
(semop(2)).
* The child does not inherit process-associated record locks from
its parent (fcntl(2)). (On the other hand, it does inherit
fcntl(2) open file description locks and flock(2) locks from its
parent.)
* The child does not inherit timers from its parent (setitimer(2),
alarm(2), timer_create(2)).
* The child does not inherit outstanding asynchronous I/O operations
from its parent (aio_read(3), aio_write(3)), nor does it inherit
any asynchronous I/O contexts from its parent (see io_setup(2)).
The process attributes in the preceding list are all specified in
POSIX.1. The parent and child also differ with respect to the
following Linux-specific process attributes:
* The child does not inherit directory change notifications
(dnotify) from its parent (see the description of F_NOTIFY in
fcntl(2)).
* The prctl(2) PR_SET_PDEATHSIG setting is reset so that the child
does not receive a signal when its parent terminates.
* The default timer slack value is set to the parent's current timer
slack value. See the description of PR_SET_TIMERSLACK in
prctl(2).
* Memory mappings that have been marked with the madvise(2)
MADV_DONTFORK flag are not inherited across a fork().
* The termination signal of the child is always SIGCHLD (see
clone(2)).
* The port access permission bits set by ioperm(2) are not inherited
by the child; the child must turn on any bits that it requires
using ioperm(2).
Note the following further points:
* The child process is created with a single thread—the one that
called fork(). The entire virtual address space of the parent is
replicated in the child, including the states of mutexes,
condition variables, and other pthreads objects; the use of
pthread_atfork(3) may be helpful for dealing with problems that
this can cause.
* After a fork(2) in a multithreaded program, the child can safely
call only async-signal-safe functions (see signal(7)) until such
time as it calls execve(2).
* The child inherits copies of the parent's set of open file
descriptors. Each file descriptor in the child refers to the same
open file description (see open(2)) as the corresponding file
descriptor in the parent. This means that the two file
descriptors share open file status flags, file offset, and signal-
driven I/O attributes (see the description of F_SETOWN and
F_SETSIG in fcntl(2)).
* The child inherits copies of the parent's set of open message
queue descriptors (see mq_overview(7)). Each file descriptor in
the child refers to the same open message queue description as the
corresponding file descriptor in the parent. This means that the
two file descriptors share the same flags (mq_flags).
* The child inherits copies of the parent's set of open directory
streams (see opendir(3)). POSIX.1 says that the corresponding
directory streams in the parent and child may share the directory
stream positioning; on Linux/glibc they do not.
Yes, using the same file descriptor. No need for shared memory.
It would kind of work using a stdio FILE* but I would no advise doing so for files you plan to write to, as buffering in the two separate processes would lead to unexpected and confusing results.
To prevent this sharing of file descriptors - if you want to - you can of course call close() on the file descriptor as soon as the fork() call returns 0.
As far I understand it, Fds are integers that are used to look up the open files in the kernel's file description table. So therefore if you have a code segment like this:
int fd[2], temp1, temp2;
pipe(fd);
temp1 = fd[0];
temp2 = fd[1];
close(temp1);
close(temp2);
All the file descriptors to the pipe are closed and thus the pipe would no longer exist. Since FDs are simply ints, saying close(temp1) is identical to saying close(fd[0]).
In light of all this (and please let me know if I am misunderstanding) I am confused by what happens after a fork() call. Since the child process inherits the same FDs and state of the parent, the child's FDs should be the same ints as the parents. So by that logic, if I close(fd[0]) in the child, I believe it would also prevent the parent from accessing the file. Since close() "frees" that integer from the file descriptor table, the parent should not have any way to reference the file.
Is this the case? It seems very unlikely that this is the actual case, since it would cause FDs between parents and children to be very difficult to use (especially since you do not know which process will run first). So if this logic is incorrect, are the FDs duplicated on fork()? How in the file descriptor table are the parent and child Fds related, especially between close() calls? It helps me a lot to be able to draw out the file descriptor table, so I would like as specific as an answer as possible.
Thanks for any help with this!
No. What the child does in the way of closing files affects just the child's copy of the file descriptors, without affecting the parent's copy of the file descriptors.
However, immediately after forking, both sets of file descriptors (in parent and child) refer to the same set of open file descriptions (note the 'descriptor' vs 'description' terminology). If the child does things such as read or seek that affect the file description, then the child's activities affect the parent too.
You'll want to study the POSIX specifications of open(),
fork() and execve() (especially the execve() page) with considerable care.
The parent and the child have their own file descriptor tables.
If the child closes (say) file descriptor 5, then the parent still has file descriptor 5 open.
If the child then opened another file, and it happened to get descriptor 5, then the child's file descriptor 5 would refer to a different file than the parent's file descriptor 5 (which would not have changed).
I am attempting to write a program which forks and waits for his child to finish, then the child does some work on an input and then forks the same way it's parent does and so on.
Now, I know that forking copies to the child the array of file descriptors and that I should close the ones associated with the parent, but I can't figure out which are the parents. Do I need to give to my child it's parents pid?
I've been trying to wrap my head around it for the better part of an hour and I think I have some kind of a mind block because I can't come to a conclusion.
TL;DR: As a child process how do I know which file descriptors belong to my parent?
Just after the fork (and before any exec function) your child process has the same state as its parent process (except for the result of the fork, which is 0 only in the child). So you know what are the file descriptors, since you have coded the program running in parent&child. On Linux you might also read the /proc/self/fd/ directory, see proc(5).
You might close most file descriptors after the fork and before the exec; you could code something like
for (int fd=3; fd<64; fd++) (void) close(fd);
we are starting from 3 which is after STDERR_FILENO which is 2, and we are stopping arbitrarily at 64, and the cast to (void) on the close call means to the reader that we don't care about failing close.... Of course, if you have e.g. some pipe(7)-s to communicate between parent and child you'll be careful to avoid closing their relevant file descriptor(s).
(However, doing a closing loop like above is poor taste and old fashion)
In general, you'll be careful in your program to set the close-on-exec flag on most file descriptors (e.g. fcntl(2) on F_SETFD operation and FD_CLOEXEC flag, or directly open(2) with O_CLOEXEC), then the execve(2) (done in most child processes after the fork) would close them.
I read from man pages of execve that if a process(A) call execve, the already opened file descriptors are copied to the new process(B).
Two possiblities arise here :-
1) Does it mean that a new file descriptor table is created for process B, the entries to which are copied from older file descriptor table of process A
2) Or process B gets the file descriptor table of process A as after execve process A will cease to exist and the already opened files could only be closed from process B, if it gets the file descriptor table of process A.
Which one is correct?
execve does not create a new process. It replaces the calling process's program image, memory space, etc. with new ones based on an executable file from the filesystem. The file descriptor table is modified by closing any descriptors with close-on-exec flag set; the rest remain open and in the same state they were in (current position, locks, etc.) prior to execve.
You're probably confusing this with what happens on fork since execve is usually preceded by fork. When a process forks, the child process has a new file descriptor table referring to the same open file descriptions as the parent process's file descriptor table.
Which one is correct?
#2
Though what you ask is more of OS implementation details and that is rarely if ever important to applications, completely transparent to the applications and depends on the OS.
It is normally said that the new process inherits file descriptors. Except those having FD_CLOEXEC flag set, obviously.
Even in case of #1, if we would presume that for some short time both process A and B are in memory (not really, that's fork() territory) copying of the fd table would be OK. Since process A would be terminated (by the exec()) all its file descriptors would be close()d. And that would have no effect on the already-copied file descriptors in the process B. File descriptors are like pointers to the corresponding kernel structure containing actual information about what the file descriptor actually points to. Copying the fd table doesn't make copies of the underlying structures - it copies only the pointers. The kernel structure contains reference counter (required to implement the fork()) which is incremented when copy is made and thus knows how many processes are using it. Calling close() on file descriptor first of all does decrement of the reference counter. And only if the counter goes to zero (no more processes are using the structure) only then OS actually closes the underlying file/socket/pipe/etc. (But obviously even if inside of the kernel two processes are present for some short time simultaneously, user space applications can't see that since the new process after exec() also inherits the PID of the original process.)