Here is the code:
int main() {
int fd[2];
pipe(fd);
int r = fork();
if (r > 0) { //parent
close(fd[0]);
// do a bunch of things
} else { //child
close(fd[1]);
// do a bunch of things
return 0;
}
This is a piece of code where the parent writes to the pipe and child reads from the pipe. My question is: for the two close statements, what exactly are they closing? The parent and the child should share the same file, i.e. fd[0] and fd[1]. If fd[0] is closed in the parent, shouldn't it also be closed in child?
From http://linux.die.net/man/2/pipe pipe() creates a pipe which consists of two file descriptors which correspond with the two "ends" of the pipe, the read end and the write end. It's not really the same thing as a file. The kernel is reading data from the write end, buffering it for you, and transferring it it to the read end.
This should make it obvious why pipe() creates two file descriptors. The writer writes all the data it needs into the write fd and closes the fd. This also triggers an EOF to be sent. The reader would usually keep reading data until it encounters the EOF and closes its end. In this scenario, there's a period of time where the write fd is closed but data is still buffered in the pipe, waiting to be read out by the reader. It doesn't make sense to have a single fd, as you'll need another layer of coordination between the writer and reader processes, otherwise who will do the closing, and when?
The pipe() call always returns an integer array where the first element of array is the read descriptor to read from pipe and second element is the write descriptor to write into the pipe. The pipes provide one way communication. If you close fd[0] in parent and also in child there is from nowhere you can read from the pipe, in the reverse case if you close fd[1] in both the processes you cannot write into pipe, So we close the read descriptor in one process so that the process can only write and the other process will close write descriptor which will enable the process to only read from pipe.
Related
I am confused as to how to properly use close to close pipes in C. I am fairly new to C so I apologize if this is too elementary but I cannot find any explanations elsewhere.
#include <stdio.h>
int main()
{
int fd[2];
pipe(fd);
if(fork() == 0) {
close(0);
dup(fd[0]);
close(fd[0]);
close(fd[1]);
} else {
close(fd[0]);
write(fd[1], "hi", 2);
close(fd[1]);
}
wait((int *) 0);
exit(0);
}
My first question is: In the above code, the child process will close the write side of fd. If we first reach close(fd[1]), then the parent process reach write(fd[1], "hi", 2), wouldn't fd[1] already been closed?
int main()
{
char *receive;
int[] fd;
pipe(fd);
if(fork() == 0) {
while(read(fd[0], receive, 2) != 0){
printf("got u!\n");
}
} else {
for(int i = 0; i < 2; i++){
write(fd[1], 'hi', 2);
}
close(fd[1]);
}
wait((int *) 0);
exit(0);
}
The second question is: In the above code, would it be possible for us to reach close(fd[1]) in the parent process before the child process finish receiving all the contents? If yes, then what is the correct way to communicate between parent and child. My understanding here is that if we do not close fd[1] in the parent, then read will keep being blocked, and the program won't exit either.
First of all note that, after fork(), the file descriptors fd would also get copied over to the child process. So basically, a pipe acts like a file with each process having its own references to the read and write end of the pipe. Essentially there are 2 read and 2 write file descriptors, one for each process.
My first question is: In the above code, the child process will close
the write side of fd. If we first reach close(fd[1]), then the parent
process reach write(fd[1], "hi", 2), wouldn't fd[1] already been
closed?
Answer: No. The fd[1] in parent process is the parent's write end. The child has forsaken its right to write on the pipe by closing its fd[1], which does not stop the parent from writing into it.
Before answering the second question, I fixed your code to actually run it and produce some results.
int main()
{
char receive[10];
int fd[2];
pipe(fd);
if(fork() == 0) {
close(fd[1]); <-- Close UNUSED write end
while(read(fd[0], receive, 2) != 0){
printf("got u!\n");
receive[2] = '\0';
printf("%s\n", receive);
}
close(fd[0]); <-- Close read end after reading
} else {
close(fd[0]); <-- Close UNUSED read end
for(int i = 0; i < 2; i++){
write(fd[1], "hi", 2);
}
close(fd[1]); <-- Close write end after writing
wait((int *) 0);
}
exit(0);
}
Result:
got u!
hi
got u!
hi
Note: We (seemingly) lost one hi because we are reading it into same array receive which essentially overrides the first hi. You can use 2D char arrays to retain both the messages.
The second question is: In the above code, would it be possible for us
to reach close(fd[1]) in the parent process before the child process
finish receiving all the contents?
Answer: Yes. Writing to a pipe() is non-blocking (unless otherwise specified) until the pipe buffer is full.
If yes, then what is the correct
way to communicate between parent and child. My understanding here is
that if we do not close fd[1] in the parent, then read will keep being
blocked, and the program won't exit either.
If we close fd[1] in parent, it will signal that parent has closed its write end. However, if the child did not close its fd[1] earlier, it will block on read() as the pipe will not send EOF until all the write ends are closed. So the child will be left expecting itself to write to the pipe, while reading from it simultaneously!
Now what happens if the parent does not close its unused read end? If the file had only one read descriptor (say the one with the child), then once the child closes it, the parent will receive some signal or error while trying to write further to the pipe as there are no readers.
However in this situation, parent also has a read descriptor open and it will be able to write to the buffer until it gets filled, which may cause problems to the next write call, if any.
This probably won't make much sense now, but if you write a program where you need to pass values through pipe again and again, then not closing unused ends will fetch you frustrating bugs often.
what is the correct way to communicate between parent and child[?]
The parent creates the pipe before forking. After the the fork, parent and child each close the pipe end they are not using (pipes should be considered unidirectional; create two if you want bidirectional communication). The processes each have their own copy of each pipe-end file descriptor, so these closures do not affect the other process's ability to use the pipe. Each process then uses the end it holds open appropriately for its directionality -- writing to the write end or reading from the read end.
When the writer finishes writing everything it intends ever to write to the pipe, it closes its end. This is important, and sometimes essential, because the reader will not perceive end-of-file on the read end of the pipe as long as any process has the write end open. This is also one reason why it is important for each process to close the end it is not using, because if the reader also has the write end open then it can block indefinitely trying to read from the pipe, regardless of what any other process does.
Of course, the reader should also close the read end when it is done with it (or terminate, letting the system handle that). Failing to do so constitutes excess resource consumption, but whether that is a serious problem depends on the circumstances.
I am creating a TCP service that forks a new process each time a client connects. Before the fork I set up a pipe so the child can send statistics gathered during the connection back to the parent. The parent closes the writing end and the child closes the reading end, and the parent maintains an array of reading-end file descriptors, one per child.
I am not sure what to do with these file descriptors when the child finishes with the connection and exits. Does the child need to notify the parent via the pipe that it is about to exit so the parent can close the pipe? Or can the parent detect the broken pipe automatically after the child exits and close it?
The code in the parent program is running a loop with select() detecting activity on the listening socket and on the read ends of the children's pipes. Each child may send multiple messages to the parent as it runs.
In general, what should the parent process do with a pipe file descriptor when a child exits?
First pass: before it was clear that there was a loop using select() and that children sent multiple messages.
If the parent process maintains an array of file descriptors, it needs to also associate each file descriptor with a child process. If the children send a single small statistics message before they die, then when the main program waits for dead children, it knows which child died, so it can then close the file descriptor for the child that it just spotted dieing (after making sure the pipe is empty by doing one or more final reads).
An alternative mechanism uses select() or poll() or a related function that reports when a read operation on a file descriptor would not hang. When it detects EOF (zero bytes read) from a pipe, it knows the child died. However, this is probably fiddlier to deal with.
It isn't entirely clear from your question whether there's a single message from the child process as it exits, or whether there is a 'stream of consciousness' statistics reports as the child is working. If there's a single message (that's smaller than the pipe buffer size), then life is easy. If there's a stream of messages or the message is longer than the pipe buffer size, you have to think more carefully about coordination — you can't detect messages only when the child dies.
Second pass: after the extra information became available.
If you're already using select() in a loop, then when a child dies, you will get a 'pipe ready for reading' indication from select() and you will get 0 bytes from read() which indicates EOF on that pipe. You should then close that pipe (and wait for one or more children with waitpid(), probably using W_NOHANG — there should be at least one corpse to be collected — so you don't have zombies kicking around for protracted times).
A strict answer to your last question is: when the only child with the write end of a pipe dies, the parent should close the read end of that pipe to release resources for later reuse.
In your case, the parent process shall close writing end of the pipe right after fork. Then it can read its statistic data until EOF (end-of-file) and then close the reading end of the pipe.
broken pipe happens when you write to the pipe but no fd is open for reading from that pipe. So it doesn't apply to your case. In your case, since your parent is reading from the pipe, it should read EOF when child exits (if you have closed the write end in your parent process correctly, otherwise it will just block since it assumes there will still be things to read in the future). Then you can safely close the read fd in your parent process.
In general
If parent writes and child reads, you do need to worry about broken pipe which is when the child closes the read fd, and parent gets SIGPIPE as it keeps writing to the pipe. SIGPIPE by default terminates the process, so you may want to set up a signal handler to make it do whatever you want (if you don't want it to just terminate).
Let's see the see the cases differently for Parent having 1 child and children.
Parent having 1 child. When child process exits and parent is waiting on read end, parent will also exit. Here's the code
//fd[0] //read end
//fd[1] //write end
#include <unistd.h>
#include <stdio.h>
#include <errno.h> //For errno
#include <stdlib.h> //exit()
void DumpAndExit(char* str){
perror (str);
printf ("errno: %d", errono);
exit(0);
}
int main(){
int fd[2], pid = -1;
if (pipe(fd) < 0)
DumpAndExit ("pipe");
if (pid = fork() < 0) {
DumpAndExit ("fork");
}
else if (pid == 0) { //Child
close(fd[0]); //Close read end
printf("In Child \n");
sleep(2);
exit(0);
} else { //Parent
close(fd[1]); //close write
waitpid(-1); //Parent will wait for child
printf("Parent waiting\n");
char buf[4] = {};
read(fd[0], buf, sizeof(buf)); //reads from pipe
printf ("Read from child: %s", buf);
}
}
# ./a.out
In child
Parent waiting
Read from child:
#
In very simple words:
Every process have a PCB(struct task_struct) which has all information of process, In case of fork() it will have child's contexts as well. Means pointers to child's PCB.
Since pipe ie int fd[2] is created on stack of parent, then duplicated to child's stack. When child exits, its PCB is cleared, PCB of parent is updated and Parent knows there's no one connected at other end of pipe.
This is the code i found for my own shell. It works fine, but the thing i can't understand is pipe section of the code.
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>
char* cmndtkn[256];
char buffer[256];
char* path=NULL;
char pwd[128];
int main(){
//setting path variable
char *env;
env=getenv("PATH");
putenv(env);
system("clear");
printf("\t MY OWN SHELL !!!!!!!!!!\n ");
printf("_______________________________________\n\n");
while(1){
fflush(stdin);
getcwd(pwd,128);
printf("[MOSH~%s]$",pwd);
fgets(buffer,sizeof(buffer),stdin);
buffer[sizeof(buffer)-1] = '\0';
//tokenize the input command line
char* tkn = strtok(buffer," \t\n");
int i=0;
int indictr=0;
// loop for every part of the command
while(tkn!=NULL)
{
if(strcoll(tkn,"exit")==0 ){
exit(0);
}
else if(strcoll(buffer,"cd")==0){
path = buffer;
chdir(path+=3);
}
else if(strcoll(tkn,"|")==0){
indictr=i;
}
cmndtkn[i++] = tkn;
tkn = strtok(NULL," \t\n");
}cmndtkn[i]='\0';
// execute when command has pipe. when | command is found indictr is greater than 0.
if(indictr>0){
char* leftcmnd[indictr+1];
char* rightcmnd[i-indictr];
int a,b;
for(b=0;b<indictr;b++)
leftcmnd[b]=cmndtkn[b];
leftcmnd[indictr]=NULL;
for(a=0;a<i-indictr-1;a++)
rightcmnd[a]=cmndtkn[a+indictr+1];
rightcmnd[i-indictr]=NULL;
if(!fork())
{
fflush(stdout);
int pfds[2];
pipe(pfds);
if(!fork()){
close(1);
dup(pfds[1]);
close(pfds[0]);
execvp(leftcmnd[0],leftcmnd);
}
else{
close(0);
dup(pfds[0]);
close(pfds[1]);
execvp(rightcmnd[0],rightcmnd);
}
}else
wait(NULL);
//command not include pipe
}else{
if(!fork()){
fflush(stdout);
execvp(cmndtkn[0],cmndtkn);
}else
wait(NULL);
}
}
}
What is the purpose of the calls to close() with parameters of 0 and 1 mean and what does the call to dup() do?
On Unix, the dup() call uses the lowest numbered unused file descriptor. So, the close(1) before the call to dup() is to coerce dup() to use file descriptor 1. Similarly for close(0).
So, the aliasing is to get the process to use the write end of the pipe for stdout (file descriptor 1 is used for console output), and the read end of the pipe for stdin (file descriptor 0 is used for console input).
The code may have been more clearly expressed with dup2() instead.
dup2(fd[1], 1); /* alias fd[1] to 1 */
From your question about how ls | sort works, your question is not limited to why the dup() system call is being made. Your question is actually how pipes in Unix work, and how a shell command pipeline works.
A pipe in Unix is a pair of file descriptors that are related in that writing data on tje writable descriptor allows that data to be read from the readable descriptor. The pipe() call returns this pair in an array, where the first array element is readable, and second array element is writable.
In Unix, a fork() followed by some kind of exec() is the only way to produce a new process (there are other library calls, such as system() or popen() that create processes, but they call fork() and do an exec() under the hood). A fork() produces a child process. The child process sees the return value of 0 from the call, while the parent sees a non-zero return value that is either the PID of the child process, or a -1 indicating that an error has occurred.
The child process is a duplicate of the parent. This means that when a child modifies a variable, it is modifying a copy of the variable that resides in its own process. The parent does not see the modification occur, as the parent has the original copy). However, a duplicated pair of file descriptors that form a pipe can be used to allow a child process its parent to communicate with each other.
So, ls | sort means that there are two processes being spawned, and the output written by ls is being read as input by sort. Two processes means two calls to fork() to create two child processes. One child process will exec() the ls command, the other child process will exec() the sort command. A pipe is used between them to allow the processes to talk to each other. The ls process writes to the writable end of the pipe, the sort process reads from the readable end of the pipe.
The ls process is coerced into writing into the writable end of the pipe with the dup() call after issuing close(1). The sort process is coerced into reading the readable end of the pipe with the dup() call after close(0).
In addition, the close() calls that close the pipe file descriptors are used to make sure that the ls process is the only process to have an open reference to the writable fd, the the sort process is the only process to have an open reference to the readable fd. That step is important because after ls exits, it will close the writable end of the fd, and the sort process will expect to see an EOF as a result. However, this will not occur if some other process still has the writable fd open.
http://en.wikipedia.org/wiki/Standard_streams#Standard_input_.28stdin.29
stdin is file descriptor 0.
stdout is file descriptor 1.
In the !fork section, the process closes stdout then calls dup on pfds[1] which according to:
http://linux.die.net/man/2/dup
Creates a duplicate of the specified file descriptor at the lowest available position, which will be 1, since it was just closed (and stdin hasn't been closed yet). This means everything sent to stdout will really go to pfds[1].
So, basically, it's setting up the two new processes to talk to each other. the !fork section is for the new child which will send data to stdout (file descriptor 1), the parent (the else block) closes stdin, so it really reads from pfds[0] when it tries to read from stdout.
Each process has to close the file descriptor in pfds it's not using, as there are two open handles to the file now that the process has forked. Each process now execs to left/right-cmnd, but the new stdin and stdout mappings remain for the new processes.
Forking twice is explained here: Why fork() twice
Assuming I have a parent process that forks a child process, writes to the child, and then waits to read something from the child, can I implement this with one pipe? It would look something like:
int main(){
pid_t pid1;
int pipefd[2];
char data[]="some data";
char rec[20];
if(pipe(pipefd) == -1){
printf("Failed to pipe\n");
exit(0);
}
pid1 = fork();
if(pid1<0){
printf("Fork failed\n");
exit(0);
}else if(pid1==0){
close(pipefd[1]);
read(pipefd[0],rec,sizeof(rec));
close(pipefd[0]);
//do some work and then write back to pipe
write(pipefd[1],data,sizeof(data));
}else{
close(pipefd[0]);
write(pipefd[1],data,sizeof(data));
close(pipefd[1]);
//ignoring using select() for the moment.
read(pipedfd[0],rec,sizeof(rec));
}
When trying to learn more about this, the man pages state that pipes are unidirectional. Does this mean that when you create a pipe to communicate between a parent and child, the process that writes to the pipe can no longer read from it, and the process that reads from the pipe can no longer write to it? Does this mean you need two pipes to allow back and forth communication? Something like:
Pipe1:
P----read----->C
P<---write-----C
Pipe2:
P----write---->C
P<---read------C
No. Pipes by definition are one-way. The problem is, that without any synchronization you will have both processes reading from the same filedescriptor. If you, however, use semaphores you could do something like that
S := semaphore initiated to 0.
P writes to pipe
P tries down on S (it blocks)
P reads from pipe
C reads from pipe
C writes to pipe
C does up on S (P wakes up and continues)
The other way is to use two pipes - easier.
It is unspecified whether fildes[0] is also open for writing and whether fildes[1] is also open for reading.
That being said, the easiest way would be to use two pipes.
Another way would be to specify a file descriptor/name/path to the child process through the pipe. In the child process, instead of writing to filedes[1], you can write to the file descriptor/name/path specified in filedes[1].
Given the following code:
int main(int argc, char *argv[])
{
int pipefd[2];
pid_t cpid;
char buf;
if (argc != 2) {
fprintf(stderr, "Usage: %s \n", argv[0]);
exit(EXIT_FAILURE);
}
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child reads from pipe */
close(pipefd[1]); /* Close unused write end */
while (read(pipefd[0], &buf, 1) > 0)
write(STDOUT_FILENO, &buf, 1);
write(STDOUT_FILENO, "\n", 1);
close(pipefd[0]);
_exit(EXIT_SUCCESS);
} else { /* Parent writes argv[1] to pipe */
close(pipefd[0]); /* Close unused read end */
write(pipefd[1], argv[1], strlen(argv[1]));
close(pipefd[1]); /* Reader will see EOF */
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
return 0;
}
Whenever the child process wants to read from the pipe, it must first close the pipe's side from writing. When I remove that line close(pipefd[1]); from the child process's if,
I'm basically saying that "okay, the child can read from the pipe, but I'm allowing the parent to write to the pipe at the same time"?
If so, what would happen when the pipe is open for both reading & writing? No mutual exclusion?
Whenever the child process wants to read from the pipe, it must first close the pipe's side from writing.
If the process — parent or child — is not going to use the write end of a pipe, it should close that file descriptor. Similarly for the read end of a pipe. The system will assume that a write could occur while any process has the write end open, even if the only such process is the one that is currently trying to read from the pipe, and the system will not report EOF, therefore. Further, if you overfill a pipe and there is still a process with the read end open (even if that process is the one trying to write), then the write will hang, waiting for the reader to make space for the write to complete.
When I remove that line close(pipefd[1]); from the child's process IF, I'm basically saying that "okay, the child can read from the pipe, but I'm allowing the parent to write to the pipe at the same time"?
No; you're saying that the child can write to the pipe as well as the parent. Any process with the write file descriptor for the pipe can write to the pipe.
If so, what would happen when the pipe is open for both reading and writing — no mutual exclusion?
There isn't any mutual exclusion ever. Any process with the pipe write descriptor open can write to the pipe at any time; the kernel ensures that two concurrent write operations are in fact serialized. Any process with the pipe read descriptor open can read from the pipe at any time; the kernel ensures that two concurrent read operations get different data bytes.
You make sure a pipe is used unidirectionally by ensuring that only one process has it open for writing and only one process has it open for reading. However, that is a programming decision. You could have N processes with the write end open and M processes with the read end open (and, perish the thought, there could be processes in common between the set of N and set of M processes), and they'd all be able to work surprisingly sanely. But you'd not readily be able to predict where a packet of data would be read after it was written.
fork() duplicates the file handles, so you will have two handles for each end of the pipe.
Now, consider this. If the parent doesn't close the unused end of the pipe, there will still be two handles for it. If the child dies, the handle on the child side goes away, but there's still the open handle held by the parent -- thus, there will never be a "broken pipe" or "EOF" arriving because the pipe is still perfectly valid. There's just nobody putting data into it anymore.
Same for the other direction, of course.
Yes, the parent/child could still use the handle to write into their own pipe; I don't remember a use-case for this, though, and it still gives you synchronization problems.
When the pipe is created it is having two ends the read end and write end. These are entries in the User File descriptor table.
Similarly there will be two entries in the File table with 1 as reference count for both the read end and the write end.
Now when you fork, a child is created that is the file descriptors are duplicated and thus the reference count of both the ends in the file table becomes 2.
Now "When I remove that line close(pipefd[1])" -> In this case even if the parent has completed writing, your while loop below this line will block for ever for the read to return 0(ie EOF). This happens since even if the parent has completed writing and closed the write end of the pipe, the reference count of the write end in the File table is still 1 (Initially it was 2) and so the read function still is waiting for some data to arrive which will never happen.
Now if you have not written "close(pipefd[0]);" in the parent, this current code may not show any problem, since you are writing once in the parent.
But if you write more than once then ideally you would have wanted to get an error (if the child is no longer reading),but since the read end in the parent is not closed, you will not be getting the error (Even if the child is no more there to read).
So the problem of not closing the unused ends become evident when we are continuously reading/writing data. This may not be evident if we are just reading/writing data once.
Like if instead of the read loop in the child, you are using only once the line below, where you are getting all the data in one go, and not caring to check for EOF, your program will work even if you are not writing "close(pipefd[1]);" in the child.
read(pipefd[0], buf, sizeof(buf));//buf is a character array sufficiently large
man page for pipe() for SunOS :-
Read calls on an empty pipe (no buffered data) with only one
end (all write file descriptors closed) return an EOF (end
of file).
A SIGPIPE signal is generated if a write on a pipe with only
one end is attempted.