I am trying to implement a code that does ls| grep "pipes"| wc -l. For this I have created 3 child processes and used 2 pipes. Please find the code used:
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <sys/wait.h>
#include <stdlib.h>
int main(void)
{
//pid_t p;
int pfds1[2],pfds2[2],s;
pipe(pfds1);
pipe(pfds2);
if(!fork()) //first child ls
{
//printf("ls ppid is:%d\n", getppid());
dup2(pfds1[1],1);
close(pfds1[0]);
close(pfds2[0]);
close(pfds2[1]);
close(pfds1[1]);
if(execlp("ls", "ls", NULL)==-1)
{
perror("Error in exec line 24\n");
exit(1);
}
}
else{
if(!fork())
{
//printf("grep ppid is:%d\n", getppid());
dup2(pfds1[0],0);
dup2(pfds2[1],1);
close(pfds1[0]);
close(pfds2[0]);
close(pfds2[1]);
close(pfds1[1]);
if(execlp("grep","grep","pipes",NULL)==-1)
{
perror("Error in exec line 41\n");
exit(1);
}
}
else{
if(!fork())
{ //printf("wc ppid is:%d\n", getppid());
dup2(pfds2[0],0);
close(pfds1[0]);
close(pfds2[0]);
close(pfds2[1]);
close(pfds1[1]);
if(execlp("wc","wc","-l",NULL)==-1)
{
perror("Error in exec line 56\n");
exit(1);
}
}
else{
close(pfds1[0]);
close(pfds2[0]);
close(pfds1[1]);
close(pfds1[1]);
wait(&s);
wait(&s);
wait(&s);
//printf("parent pid is:%d\n", getpid());
//printf("grandparent pid is:%d\n", getppid());
exit(0);
}
}
}
}
The code works as intended when only 2 wait(&s) are used instead of 3, even though there are three children processes. The current code gets stuck and doesn't finish executing. Could someone pls elaborate on why this is happening?
Thanks
Before all, there's a typo in your code that makes one end of the pipe in the last process to remain open and this makes the block to happen. See below.
The problem is that when you fork your two file descriptors (from which you are closing one, the unused one, in the child, before calling exec) convert in four (two in the child, and also two in the parent, and you don't close one of the file descriptors in the parent process)
if you don't close the descriptors you don't use (either in the parent or in the child process) there will be cases in which both descriptors of the pipe will still be open, and while that happens, the reading process is blocked, waiting for some input (or EOF) to come. when you create a pipe (by using the pipe(2) system call, as soon as you fork(), close the file descriptor of the pipe you are not going to use, because you can block because everything is finished, but you still wait for input to come (being the parent or the child process, depending on how you organized the information to flow)
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <sys/wait.h>
#include <stdlib.h>
int main(void)
{
//pid_t p;
int pfds1[2],pfds2[2], s;
pipe(pfds1);
pipe(pfds2);
if(!fork()) { //first child ls
//printf("ls ppid is:%d\n", getppid());
dup2(pfds1[1],1);
here, you have installed the writing end of the pipe as the standard output for ls, but you don't read that.
close(pfds1[0]);
close(pfds2[0]);
close(pfds2[1]);
close(pfds1[1]);
This is ok. All file descriptors will be closed on exit, so no need to close them explicitly.
if(execlp("ls", "ls", NULL)==-1) {
perror("Error in exec line 24\n");
exit(1);
}
} else { // parent
if(!fork()) {
//printf("grep ppid is:%d\n", getppid());
dup2(pfds1[0],0);
dup2(pfds2[1],1);
Here you connect the standard input to the reading side of pipe1 and the standard output of pipe2 to the standard input. grep will wait for input on that pipe.
close(pfds1[0]);
close(pfds2[0]);
close(pfds2[1]);
close(pfds1[1]);
as said before, all pipes will be closed when grep finishes... so no need to explicitly close them here.
if(execlp("grep","grep","pipes",NULL)==-1) {
perror("Error in exec line 41\n");
exit(1);
}
} else { // parent
if(!fork()) {
//printf("wc ppid is:%d\n", getppid());
dup2(pfds2[0],0);
you connect also the input edge of pipe2 to standard input, so wc -l will wait for your input on pipe2.
close(pfds1[0]);
close(pfds2[0]);
close(pfds2[1]);
close(pfds1[1]);
as always, no problem if you don't close because this process will close all of them when it finishes.
if(execlp("wc","wc","-l",NULL)==-1) {
perror("Error in exec line 56\n");
exit(1);
}
} else{ // parent
close(pfds1[0]);
close(pfds2[0]);
close(pfds1[1]);
close(pfds1[1]); /* mistake!!!! */
You have a typo above, you close twice the descriptor pdfs1[1] and pdfs2[1] remains open, so the last process remains waiting for more input on the pipe, and never ends.
wait(&s);
wait(&s);
wait(&s);
//printf("parent pid is:%d\n", getpid());
//printf("grandparent pid is:%d\n", getppid());
exit(0);
}
}
}
}
Edit
The pipe requires that all the descriptors belonging to the write side to be closed, in order to notify the readers that end of file has occured and so, unlock the processes read()ing on them. In the fork, the descriptors of the pipe are implicitly dup()ed, and you again dup() them, when you redirect the input or output, by doing an explicit loop. The children processes just need to close the descriptors they are not going to use (you do well, when you do the dup() to redirect, and then close all the pipe descriptors)
In your case you use the pipes just to connect the children, and no communication is made from the parent... but its pipe descriptors also count to consider is EOF will be signalled to a reader. For a reader to be unlocked from read with EOF, you need to close all the dupped writing descriptors. This includes the ones of the parent, and the ones of the other children.
Related
Here i try to implement linux shell script with piping in c, and i try to do it by passing the output of 1st child process to the 2nd child, and then do "grep a", then it should return sth like this
a 1
a 4
,and it should end the program.
But what i encounter is that, the output of 2nd child process is correct,output of "grep a" did come out, but the child process get stuck there and does not terminate itself, can anyone explain to me why this is happening? My parent process is keep waiting for the 2nd child process to end. But it just stuck there foreverfor some reason.
/* pipe4.c */
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include<sys/wait.h>
#include<stdio.h>
#include<stdlib.h>
#include <unistd.h>
int main(int agrc, char* agrv[])
{
int pipefds[2];
pid_t pid;
pid_t pid2;
int status;
if(pipe(pipefds) == -1){
perror("pipe");
exit(EXIT_FAILURE);
}
pid = fork();
if(pid == -1){
perror("fork");
exit(EXIT_FAILURE);
}
if(pid == 0){
//replace stdout with the write end of the pipe
dup2(pipefds[1],STDOUT_FILENO);
//close read to pipe, in child
close(pipefds[0]);
execlp("cat","cat","try.txt",NULL);
}else{
waitpid(pid, &status, 0);
printf("first child done\n");
pid2 = fork();
if(pid2 == 0){
printf("second child start\n");
dup2(pipefds[0],STDIN_FILENO);
close(pipefds[1]);
execlp("grep","grep","a",NULL);
}
else{
waitpid(pid2, &status, 0);
printf("second child end\n");
close(pipefds[0]);
close(pipefds[1]);
exit(EXIT_SUCCESS);
printf("end\n");
}
}
}
The grep is waiting for all processes to close the write side of the pipe. The parent is waiting for grep to finish before it closes the write side of the pipe. That's a deadlock. The parent needs to close the pipe ends before it calls waitpid
Note that the boiler plate for dup2 is:
dup2(pipefds[1],STDOUT_FILENO);
close(pipefds[0]);
close(pipefds[1]);
as you need to close both ends of the pipe. I believe this is not causing an issue in your current setup, but it's not worth thinking too hard about. Just close both ends of the pipe.
In this program 2 children process are forked, then one send string to another through pipe. When communication finished, the parent get stuck in waiting the exit of the child that reads from the pipe (waitpid(read_child, NULL, 0);). It works fine without any waitpid (both children processes exit) or just wait for the write_child. Why is that?
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/wait.h>
int main(int argc, char *argv[])
{
int pipe_fd[2];
if (pipe(pipe_fd) == -1)
{
// fail to build pipe
perror("pipe");
exit(EXIT_FAILURE);
}
int read_child = fork();
if (read_child == 0)
{
sleep(0.5);
close(pipe_fd[1]);
printf("child %d read from pipe:\n", (int)getpid());
char buffer;
while (read(pipe_fd[0], &buffer, 1) > 0)
{
write(STDOUT_FILENO, &buffer, 1);
}
write(STDOUT_FILENO, "\n", 1);
close(pipe_fd[0]);
printf("read child exits\n");
exit(0);
}
int write_child = fork();
if (write_child == 0)
{
sleep(0.5);
close(pipe_fd[0]);
printf("child %d writes to pipe\n", (int)getpid());
char message[100];
sprintf(message, "greeting from brother %d", (int)getpid());
write(pipe_fd[1], message, strlen(message));
close(pipe_fd[1]);
printf("write child exits\n");
exit(0);
}
waitpid(read_child, NULL, 0);
// waitpid(write_child, NULL, 0);
return 0;
}
TL;DR
add close(pipe_fd[1]); just before parent process's waitpid(read_child, NULL, 0); will solve the problem.
The problem here is that, parent process also holds a reference to the two pipe fds.
The read blocks until some data available or, when the pipe identified by the fd is closed and the read returns immediately with 0 byte.
Initially the refcount of the write pipe fd is 2, from the writer child and the parent. The parent block waiting for writer, and then the writer exits, refcount decreases to 1. As the writer has exited, the parent's blocking wait returns, and the parent also exit. Refcount decreases to 0, so the write side of pipe is closed, so the reader's blocking read returns with 0 byte, then the reader exits.
However if the parent wait for the reader without first releasing its reference to write side of the pipe fd, the pipe will not be closed even if the writer has exited, due to the final reference from the parent. This means, the read of the reader child shall block forever...
On Ubuntu 16 I am trying to write a program exercising pipes, forking, and execing:
the program will accept a file name via a command-line argument;
a child process will open the named file and exec cat to transfer the content to a second child process; and
the second child will exec grep to select the lines that contain numbers for forwarding to a third child process
the third child process prints the received lines.
Here's my code:
#include<stdio.h>
#include<stdlib.h>
#include<unistd.h>
#include <string.h>
#include <sys/types.h>
#include<sys/wait.h>
#include<fcntl.h>
#define BLOCK_SIZE 4096
int main(int argc, char** argv)
{
int PID;
int pipe1[2];
int pipe2[2];
int pipe3[2];
char fileName[256];
int lengthfileName = strlen(argv[1]);
char content[BLOCK_SIZE];
char modifiedContent[BLOCK_SIZE];
int file;
if(argc < 2)
{
printf("Usage prog file\n");
exit(1);
}
if(pipe(pipe1) < 0)
{
printf("Error at pipe\n");
exit(1);
}
if(pipe(pipe2) < 0)
{
printf("Error at pipe\n");
exit(1);
}
if(pipe(pipe3) < 0)
{
printf("Error at pipe\n");
exit(1);
}
if((PID = fork()) < 0)
{
printf("Error at process\n");
exit(1);
}
if(PID == 0) //first child
{
close(pipe1[1]);
read(pipe1[0],fileName,lengthfileName);
close(pipe1[0]);
close(pipe2[0]);
dup2(pipe2[1],1);
close(pipe2[1]);
execlp("/bin/cat","cat",fileName,NULL);
exit(0);
}
else // parent
{
close(pipe1[0]);
write(pipe1[1],argv[1],lengthfileName);
close(pipe1[1]);
int status;
if((PID = fork()) < 0)
{
printf("Error at process\n");
exit(1);
}
if(PID == 0) // child 2
{
close(pipe2[1]);
//read(pipe2[0],content,BLOCK_SIZE);
//dup2(pipe2[0],0);// ***********************MARKED LINE HERE *****************************************
close(pipe2[0]);
close(pipe3[0]);
dup2(pipe3[1],1);
close(pipe3[1]);
execlp("grep","grep","[0-9]",NULL);
exit(0);
}
if((PID = fork()) < 0)
{
printf("Error at process\n");
exit(1);
}
if(PID == 0) //cod fiu 2
{
close(pipe3[1]);
read(pipe3[0],modifiedContent,BLOCK_SIZE);
close(pipe3[0]);
printf("GOT FROM PIPE:%s",modifiedContent);
exit(0);
}
waitpid(PID, &status, 0);
}
return 0;
}
My problem is inside the child process 2 code, where I try to use the pipeline as input for grep. As presented the input is taken from the terminal; if I uncomment the marked lines then the program hangs, and I have to manually kill it to make it stop.
What's wrong with how I'm using pipe2 to feed data to grep in child process 2? Or is the problem somewhere else?
It's a bit silly that you transfer the file name to the first child via a pipe, but rely on that child inheriting its length from its parent. If you're going to inherit the name's length, then you might as well inherit the whole file name, dispensing with the first pipe.
You could conceivably send the (fixed-size) length value over the pipe first to avoid inheriting it, but such a scheme is pointless -- not only do forked child processes inherit data from their parents, you cannot avoid relying on that in your program. In particular, the children must inherit the open pipe ends and the arrays of pipe file descriptors from the parent for the single-parent approach to work at all.
Note also that you are (maybe) lucking into null termination of the file name received over the pipe. The first child neither reads it from the pipe nor sets it explicitly.
But the main problem appears to be that you have stray open pipe ends. You create all three pipes in the parent, before forking any children. At each fork, the child will therefore inherit the open file descriptors for all pipe ends that the parent has not yet closed. The child processes should close all of the open pipe ends they do not use, but they only close some of them. Programs such as grep (and cat) don't exit until they see the end of the file, and they won't see that on a pipe while any process holds the write end open.
Specifically, the parent process never closes the write end of pipe2, and in fact the third child inherits that open descriptor and also does not close it. The first child closes its copy of that FD when it exits, but with two other handles on the pipe end open, that end remains open. Therefore, when the second child is taking its input from that pipe, it never sees end-of-file, and never exits. Making the parent close both ends of pipe2 between forking the second child and forking the third child should solve that problem.
I am trying to execute a linux command by using execvp in c and it does not show anything. I dont quite understand the dup2() function call. Can any one tell me what i am doing wrong here?
Command To be executed: ls | sort
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <stdbool.h>
int main()
{
char *cmd[2], *cmd2[3];
cmd[0] = "ls";
cmd[1] = NULL;
cmd2[0] = "sort";
cmd2[1] = NULL;
pid_t id;
id = fork();
int pipe1[2];
int pipe2[2];
pipe(pipe1);
pipe(pipe2);
if(id==0)
{
printf("child process with pid =%d \r\n", getpid());
//the read end of pipe not needed,
close(pipe1[0]);
// we want the stdout (1) of this process to be connected to 1 of pipe1.
dup2(pipe1[1],1); //or dups(pipe1[1], stdout_fileno)
//close it aswell not needed
//close(pipe1[1]);
execvp("ls",cmd);
return 1;
}
pid_t id2 = fork();
if(id2==0)
{
close(pipe1[1]);
dup2(pipe1[0], 0);
close(pipe1[0]);
execvp("sort",cmd2);
//return 1;
}
close(pipe1[0]);
close(pipe1[1]);
waitpid(id,NULL,0);
waitpid(id2, NULL, 0);
//execvp("sort",cmd2);
//printf("parent process with pid =%d \r\n", getpid());
return 0;
}
You need only one pipe, and in fact you use only one, even though you create two. Except you actually create four, because you create the pipes after the first fork(), thus the parent and first child process create two each.
And that appears to be the problem. If you want the two child processes to communicate over a pipe, then they must use the same pipe. They would do so by inheriting the open file descriptors for the pipe ends from the parent process, which requires that they use a pipe created by a process from which both descend (i.e. the parent process), and which was created before either child was forked. As it stands, only the second child is doing that; the first is using a pipe that it creates itself, and which therefore is not connected to anything else.
I'm creating various processes (3 to be precise) and making them do different things.
So far so good. I'm trying to wait in the parent until all children are completed. I've played around with many options (such as the one listed below) but either the parent waits but I have to press enter to return to the shell (meaning that some child completes after the parent?) or the parent never returns to the shell. Any ideas? pointers to where to look for more help? Thanks
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#define READ_END 0
#define WRITE_END 1
int main (int argc, char **argv)
{
pid_t pid;
int fd[2];
int fd2[2];
pipe(fd);
pipe(fd2);
for (int i=0; i<3; i++) {
pid=fork();
if (pid==0 && i==0) {
//never uses fd2, so close both descriptors
close(fd2[READ_END]);
close(fd2[WRITE_END]);
printf("i'm the child used for ls \n");
close(fd[READ_END]); /*close read end since I don't need it */
dup2(fd[WRITE_END], STDOUT_FILENO);
close(fd[WRITE_END]);
execlp("ls", "ls", "-hal", NULL);
break; /*exit for loop to end child's code */
}
else if (pid==0 && i==1) {
printf("i'm in the second child, which will be used to run grep\n");
close(fd[WRITE_END]);
dup2(fd[READ_END], STDIN_FILENO);
close(fd[READ_END]);
close(fd2[READ_END]);
dup2(fd2[WRITE_END], STDOUT_FILENO);
close(fd2[WRITE_END]);
execlp("grep", "grep","p",NULL);
break;
}
else if (pid==0 && i==2) {
//never uses fd so close both descriptors
close(fd[READ_END]);
close(fd[WRITE_END]);
printf("i'm in the original process which will be replaced with wc\n");
close(fd2[WRITE_END]);
dup2(fd2[READ_END], STDIN_FILENO);
close(fd2[READ_END]);
printf("going to exec wc\n");
execlp("wc","wc","-w",NULL);
break;
}
else {
//do parenty things
}
}
wait(NULL);
while (1){
wait(NULL);
if(errno== ECHILD) {
printf("all children ended\n");
break;
}
}
close(fd[READ_END]);
close(fd[WRITE_END]);
close(fd2[READ_END]);
close(fd2[WRITE_END]);
return 0;
}
grep and wc never exit.
Why? They never receive an EOF on stdin.
Why? Because, even though ls has exited and closed the write end of pipe(fd), the main process still has the write end of pipe(fd) open, thus the read end of pipe(fd) is still waiting for more data. Similar thing goes for fd2: even if grep exited, wc wouldn't get an EOF on stdin.
Solution: close all the pipe fds in the main process before you wait.