Piping implementation stuck on second command - c

Question

int main() {
char *cmd1[2] = { "ls", NULL };
char *cmd2[3] = { "grep", "a", NULL };
char *cmd3[3] = { "wc", "-l", NULL };
char *cmd4[5] = { "cat", NULL };
char *cmd5[5] = { "cat", NULL };
int pipe_count = 2;
int pid1, pid2, pid3, pid4, pid5;
int pfd1[2];
int pfd2[2];
pipe(pfd1);
pipe(pfd2);
if ((pid1 = fork()) == 0) {
close(pfd2[0]);
close(pfd2[1]);
close(pfd1[0]);
dup2(pfd1[1], 1);
if (execvp(cmd1[0], cmd1) == -1) {
exit(-1);
}
} else if (pid1 > 0) {
waitpid(pid1, NULL, 0);
}
if ((pid2 = fork()) == 0) {
if (pipe_count >= 2) {
close(pfd1[1]);
close(pfd2[0]);
dup2(pfd1[0], 0);
dup2(pfd2[1], 1);
} else {
close(pfd1[1]);
close(pfd2[0]);
close(pfd2[1]);
dup2(pfd1[0], 0);
}
if (execvp(cmd2[0], cmd2) == -1) {
exit(-1);
}
if (pipe_count == 1) {
printf("\n");
return 0;
}
} else if (pid2 > 0) {
waitpid(pid2, NULL, 0);
}
if (pipe_count >= 2) {
if ((pid3 = fork()) == 0) {
if (pipe_count >= 3) {
close(pfd1[0]);
close(pfd2[1]);
dup2(pfd2[0], 0);
dup2(pfd1[1], 1);
} else {
close(pfd1[0]);
close(pfd1[1]);
close(pfd2[1]);
dup2(pfd2[0], 0);
}
if (execvp(cmd3[0], cmd3) == -1) {
exit(-1);
}
if (pipe_count == 2) {
printf("\n");
}
} else if (pid3 > 0) {
waitpid(pid3, NULL, 0);
}
}
if (pipe_count >= 3) {
if ((pid4 = fork()) == 0) {
close(pfd1[1]);
close(pfd2[0]);
dup2(pfd1[0], 0);
if (pipe_count == 4)
dup2(pfd2[1], 1);
else
close(pfd2[1]);
if (execvp(cmd4[0], cmd4) == -1) {
exit(-1);
}
} else if (pid4 > 0) {
waitpid(pid4, NULL, 0);
}
}
if (pipe_count == 4) {
if ((pid5 = fork()) == 0) {
close(pfd1[0]);
close(pfd2[1]);
dup2(pfd2[0],0);
close(pfd1[1]);
if (execvp(cmd5[0], cmd5) == -1) {
exit(-1);
}
} else if (pid5 > 0) {
waitpid(pid5, NULL, 0);
}
}
return 0;
}
I'm trying to build a shell with piping command. When I input ls | grep a | wc -l for example, I realize that the program is stuck on grep a when I use ps f on the terminal. The shell is not responsive.
When I kill the child process for grep a, I'm again stuck on wc -l and have to kill it on the terminal again.
After killing the processes, no output is printed (My desired output is 2).
Any help would be appreciated.

Answer 1

As already diagnosed in the comments, there are many problems with the original code, including:
The most likely problem is that your parent process isn't closing the pipes before waiting for the child processes to die, so the child processes don't get EOF and don't terminate. (This was one of the problems, but far from the only problem.)
If you have N processes to run, you need N-1 pipes. You have only two pipes here; you've got a lot of work to do before you can make it work with just two pipes. The case of N=2 still has special cases: the first and last processes need to be treated a bit different from the way you treat processes 2..N-1.
You also need to run the processes in a pipeline concurrently. The controlling process should not wait for any of the children until the whole pipeline has been launched. This is because, for example, process P1 may generate so much data that it fills the pipe buffer connecting it to process P2, at which point it will block waiting for P2 to read some data. But if P2 hasn't been launched yet, P1 will never be unblocked, so the pipeline will make no progress. You need to rethink the waiting code as well as the piping code. You end up closing a lot of file descriptors.
You aren't closing enough file descriptors. Rule of thumb: If you dup2() one end of a pipe to standard input or standard output, close both of the original file descriptors from pipe() as soon as possible. In particular, that means before using any of the exec*() family of functions. The rule also applies with either dup() or fcntl() with F_DUPFD.
Note that there is no need to test the return value from the exec*() family of functions. If they succeed, they do not return; if they return, they failed. I think it is good practice, in most cases, to generate an error message before exiting after an exec*() call fails.
Putting those observations together leads to code like this:
/* SO 7412-0402 */
#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>
#include <stderr.h>
int main(void)
{
char *cmd1[2] = { "ls", NULL };
char *cmd2[3] = { "grep", "a", NULL };
char *cmd3[3] = { "wc", "-l", NULL };
char *cmd4[5] = { "cat", NULL };
char *cmd5[5] = { "cat", NULL };
int pid1, pid2, pid3, pid4, pid5;
int pfd1[2];
int pfd2[2];
int pfd3[2];
int pfd4[2];
err_setarg0("pipe61");
err_setlogopts(ERR_PID | ERR_MILLI);
err_remark("Parent process\n");
if (pipe(pfd1) != 0 ||
pipe(pfd2) != 0 ||
pipe(pfd3) != 0 ||
pipe(pfd4) != 0)
err_syserr("failed to create a pipe: ");
if ((pid1 = fork()) < 0)
err_syserr("failed to fork(): ");
if (pid1 == 0)
{
err_remark("Child process 1\n");
dup2(pfd1[1], 1);
close(pfd1[0]); close(pfd1[1]);
close(pfd2[0]); close(pfd2[1]);
close(pfd3[0]); close(pfd3[1]);
close(pfd4[0]); close(pfd4[1]);
execvp(cmd1[0], cmd1);
err_syserr("failed to execute '%s': ", cmd1[0]);
}
if ((pid2 = fork()) < 0)
err_syserr("failed to fork(): ");
else if (pid2 == 0)
{
err_remark("Child process 2\n");
dup2(pfd1[0], 0);
dup2(pfd2[1], 1);
close(pfd1[0]); close(pfd1[1]);
close(pfd2[0]); close(pfd2[1]);
close(pfd3[0]); close(pfd3[1]);
close(pfd4[0]); close(pfd4[1]);
execvp(cmd2[0], cmd2);
err_syserr("failed to execute '%s': ", cmd2[0]);
}
if ((pid3 = fork()) < 0)
err_syserr("failed to fork(): ");
else if (pid3 == 0)
{
err_remark("Child process 3\n");
dup2(pfd2[0], 0);
dup2(pfd3[1], 1);
close(pfd1[0]); close(pfd1[1]);
close(pfd2[0]); close(pfd2[1]);
close(pfd3[0]); close(pfd3[1]);
close(pfd4[0]); close(pfd4[1]);
execvp(cmd3[0], cmd3);
err_syserr("failed to execute '%s': ", cmd3[0]);
}
if ((pid4 = fork()) < 0)
err_syserr("failed to fork(): ");
else if (pid4 == 0)
{
err_remark("Child process 4\n");
dup2(pfd3[0], 0);
dup2(pfd4[1], 1);
close(pfd1[0]); close(pfd1[1]);
close(pfd2[0]); close(pfd2[1]);
close(pfd3[0]); close(pfd3[1]);
close(pfd4[0]); close(pfd4[1]);
execvp(cmd4[0], cmd4);
err_syserr("failed to execute '%s': ", cmd4[0]);
}
if ((pid5 = fork()) < 0)
err_syserr("failed to fork(): ");
else if (pid5 == 0)
{
err_remark("Child process 5\n");
dup2(pfd4[0], 0);
close(pfd1[0]); close(pfd1[1]);
close(pfd2[0]); close(pfd2[1]);
close(pfd3[0]); close(pfd3[1]);
close(pfd4[0]); close(pfd4[1]);
execvp(cmd5[0], cmd5);
err_syserr("failed to execute '%s': ", cmd5[0]);
}
close(pfd1[0]); close(pfd1[1]);
close(pfd2[0]); close(pfd2[1]);
close(pfd3[0]); close(pfd3[1]);
close(pfd4[0]); close(pfd4[1]);
int status;
int corpse;
while ((corpse = wait(&status)) > 0)
printf("%d: child %d exited with status 0x%.4X\n", getpid(), corpse, status);
return 0;
}
Notice that the blocks for pid2, pid3 and pid4 are almost the same; the block for pid1 only duplicates a pipe descriptor to stdout, while the block for pid5 only duplicates a pipe descriptor to stdin.
The code for the error reporting routines is available in my SOQ (Stack Overflow Questions) repository on GitHub as files stderr.c and stderr.h in the src/libsoq sub-directory. The err(3) functions on Linux and BSD have similar functionality but different function names.
Here is the output from a sample run (of the program pipe61 compiled from source code pipe61.c shown above.
pipe61: 2022-10-19 23:52:03.833 - pid=50391: Parent process
pipe61: 2022-10-19 23:52:03.834 - pid=50392: Child process 1
pipe61: 2022-10-19 23:52:03.834 - pid=50393: Child process 2
pipe61: 2022-10-19 23:52:03.834 - pid=50394: Child process 3
pipe61: 2022-10-19 23:52:03.834 - pid=50395: Child process 4
pipe61: 2022-10-19 23:52:03.834 - pid=50396: Child process 5
50391: child 50392 exited with status 0x0000
50391: child 50393 exited with status 0x0000
16
50391: child 50394 exited with status 0x0000
50391: child 50395 exited with status 0x0000
50391: child 50396 exited with status 0x0000
Clearly, this code is not easily configurable to deal with 6 or more stages in the pipeline except by cut'n'paste programming, nor is it trivial to remove any stages from the pipeline (variable renaming). For 'real' code, you'd need to use an array-driven approach to avoid unnecessary duplication of code. You'd have the pipe descriptors stored in an array; you'd have the PIDs stored in an array. You'd probably have an array of pointers to the lists of command arguments — three-star programming. And you'd probably use a function to launch the Nth child process.