How to chain multiple pipes? (plus weird buggs) - c

I'm trying to write a simple shell that can handle pipe commands. I want to be able to handle multiple pipes all chained together but I'm having a hard time figuring out how to implement something like this.
This is my current attempt:
int status;
int lastToken = 0;
int pipe_pid;
//create the pipes
int pipefd[pipes][2];
// Loop to run all commands in the vertical list.
while(1){
if (c->type == TOKEN_PIPE){
// Here is where we deal with pipes
for (int i = 0; i < pipes; i++){
pipe(pipefd[i]);
pipe_pid = fork();
//this is a receiving pipe
if (pipe_pid == 0){
// create the write end of the pipe
dup2(pipefd[i][WRITE_SIDE], STDOUT_FILENO);
close(pipefd[i][READ_SIDE]);
close(pipefd[i][WRITE_SIDE]);
execvp(c->argv[0], c->argv);
// printf("parent pipe\n");
}
//this is a writing pipe
else{
close(pipefd[i][WRITE_SIDE]);
dup2(pipefd[i][READ_SIDE], STDIN_FILENO);
close(pipefd[i][READ_SIDE]);
// printf("child pipe\n");
}
}
// This stuff happens for all commands
lastToken = c->type;
// If it's the last command, we're done
if (c->next == NULL){
break;
}
else{
c = c->next;
}
}
the commands are chained together in a linked list, c is my command pointer
pipes is a variable that I create as I parse the in-string, so I know how many '|' I saw in the command. This should tell me the number of child processes I need to fork.
I use pipes to create a 2d array for the pipe descriptors.
Then I want to loop over the pipes and fork once for each, and use dup2 to map the inputs and outputs.
I'm getting inconsistent errors that I can't figure out. First of all, every time I run a pipe command, my shell immediately crashes with no segfault or other printed errors.
Second, if I run commands like echo foo | wc -c I sometimes get 4 and sometimes get 0 as the output.
I'm sure I'm just doing something dumb but I'm not sure what :/

I figured out what I was doing wrong, I was closing the pipes before all the threads were finished using them. I fixed it by pulling out the close calls.
// writing side of the pipe
if (c->type == TOKEN_PIPE){
close(c->pipefd[READ_SIDE]);
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
}
// receiving side of the pipe
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[WRITE_SIDE]);
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
}
And then in the parent thread, right before I reep my zombies, I check for pipes that are finished being used and close them.
// writing side of the pipe
if (c->type == TOKEN_PIPE){
close(c->pipefd[READ_SIDE]);
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
}
// receiving side of the pipe
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[WRITE_SIDE]);
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
close(commandPrev->pipefd[READ_SIDE]);
I'm not sure if this is the optimal way to do it but it works without errors for me.

Related

Child not reading output from another child that put it in the pipe

I've been working on this school assignment forever now, and I'm super close to finishing.
The assignment is to create a bash shell in C, which sounds basic enough, but it has to support piping, IO redirect, and flags within the piped commands. I have it all working except for one thing; the | piping child isn't getting any of the data written to the pipe by the user command process child. If I were to remove the child fork for pipechild, and have everything from if(pipe_cmd[0] != '\0') run as the parent, it would work just fine (minus ending the program because of execlp). If I were to use printf() inside the pipe section, the output would be in the right file or terminal, which just leaves the input from the user command process child not getting to where it needs to be as a culprit.
Does anyone see an issue on how I'm using the pipe? It all felt 100% normal to me, given the definition of a pipe.
int a[2];
pipe(a);
//assume file_name is something like file.txt
strcat(file_name, "file.txt");
strcat(pipe_cmd, "wc");
if(!fork())
{
if(pipe_cmd[0] != '\0') // if there's a pipe
{
close(1); //close normal stdout
dup(a[1]); // making stdout same as a[1]
close(a[0]); // closing other end of pipe
execlp("ls","ls",NULL);
}
else if(file_name[0] != '\0') // if just a bare command with a file redirect
{
int rootcmd_file = open(file_name, O_APPEND|O_WRONLY|O_CREAT, 0644);
dup2(rootcmd_file, STDOUT_FILENO);
execlp("ls","ls",NULL); // writes ls to the filename
}
// if no pipe or file name write...
else if(rootcmd_flags[0] != '\0') execlp("ls","ls",NULL)
else execlp("ls","ls",NULL);
} else wait(0);
if(pipe_cmd[0] != '\0') // parent goes here, if pipe.
{
pipechild = fork();
if(pipechild != 0) // *PROBLEM ARISES HERE- IF THIS IS FORKED, IT WILL HAVE NO INFO TAKEN IN.
{
close(0); // closing normal stdin
dup(a[0]); // making our input come from the child above
close(a[1]); // close other end of pipe
if(file_name[0] != '\0') // if a filename does exist, we must reroute the output to the pipe
{
close(1); // close normal stdout
int fileredir_pipe = open(file_name, O_APPEND|O_WRONLY|O_CREAT, 0644);
dup2(fileredir_pipe, STDOUT_FILENO); //redirects STDOUT to file
execlp("wc","wc",NULL); // this outputs nothing
}
else
{
// else there is no file.
// executing the pipe in stdout using execlp.
execlp("wc","wc",NULL); // this outputs nothing
}
}
else wait(0);
}
Thanks in advance. I apologize for some of the code being withheld. This is still an active assignment and I don't want any cases of academic dishonesty. This post was risky enough.
} else wait(0);
The shown code forks the first child process and then waits for it to terminate, at this point.
The first child process gets set up with a pipe on its standard output. The pipe will be connected to the second child process's standard input. The fatal flaw in this scheme is that the second child process isn't even started yet, and won't get started until the first process terminates.
Pipes have limited internal buffering. If the first process generates very little output chances are that its output will fit inside the tiny pipe buffer, it'll write its output and then quietly terminate, none the wiser.
But if the pipe buffer becomes full, the process will block and wait until something reads from the pipe and clears it. It will wait as long as it takes for that to happen. And wait, and wait, and wait. And since the second child process hasn't been started yet, and the parent process is waiting for the first process to terminate it will wait, in vain, forever.
This overall logic is fatally flawed for this reason. The correct logic is to completely fork and execute all child processes, close the pipe descriptors in the parent (this is also important), and then wait for all child processes to terminate. wait must be the very last thing that happens here, otherwise things will break in various amazing and mysterious ways.

Piping output to forks in C

I have a multiple processes in progs and I wish to pipe the output from one process to another sequentially. I believe I have already linked the stdin of a process to the read end of a previous process, and linked the stdout to the write end of the pipe. I still am not seeing an output. Am I missing something with the links here?
int pipeFds[numProg][2];
for (int i = 0; i < numProg; i++) {
if (pipe(pipeFds[i]) != 0) { // create pipe for each process
printf("Failed to create pipe!");
exit(1);
}
child_pid = fork();
if (child_pid == -1) {
printf("Error while creating fork!");
exit(1);
}
if (child_pid == 0) {
if (i == 0) {
close(pipeFds[i][READ_END]); // close STDIN first process since it does not read
} else {
// change stdin to read end of pipe for intermediary processes
close(pipeFds[i]);
dup2(pipeFds[i - 1][READ_END], STDIN_FILENO);
}
dup2(pipeFds[i][WRITE_END], STDOUT_FILENO); // change stdout to write end of pipe
execvp(progs[i][0], (char *const * )progs[i]);
} else {
// parent process stuff
}
}
// Close pipes except last pipe for EOF
for (int i = 0; i < numProg - 1; i++) {
close(pipeFds[i][READ_END]);
close(pipeFds[i][WRITE_END]);
}
Remember you need to close all pipes, in each process, for it to work.
Example:
If numProg=2 you create 2 pipes and 2 child processes. Adding the parent, there a 3 processes running, and in each of them you eventually need to close all pipes.
For child_pid==0 you close the [READ_END] but never the [WRITE_END].
For child_pid==1 you do a close(pipeFds[1]). You need to specify [READ_END] and [WRITE_END].
Then each child process exits via execvp which, however, may return control if it fails. From the man page:
The exec() family of functions replaces the current process image with a new process image. .. The exec() functions only return if an error has occurred.
So you may want to add a _exit(0); after execvp to ensure each child process exits properly even if execvp fails.
The parent process closes all pipes but the last. So in the example of NumProg=2, [READ_END] and [WRITE_END] of pipeFd[1] both are never closed.
Lastly, the parent process should wait for all child processes to close (using while(wait(NULL) != -1);) otherwise you may end up with zombie or orphan processes.
your code contains a stray close(pipeFds[i]);
you have to close the pipes within the // parent process stuff. With your code, every child keeps the pipeFds of the previous children open. E.g.
} else {
// parent process stuff
if (i > 0)
close(pipeFds[i - 1][READ_END]);
close(pipeFds[i - 1][READ_END]);
}
it might be more effective to have a single fds[2] pair instead of numProg ones.

Why do my pipes not talk to each other?

I am trying to write a simple shell in c. Right now I'm trying to get pipes to work. I have a struct c which I'm feeding into this function, it contains a place to store the pipe file descriptor pipfd and also contains information about the ending tag of each command c->type (this can be | || && & etc). CommandPrev is just tracking the last command so I can see if the command immediately before had a pipe tag.
After I finish this function, I give the child pid (the return value) to waitpid to wait on the command I called with execvp
When I run commands such as echo foo | echo bar I get bar as an output exactly as I would expect and everything works great. My problem is when I try to run any command that actually uses the input from the first half of the pipe, everything gets stuck. If I run something like echo foo | wc -c I get no output and it just hangs forever.
I can see that this function finishes for these sort of commands because I print when it returns. What's happening is that the command that I'm calling with execvp is never happening so my waitpid waits forever.
I think that somehow my connection between the two ends of my pipe is broken. Either things are never getting written, or they're never being read, or the receiving end of the pipe never realizes that the writing side is finished and is just waiting forever. I call close immediately on all my pipes so I tend to doubt its the last one... but I'm really not sure how to go about testing any of these three scenarios.
This is my code:
pid_t start_command(command* c, pid_t pgid) {
(void) pgid;
// If its a pipe token, create a shared pipe descriptor
if (c->type == TOKEN_PIPE){
pipe(c->pipefd);
}
// Fork a child process, run the command using `execvp`
pid_t child = fork();
if (child == 0) {
// writing side of the pipe
if (c->type == TOKEN_PIPE){
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
close(c->pipefd);
}
// receiving side of the pipe
else if (commandPrev->type == TOKEN_PIPE){
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
close(commandPrev->pipefd);
}
// run the command
if (execvp(c->argv[0], c->argv) == -1) {
// fork failed
exit(-1);
}
}
else{
// clean up, clean up, everybody, everywhere
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd);
}
}
printf("return %i\n", getpid() );
return child;
}
Thank you!
As the other commenter says, you look like you're trying to close an array.
Something like this should work better:
// writing side of the pipe
if (c->type == TOKEN_PIPE){
close(c->pipefd[READ_SIDE]);
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
close(c->pipefd[WRITE_SIDE]);
}
// receiving side of the pipe
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[WRITE_SIDE]);
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
close(commandPrev->pipefd[READ_SIDE]);
}
Alternatively, you can close the active sides of the pipe after a waitpid call in the parent. Something like this:
waitpid(child, &status, 0);
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[READ_SIDE]);
}
if (c->type == TOKEN_PIPE){
close(c->pipefd[WRITE_SIDE]);
}

Supporting a single pipe with a custom shell in C

I was implementing a custom shell in C which required me to implement simple commands like echo,ls etc and input/output redirection.My project is done but when I was searching for some tips I found that there is a book which asked for a custom shell to support a single pipe so I tried to code that.First of all I tried to make a pipe in a normal c program to make sure that I can implement pipes.I was able to do it but somehow I have difficulties in doing it in my custom shell.
int mypipe[2];
if (pipe(mypipe) == -1) {
perror("Pipe failed");
exit(1);
}
firstchild=fork();
if (firstchild == 0) { // first child
close(mypipe[1]); //Closing the output of pipe
read1 = read(mypipe[0], buffer, sizeof(buffer));
printf(" %s\n", buffer);
myDate(child1, child2, result);
write(mypipe2[1], result, strlen(result)+1);
} else {
secondchild=fork(); //Creating second child
if(secondchild == 0) { //2nd child
read1 = read(mypipe2[0], buffer, sizeof(buffer));
printf(" %s\n", buffer);
I believe this code is correct since I could send time from child1 to child 2 and print it with child 2 but I cannot apply the same concept to my own shell where there is a parent who is waiting for child to terminate.And there are so many execv in my code which confuses me about what will happen in the future.And I only created one child process in my custom shell so I believe I need to create another which again is another problem for me since it will be hard for me to synchronize it with exec vs.Here is a small part of my own shell.
}if(!strcmp(args[0] ,"clear")){
execv("/usr/bin/clear",args);
}if(!strcmp(args[0] ,"env")){
execv("/usr/bin/env",args);
}
perror("Error");
exit(1); //Failure
}else{
do{
waitpid(child,&status,WUNTRACED);
}while(!WIFEXITED(status) && !WIFSIGNALED(status));
As you can guess else statement is coming from if(child=0).I would be really glad if someone can show me how to support a single pipe in a structure like this.My ultimate purpose is trying to support multiple pipes but I think I can figure it out myself when I see a single pipe

execvp() inside a pipeline causing whole program to die

I'm currently writing a shell in C, and I'm having trouble with my pipeline.
I call the entire function process() line by line, with a parsed tree of commands as the input. process() works perfectly fine on simple commands and redirections so far, so I know that's not the problem.
In the below, my cmdList is an already-parsed I've structured my pipeline such that the child pipes its stdout into the stdin of the parent function. The child represents the left tree of cmdList, while the parent represents the right side. Here's what it looks like when I try to recursively call process():
else if (cmdList->type==PIPE)
{
int pipefd[2];
pid_t fork_result;
pipe(pipefd);
fork_result=fork();
if (fork_result < 0) {
fprintf(stderr, "Fork failure2\n");
exit(EXIT_FAILURE);
}
if (fork_result==0) {
int jo3 = close(pipefd[0]);
int jo1 = dup2(pipefd[1], STDOUT_FILENO);
process(cmdList->left);
exit(EXIT_failure);
}
else
{
close(pipefd[1]);
int status;
wait(&status);
dup2(pipefd[0], STDIN_FILENO);
process(cmdList->right); //dies here
fprintf(stderr,"FAIL\n");
exit(EXIT_FAILURE);
}
}
This causes my entire program to die, including the main function that calls process(). If I replace my recurisve process() calls with
execvp(cmdList->left->argv[0],cmdList->left->argv);
execvp(cmdList->right->argv[0],cmdList->right->argv);
then my program runs forever.
Things I've already tried:
All dup2's and wait's return as expected, without error.
Using a debugger, the program always gets to the execvp's with everything doing fine.
As stated, the rest of process() works perfectly on everything not having to do with the pipeline.
I'm not really sure what's going wrong with this. Is there anything that stands out?

Resources