I am trying to write a simple shell in c. Right now I'm trying to get pipes to work. I have a struct c which I'm feeding into this function, it contains a place to store the pipe file descriptor pipfd and also contains information about the ending tag of each command c->type (this can be | || && & etc). CommandPrev is just tracking the last command so I can see if the command immediately before had a pipe tag.
After I finish this function, I give the child pid (the return value) to waitpid to wait on the command I called with execvp
When I run commands such as echo foo | echo bar I get bar as an output exactly as I would expect and everything works great. My problem is when I try to run any command that actually uses the input from the first half of the pipe, everything gets stuck. If I run something like echo foo | wc -c I get no output and it just hangs forever.
I can see that this function finishes for these sort of commands because I print when it returns. What's happening is that the command that I'm calling with execvp is never happening so my waitpid waits forever.
I think that somehow my connection between the two ends of my pipe is broken. Either things are never getting written, or they're never being read, or the receiving end of the pipe never realizes that the writing side is finished and is just waiting forever. I call close immediately on all my pipes so I tend to doubt its the last one... but I'm really not sure how to go about testing any of these three scenarios.
This is my code:
pid_t start_command(command* c, pid_t pgid) {
(void) pgid;
// If its a pipe token, create a shared pipe descriptor
if (c->type == TOKEN_PIPE){
pipe(c->pipefd);
}
// Fork a child process, run the command using `execvp`
pid_t child = fork();
if (child == 0) {
// writing side of the pipe
if (c->type == TOKEN_PIPE){
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
close(c->pipefd);
}
// receiving side of the pipe
else if (commandPrev->type == TOKEN_PIPE){
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
close(commandPrev->pipefd);
}
// run the command
if (execvp(c->argv[0], c->argv) == -1) {
// fork failed
exit(-1);
}
}
else{
// clean up, clean up, everybody, everywhere
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd);
}
}
printf("return %i\n", getpid() );
return child;
}
Thank you!
As the other commenter says, you look like you're trying to close an array.
Something like this should work better:
// writing side of the pipe
if (c->type == TOKEN_PIPE){
close(c->pipefd[READ_SIDE]);
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
close(c->pipefd[WRITE_SIDE]);
}
// receiving side of the pipe
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[WRITE_SIDE]);
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
close(commandPrev->pipefd[READ_SIDE]);
}
Alternatively, you can close the active sides of the pipe after a waitpid call in the parent. Something like this:
waitpid(child, &status, 0);
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[READ_SIDE]);
}
if (c->type == TOKEN_PIPE){
close(c->pipefd[WRITE_SIDE]);
}
Related
I've been working on this school assignment forever now, and I'm super close to finishing.
The assignment is to create a bash shell in C, which sounds basic enough, but it has to support piping, IO redirect, and flags within the piped commands. I have it all working except for one thing; the | piping child isn't getting any of the data written to the pipe by the user command process child. If I were to remove the child fork for pipechild, and have everything from if(pipe_cmd[0] != '\0') run as the parent, it would work just fine (minus ending the program because of execlp). If I were to use printf() inside the pipe section, the output would be in the right file or terminal, which just leaves the input from the user command process child not getting to where it needs to be as a culprit.
Does anyone see an issue on how I'm using the pipe? It all felt 100% normal to me, given the definition of a pipe.
int a[2];
pipe(a);
//assume file_name is something like file.txt
strcat(file_name, "file.txt");
strcat(pipe_cmd, "wc");
if(!fork())
{
if(pipe_cmd[0] != '\0') // if there's a pipe
{
close(1); //close normal stdout
dup(a[1]); // making stdout same as a[1]
close(a[0]); // closing other end of pipe
execlp("ls","ls",NULL);
}
else if(file_name[0] != '\0') // if just a bare command with a file redirect
{
int rootcmd_file = open(file_name, O_APPEND|O_WRONLY|O_CREAT, 0644);
dup2(rootcmd_file, STDOUT_FILENO);
execlp("ls","ls",NULL); // writes ls to the filename
}
// if no pipe or file name write...
else if(rootcmd_flags[0] != '\0') execlp("ls","ls",NULL)
else execlp("ls","ls",NULL);
} else wait(0);
if(pipe_cmd[0] != '\0') // parent goes here, if pipe.
{
pipechild = fork();
if(pipechild != 0) // *PROBLEM ARISES HERE- IF THIS IS FORKED, IT WILL HAVE NO INFO TAKEN IN.
{
close(0); // closing normal stdin
dup(a[0]); // making our input come from the child above
close(a[1]); // close other end of pipe
if(file_name[0] != '\0') // if a filename does exist, we must reroute the output to the pipe
{
close(1); // close normal stdout
int fileredir_pipe = open(file_name, O_APPEND|O_WRONLY|O_CREAT, 0644);
dup2(fileredir_pipe, STDOUT_FILENO); //redirects STDOUT to file
execlp("wc","wc",NULL); // this outputs nothing
}
else
{
// else there is no file.
// executing the pipe in stdout using execlp.
execlp("wc","wc",NULL); // this outputs nothing
}
}
else wait(0);
}
Thanks in advance. I apologize for some of the code being withheld. This is still an active assignment and I don't want any cases of academic dishonesty. This post was risky enough.
} else wait(0);
The shown code forks the first child process and then waits for it to terminate, at this point.
The first child process gets set up with a pipe on its standard output. The pipe will be connected to the second child process's standard input. The fatal flaw in this scheme is that the second child process isn't even started yet, and won't get started until the first process terminates.
Pipes have limited internal buffering. If the first process generates very little output chances are that its output will fit inside the tiny pipe buffer, it'll write its output and then quietly terminate, none the wiser.
But if the pipe buffer becomes full, the process will block and wait until something reads from the pipe and clears it. It will wait as long as it takes for that to happen. And wait, and wait, and wait. And since the second child process hasn't been started yet, and the parent process is waiting for the first process to terminate it will wait, in vain, forever.
This overall logic is fatally flawed for this reason. The correct logic is to completely fork and execute all child processes, close the pipe descriptors in the parent (this is also important), and then wait for all child processes to terminate. wait must be the very last thing that happens here, otherwise things will break in various amazing and mysterious ways.
I am working on a tiny shell(tsh) implemented in C(it's an assignment). One part of assignment belongs to PIPING. I have to pipe a command's output to another command. e.g:ls -l | sort
When I run the shell, every command that I execute on it, is processed by a child process that it spawns. After the child finishes the result is returned. For piping I wanted to implement a harcoded example first to check how it works. I wrote a method, that partially works. The problems is when I run the pipe command, after child process finishes, the whole program quits with it! Obviously I am not handling the child process signal properly(Method code below).
My Question:
How does process management with pipe() works? if i run a command ls -l | sort does it create a child process for ls -l and another process for sort ? From the piping examples that I have seen so far, only one process is created(fork()).
When the second command (sort from our example) is processed, how can i get its process ID?
EDIT: Also while running this code I get the result twice. don't know why it runs twice, there is no loop in there.
Here is my code:
pid_t pipeIt(void){
pid_t pid;
int pipefd[2];
if(pipe(pipefd)){
unix_error("pipe");
return -1;
}
if((pid = fork()) <0){
unix_error("fork");
return -1;
}
if(pid == 0){
close(pipefd[0]);
dup2(pipefd[1],1);
close(pipefd[1]);
if(execl("/bin/ls", "ls", (char *)NULL) < 0){
unix_error("/bin/ls");
return -1;
}// End of if command wasn't successful
}// End of pid == 0
else{
close(pipefd[1]);
dup2(pipefd[0],0);
close(pipefd[0]);
if(execl("/usr/bin/tr", "tr", "e", "f", (char *)NULL) < 0){
unix_error("/usr/bin/tr");
return -1;
}
}
return pid;
}// End of pipeIt
Yes, the shell must fork to exec each subprocess. Remember that when you call one of the execve() family of functions, it replaces the current process image with the exec'ed one. Your shell cannot continue to process further commands if it directly execs a subprocess, because thereafter it no longer exists (except as the subprocess).
To fix it, simply fork() again in the pid == 0 branch, and exec the ls command in that child. Remember to wait() for both (all) child processes if you don't mean the pipeline to be executed asynchronously.
Yes, you do need to call fork at least twice, once for each program in the pipeline. Remember that exec replaces the program image of the current process, so your shell stops existing the moment you start running sort or (tr).
I'm trying to write a simple shell that can handle pipe commands. I want to be able to handle multiple pipes all chained together but I'm having a hard time figuring out how to implement something like this.
This is my current attempt:
int status;
int lastToken = 0;
int pipe_pid;
//create the pipes
int pipefd[pipes][2];
// Loop to run all commands in the vertical list.
while(1){
if (c->type == TOKEN_PIPE){
// Here is where we deal with pipes
for (int i = 0; i < pipes; i++){
pipe(pipefd[i]);
pipe_pid = fork();
//this is a receiving pipe
if (pipe_pid == 0){
// create the write end of the pipe
dup2(pipefd[i][WRITE_SIDE], STDOUT_FILENO);
close(pipefd[i][READ_SIDE]);
close(pipefd[i][WRITE_SIDE]);
execvp(c->argv[0], c->argv);
// printf("parent pipe\n");
}
//this is a writing pipe
else{
close(pipefd[i][WRITE_SIDE]);
dup2(pipefd[i][READ_SIDE], STDIN_FILENO);
close(pipefd[i][READ_SIDE]);
// printf("child pipe\n");
}
}
// This stuff happens for all commands
lastToken = c->type;
// If it's the last command, we're done
if (c->next == NULL){
break;
}
else{
c = c->next;
}
}
the commands are chained together in a linked list, c is my command pointer
pipes is a variable that I create as I parse the in-string, so I know how many '|' I saw in the command. This should tell me the number of child processes I need to fork.
I use pipes to create a 2d array for the pipe descriptors.
Then I want to loop over the pipes and fork once for each, and use dup2 to map the inputs and outputs.
I'm getting inconsistent errors that I can't figure out. First of all, every time I run a pipe command, my shell immediately crashes with no segfault or other printed errors.
Second, if I run commands like echo foo | wc -c I sometimes get 4 and sometimes get 0 as the output.
I'm sure I'm just doing something dumb but I'm not sure what :/
I figured out what I was doing wrong, I was closing the pipes before all the threads were finished using them. I fixed it by pulling out the close calls.
// writing side of the pipe
if (c->type == TOKEN_PIPE){
close(c->pipefd[READ_SIDE]);
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
}
// receiving side of the pipe
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[WRITE_SIDE]);
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
}
And then in the parent thread, right before I reep my zombies, I check for pipes that are finished being used and close them.
// writing side of the pipe
if (c->type == TOKEN_PIPE){
close(c->pipefd[READ_SIDE]);
dup2(c->pipefd[WRITE_SIDE], STDOUT_FILENO);
}
// receiving side of the pipe
if (commandPrev->type == TOKEN_PIPE){
close(commandPrev->pipefd[WRITE_SIDE]);
dup2(commandPrev->pipefd[READ_SIDE], STDIN_FILENO);
close(commandPrev->pipefd[READ_SIDE]);
I'm not sure if this is the optimal way to do it but it works without errors for me.
Using a fairly standard fork process:
int pipe_to_child[2];
int pipe_from_child[2];
int child_exit_status = -1;
pid_t child_pid = fork();
if (child_pid == 0) {
close(pipe_from_child[0]); // close their read end
close(pipe_to_child[1]); // Close their write end
dup2(pipe_to_child[0], STDIN_FILENO); // Tie the in pipe to stdin
dup2(pipe_from_child[1], STDOUT_FILENO); // Tie stdout to the out pipe
// Run the child process
execve(file_to_run, argv_for_prog, env_for_prog);
}
else {
close(pipe_from_child[1]); // close their write end
close(pipe_to_child[0]); // Close their read end
if (input_to_prog != NULL) write(pipe_to_child[1], input_to_prog, strlen(input_to_prog)); // Send the stdin stuff
close(pipe_to_child[1]); // Done so send EOF
// Wait for the child to end
waitpid(child_pid, &child_exit_status, 0);
// Do post end-of-child stuff
}
This generally works as expected.
However, when the child process, a shell script, sets a further process off in the background. Even though the child process then exits (and is no longer listed by ps), the waitpid doesn't return.
The script is this case is meant to start inadyn-mt (a DDNS updater) running in the background.
#!/bin/sh
inadyn-mt --background
(If I put an & after inadyn-mt it makes no difference)
It turns out that the issue is that the pipes don't get closed. Although the child process exits fine, because it has spawned a further process, this process (even though it doesn't want them) is tied to the pipes to the child's stdin and stdout. The solution I used was to not set up the pipes when I was going to spin off a child from the child.
I am working on a tiny shell(tsh) implemented in C(it's an assignment). One part of assignment belongs to PIPING. I have to pipe a command's output to another command. e.g:ls -l | sort
When I run the shell, every command that I execute on it, is processed by a child process that it spawns. After the child finishes the result is returned. For piping I wanted to implement a harcoded example first to check how it works. I wrote a method, that partially works. The problems is when I run the pipe command, after child process finishes, the whole program quits with it! Obviously I am not handling the child process signal properly(Method code below).
My Question:
How does process management with pipe() works? if i run a command ls -l | sort does it create a child process for ls -l and another process for sort ? From the piping examples that I have seen so far, only one process is created(fork()).
When the second command (sort from our example) is processed, how can i get its process ID?
EDIT: Also while running this code I get the result twice. don't know why it runs twice, there is no loop in there.
Here is my code:
pid_t pipeIt(void){
pid_t pid;
int pipefd[2];
if(pipe(pipefd)){
unix_error("pipe");
return -1;
}
if((pid = fork()) <0){
unix_error("fork");
return -1;
}
if(pid == 0){
close(pipefd[0]);
dup2(pipefd[1],1);
close(pipefd[1]);
if(execl("/bin/ls", "ls", (char *)NULL) < 0){
unix_error("/bin/ls");
return -1;
}// End of if command wasn't successful
}// End of pid == 0
else{
close(pipefd[1]);
dup2(pipefd[0],0);
close(pipefd[0]);
if(execl("/usr/bin/tr", "tr", "e", "f", (char *)NULL) < 0){
unix_error("/usr/bin/tr");
return -1;
}
}
return pid;
}// End of pipeIt
Yes, the shell must fork to exec each subprocess. Remember that when you call one of the execve() family of functions, it replaces the current process image with the exec'ed one. Your shell cannot continue to process further commands if it directly execs a subprocess, because thereafter it no longer exists (except as the subprocess).
To fix it, simply fork() again in the pid == 0 branch, and exec the ls command in that child. Remember to wait() for both (all) child processes if you don't mean the pipeline to be executed asynchronously.
Yes, you do need to call fork at least twice, once for each program in the pipeline. Remember that exec replaces the program image of the current process, so your shell stops existing the moment you start running sort or (tr).