I have a function which create two child process. In the first child process I am writing in a file and in the second one I am also writing in a file which is different from the first one.
In the dad process I am executing the function execvp.
What I need is the stdout and stderrof the function execvp, so that the two child process can write in files what goes out from stderr and stdout. Then at the end I am merging the two files.
I would like to know where I should close pipes and where I should use wait so that I don't have problem using function read and write and such that I don't go in a infinite loop. I didn't implement the functions that create files, the function that merge files and the function that run the shell instruction because I am just wondering if this is the best structure for the function createTwoChild.
As aligned on the comments I'm providing a skeleton here (simplified just for stdout):
if (pipe(fd)<0) goto my_sys_error; // just an example to get out of here
if ((pid_child=fork())<0) {
close(fd[0]); close(fd[1]);
goto my_sys_error; // you can also use something like e.g. "return -1" to handle the error
}
if (!pid_child) {
// the child process with exec() of which we want to get the output
close(fd[0]); close(0);
dup2(fd[1],1); close(fd[1]);
execXX(...); // some of exec() family also spawn a shell here
close(1);
_exit(127); // This must not happen
}
// master/parent
close(fd[1]); // master doesn't need, only child writes to it
i = read(fd[0],p,PIPEBUF_SIZE);
if (i>0) {
// usual handling, write to file, do whatever you like
// should be while() instead of if(), just simplified
} else {
// handle it, e.g. print "no data from extcmd"
}
close(fd[0]); // close the last fd
waitpid(pid_child, &status_child, 0);
if (!WIFEXITED(status_child)) {
kill(pid_child,SIGKILL);
}
Some notes:
This is actually a simplified implementation of popen().
When the child exited then you will get an EOF on read - in this simple way no sighandler for SIGCHLD is required.
Other signal handling not covered.
stderr can be simply added with an additional pipe.
Related
I've been working on this school assignment forever now, and I'm super close to finishing.
The assignment is to create a bash shell in C, which sounds basic enough, but it has to support piping, IO redirect, and flags within the piped commands. I have it all working except for one thing; the | piping child isn't getting any of the data written to the pipe by the user command process child. If I were to remove the child fork for pipechild, and have everything from if(pipe_cmd[0] != '\0') run as the parent, it would work just fine (minus ending the program because of execlp). If I were to use printf() inside the pipe section, the output would be in the right file or terminal, which just leaves the input from the user command process child not getting to where it needs to be as a culprit.
Does anyone see an issue on how I'm using the pipe? It all felt 100% normal to me, given the definition of a pipe.
int a[2];
pipe(a);
//assume file_name is something like file.txt
strcat(file_name, "file.txt");
strcat(pipe_cmd, "wc");
if(!fork())
{
if(pipe_cmd[0] != '\0') // if there's a pipe
{
close(1); //close normal stdout
dup(a[1]); // making stdout same as a[1]
close(a[0]); // closing other end of pipe
execlp("ls","ls",NULL);
}
else if(file_name[0] != '\0') // if just a bare command with a file redirect
{
int rootcmd_file = open(file_name, O_APPEND|O_WRONLY|O_CREAT, 0644);
dup2(rootcmd_file, STDOUT_FILENO);
execlp("ls","ls",NULL); // writes ls to the filename
}
// if no pipe or file name write...
else if(rootcmd_flags[0] != '\0') execlp("ls","ls",NULL)
else execlp("ls","ls",NULL);
} else wait(0);
if(pipe_cmd[0] != '\0') // parent goes here, if pipe.
{
pipechild = fork();
if(pipechild != 0) // *PROBLEM ARISES HERE- IF THIS IS FORKED, IT WILL HAVE NO INFO TAKEN IN.
{
close(0); // closing normal stdin
dup(a[0]); // making our input come from the child above
close(a[1]); // close other end of pipe
if(file_name[0] != '\0') // if a filename does exist, we must reroute the output to the pipe
{
close(1); // close normal stdout
int fileredir_pipe = open(file_name, O_APPEND|O_WRONLY|O_CREAT, 0644);
dup2(fileredir_pipe, STDOUT_FILENO); //redirects STDOUT to file
execlp("wc","wc",NULL); // this outputs nothing
}
else
{
// else there is no file.
// executing the pipe in stdout using execlp.
execlp("wc","wc",NULL); // this outputs nothing
}
}
else wait(0);
}
Thanks in advance. I apologize for some of the code being withheld. This is still an active assignment and I don't want any cases of academic dishonesty. This post was risky enough.
} else wait(0);
The shown code forks the first child process and then waits for it to terminate, at this point.
The first child process gets set up with a pipe on its standard output. The pipe will be connected to the second child process's standard input. The fatal flaw in this scheme is that the second child process isn't even started yet, and won't get started until the first process terminates.
Pipes have limited internal buffering. If the first process generates very little output chances are that its output will fit inside the tiny pipe buffer, it'll write its output and then quietly terminate, none the wiser.
But if the pipe buffer becomes full, the process will block and wait until something reads from the pipe and clears it. It will wait as long as it takes for that to happen. And wait, and wait, and wait. And since the second child process hasn't been started yet, and the parent process is waiting for the first process to terminate it will wait, in vain, forever.
This overall logic is fatally flawed for this reason. The correct logic is to completely fork and execute all child processes, close the pipe descriptors in the parent (this is also important), and then wait for all child processes to terminate. wait must be the very last thing that happens here, otherwise things will break in various amazing and mysterious ways.
I'm trying to do a little C program that realize a pipeline of two bash commands : echo $arithmeticOperation | bc
$arithmeticOperation is a string taken as input.
The program works fine executing first command, but when i run the second one, i get the right output but the child process executing bc remains stuck preventing the child from ending.
So in this line father process is blocked :
waitpid(pid2,NULL,0);
Where do you think the problem may be ?
Sorry if i asked the question incorrectly, it's my first one. Thanks.
#define SYSCALL(r,c,e) if((r=c)==-1) { perror(e);exit(EXIT_FAILURE);}
int main(){
char buf[128];
int pfd[2],err;
pid_t pid1,pid2;
SYSCALL(err,pipe(pfd),"pipe");
switch (pid1=fork()) {
case -1: { perror("fork"); exit(EXIT_FAILURE);}
case 0 : {
scanf("%s",buf);
SYSCALL(err,dup2(pfd[1],1),"dup");
close(pfd[1]);
close(pfd[0]);
execl("/bin/echo","echo",buf,(char *)NULL);
return 1;
}
}
switch (pid2=fork() ){
case -1 : { perror("fork"); exit(EXIT_FAILURE);}
case 0 : {
SYSCALL(err,dup2(pfd[0],0),"dup");
close(pfd[1]);
close(pfd[0]);
// execl("/usr/bin/bc","bc",(char *)NULL);
execlp("bc","bc",(char *)NULL);
return 1;
}
}
printf("waiting . . . \n");
waitpid(pid1,NULL,0);
printf("wait\n");
waitpid(pid2,NULL,0);
close(pfd[1]);
close(pfd[0]);
return 0;
}
So if i digit "1+1" as a input string i get the right output but then the process executing bc never exit
As I noted in a comment, your parent process must close the file descriptors for the pipe before waiting for bc (and you've agreed that this fixes the problem).
This arises because bc has the pipe open for reading, and the parent has the pipe open for writing, and the kernel thinks that the parent could therefore send data to bc. It won't, but it could.
You have to be very careful when managing pipes. You carefully avoided the usual problem of not closing enough file descriptors in the children.
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD
I need to extend that to cover parent processes too.
If the parent process is not going to communicate with any of its children via the pipe, it must ensure that it closes both ends of the pipe so that its children can receive EOF indications on read (or get SIGPIPE signals or write errors on write), rather than blocking indefinitely. The parent should normally close at least one end of the pipe — it would be extremely unusual for a program to read and write on both ends of a single pipe.
I'm trying to implement a Linux pipe chain in C. For example:
grep file | ls | wc
So, there is a code that splits the arguments into tokens with the pipe as the separator, and sends each part to the following function with an integer specifying whether it precedes a pipe or not:
int control_flow(char** args, int precedes){
int stdin_copy = dup(0);
int stdout_copy = dup(1);
// if the command and its args precedes a pipe
if (precedes){
int fd[2];
if (pipe(fd) == -1){
fprintf(stderr, "pipe failed\n");
}
if (dup2(fd[1], 1)!=1)
perror("dup2 error 1 to p_in\n"); // 1 points to pipe's input
status = turtle_execute(args); // executes the argument list, output should go into the pipe
// Code stops running here
if (dup2(fd[0], 0)!=0)
perror("dup2 error 0 to p_out\n"); // 0 points to pipe's output, any process that reads next will read from the pipe
if (dup2(stdout_copy, 1)!=1)
perror("dup2 error 1 to stdout_copy\n"); // 1 points back to stdout
}
// if the command does not precede a pipe
else{
status = turtle_execute(args); // input to this is coming from pipe
if (dup2(stdin_copy, 0)!=0) // 0 points back to stdin
perror("dup2 error 1 to stdin_copy");
}
return 0;
}
My code stops running after the first command executes. I suspect it is necessary to fork a process before using this pipe, why is that? If so, how do I do that in my code without changing what I intend to do?
Edit:
This is roughly what turtle_execute does:
turtle_execute(args){
if (args[0] is cd or ls or pwd or echo)
// Implement by calling necessary syscalls
else
// Do fork and exec the process
So wherever I have used exec, I have first used fork, so process getting replaced shouldn't be a problem.
The exec system call replaces the current process with the program you are executing. So your process naturally stops working after the turtle_execute, since it was replaced with the new process.
To execute a new process you normally fork to create a copy of the current process and then execute in the copy.
When you are in the shell, normally each command you type is forked and executed. Try typing exec followed by a command into a shell and you will find that the shell terminates once that command has finished executing, since it does not fork in that case.
Edit
I suggest you have a look at the example on the pipe(2) man page (http://man7.org/linux/man-pages/man2/pipe.2.html#EXAMPLE). It shows the usual way of using a pipe:
Calling pipe to get the create the pipe
Calling fork to fork the process
Depending on whether it is child or parent close one end of the pipe and use the other
I think your problem might be that you make the writing end of your pipe the stdout before forking, causing both the parent and the child to have an open writing end. That could prevent an EOF to be sent since one writing end is still open.
I can only guess what happens in most of turtle_execute, but if you fork, exec on one process, and wait for it on the other, without consuming data from the pipe, it might fill the pipe and to the point where writing is blocked. You should always consume data from the pipe while you write to it. It is a pipe after all and not a water tank. For more information have a look at the pipe(7) man page under the 'Pipe capacity' section.
My mini-shell program accepts pipe command, for example, ls -l | wc -l and uses excevp to execute these commands.
My problem is if there is no fork() for execvp, the pipe command works well but the shell terminates afterward. If there is a fork() for execvp, dead loop happens. And I cannot fix it.
code:
void run_pipe(char **args){
int ps[2];
pipe(ps);
pid_t pid = fork();
pid_t child_pid;
int child_status;
if(pid == 0){ // child process
close(1);
close(ps[0]);
dup2(ps[1], 1);
//e.g. cmd[0] = "ls", cmd[1] = "-l"
char ** cmd = split(args[index], " \t");
//if fork here, program cannot continue with infinite loop somewhere
if(fork()==0){
if (execvp(cmd[0],cmd)==-1){
printf("%s: Command not found.\n", args[0]);
}
}
wait(0);
}
else{ // parent process
close(0);
close(ps[1]);
dup2(ps[0],0);
//e.g. cmd[0] = "wc", cmd[1] = "-l"
char ** cmd = split(args[index+1], " \t");
//if fork here, program cannot continue with infinite loop somewhere
if(fork()==0){
if (execvp(cmd[0],cmd)==-1){
printf("%s: Command not found.\n", args[0]);
}
}
wait(0);
waitpid(pid, &child_status, 0);
}
}
I know fork() is needed for excevp in order to not terminate the shell program, but I still cannot fix it. Any help will be appreciated, thank you!
How should I make two children parallel?
pid = fork();
if( pid == 0){
// child
} else{ // parent
pid1 = fork();
if(pid1 == 0){
// second child
} else // parent
}
is this correct?
Yes, execvp() replaces the program in which it is called with a different one. If you want to spawn another program without ending execution of the one that does the spawning (i.e. a shell) then that program must fork() to create a new process, and have the new process perform the execvp().
Your program source exhibits a false parallelism that probably either confuses you or reflects a deeper confusion. You structure the behavior of the first child forked in just the same way as the behavior of the parent process after the fork, but what should be parallel is the behavior of the first child and the behavior of the second child.
One outcome is that your program has too many forks. The initial process should fork exactly twice -- once for each child it wants to spawn -- and neither child should fork because it's already a process dedicated to one of the commands you want to run. In your actual program, however, the first child does fork. That case is probably rescued by the child also wait()ing for the grandchild, but it's messy and poor form.
Another outcome is that when you set up the second child's file descriptors, you manipulate the parent's, prior to forking, instead of manipulating the child's after forking. Those changes will persist in the parent process, which I'm pretty confident is not what you want. This is probably why the shell seems to hang: when run_pipe() returns (the shell's standard input has been changed to the read end of the pipe).
Additionally, the parent process should close both ends of the pipe after the children have both been forked, for more or less the same reason that the children must each close the end they are not using. In the end, there will be exactly one open copy of the file descriptor for each end of the pipe, one in one child and the other in the other. Failing to do this correctly can also cause a hang under some circumstances, as the processes you fork may not terminate.
Here's a summary of what you want the program to do:
The original process sets up the pipe.
The original process forks twice, once for each command.
Each subprocess manipulates its own file descriptors to use the correct end of the pipe as the appropriate standard FD, and closes the other end of the pipe.
Each subprocess uses execvp() (or one of the other functions in that family) to run the requested program
the parent closes its copies of the file descriptors for both ends of the pipe
the parent uses wait() or waitpid() to collect two children.
Note, too, that you should check the return values of all your function calls and provide appropriate handling for errors.
I've been messing around in C trying to figure out how to do this. Let's say I have my main program, the parent process. The parent creates three child processes, each of which will eventually run programs (but that's not important right now). What I'd like to do is make it so that the first child's stdout will be received by the second child's stdin. The second child's stdout will then be received by the third child's stdin.
The parent process's stdin/stdout aren't messed with at all.
So far, what I've got is
pipe(procpipe);
parentPid = getpid();
for(i = 0; i < 3; i++)
{
if(getpid() == parentPid)
{
child[i] = fork();
if(child[i] == 0)
{
mynumber = i+1;
}
}
}
But from there I'm kind of stuck as to how to use dup2 to properly assign my pipes, and in which section of my code to insert it. There are lots of examples on Google and this website of how to pipe from a parent to a child, but I'm yet to see one that will tell me exactly how to connect a child's stdout to another child's stdin.
Edit:
Forgot to mention: assume all my variables are properly initialised. The int 'mynumber' is so a child process knows upon creation which number it is, so I can give it instructions via
if(mynumber == whatever)
So you have a loop that creates several child processes. Each of these child processes will be using two pipes: read from previous and write to the next. To set up a pipe for the reading end you need to close the write end of the pipe, and dup2 the read end into the stdin. Similar for the pipe where the process will be writing.
void set_read(int* lpipe)
{
dup2(lpipe[0], STDIN_FILENO);
close(lpipe[0]); // we have a copy already, so close it
close(lpipe[1]); // not using this end
}
void set_write(int* rpipe)
{
dup2(rpipe[1], STDOUT_FILENO);
close(rpipe[0]); // not using this end
close(rpipe[1]); // we have a copy already, so close it
}
When you fork each children you need to attach the pipes to it.
void fork_and_chain(int* lpipe, int* rpipe)
{
if(!fork())
{
if(lpipe) // there's a pipe from the previous process
set_read(lpipe);
// else you may want to redirect input from somewhere else for the start
if(rpipe) // there's a pipe to the next process
set_write(rpipe);
// else you may want to redirect out to somewhere else for the end
// blah do your stuff
// and make sure the child process terminates in here
// so it won't continue running the chaining code
}
}
With this in hand you can now write a loop that continuously forks, attaches the pipes, and then reuses the output pipe as the input pipe for the next one. Of course, once both ends of a pipe have been connected to child processes, the parent should not leave it open for itself.
// This assumes there are at least two processes to be chained :)
// two pipes: one from the previous in the chain, one to the next in the chain
int lpipe[2], rpipe[2];
// create the first output pipe
pipe(rpipe);
// first child takes input from somewhere else
fork_and_chain(NULL, rpipe);
// output pipe becomes input for the next process.
lpipe[0] = rpipe[0];
lpipe[1] = rpipe[1];
// chain all but the first and last children
for(i = 1; i < N - 1; i++)
{
pipe(rpipe); // make the next output pipe
fork_and_chain(lpipe, rpipe);
close(lpipe[0]); // both ends are attached, close them on parent
close(lpipe[1]);
lpipe[0] = rpipe[0]; // output pipe becomes input pipe
lpipe[1] = rpipe[1];
}
// fork the last one, its output goes somewhere else
fork_and_chain(lpipe, NULL);
close(lpipe[0]);
close(lpipe[1]);
The closing bits are very important! When you fork with an open pipe, there will be four open file descriptors: two on the parent process, and two others on the child process. You have to close all of those you won't be using. That's why the code above always closes the irrelevant ends of the pipes in the child processes, and both ends on the parent.
Also note that I am giving special treatment to the first and the last processes, because I don't know where the input for the chain will come from, and where the output will go to.