I have found quite a few questions similar to this one (e.g. https://www.quora.com/How-can-I-write-a-code-for-PIPE-in-C-shell-script-python), which work in the cases described by those questions, but not mine for some reason. The answers I've found so far deal with either piping output from one child to another (the aforementioned parenthesized link) or from child to parent/parent to child. What I want to do is have the parent be able to write to the child's STDIN and read from its STDOUT. Or, in a more general case, given some number of children, have the parent write to the first child's STDIN, which will pipe its output to the second child, which will pipe its output to the third, and so on, until the last child pipes its output back to the parent. That is, I would like to be able to do something like prog0 | prog1 | prog 2 | ... from C/C++ code. My current half-attempt (half because I'm only trying to read from an echo's STDOUT before attempting a full input/output attempt using something simple like cat -) looks something like this (some error checking omitted for brevity):
void test() {
int pipe_fds[2];
pid_t cpid, wpid;
int status;
char buf[16] = {0};
status = pipe(pipe_fds);
cpid = fork();
if (cpid == 0) {
dup2(pipe_fds[1], STDOUT_FILENO);
close(pipe_fds[0]);
execlp("echo", "Hello world!", NULL);
printf("Child returned abnormally: %d\n", errno);
_exit(3);
}
close(pipe_fds[1]);
wpid = waitpid(cpid, &status, 0);
printf("Waited on pid %d, which exited with %d\n", wpid, WEXITSTATUS(status));
read(pipe_fds[0], buf, 16);
printf("Child said: %s\n", buf);
close(pipe_fds[0]);
exit(0);
}
This runs, but the parent gets no visible output from the child. How would I fix this, and, more importantly, how would I go about piping to [and from] a chain of processes as I had mentioned earlier?
In this case, the problem is your invocation of execlp. Try
execlp("echo", "echo", "Hello world!", NULL);
Related
I am trying to make a program containing 2 pipes, and in my program, the child will run first, the parent will run at the end.
The result shows that Child 2, then Child1, and keep pending.
It seems my parent is still waiting for some child process to be finished, but I only got 2 child process in this program~ Please help me :) Thanks!
#include <stdio.h>
#include <unistd.h>
#include <assert.h>
#include <fcntl.h>
#include <sys/types.h>
int main(void)
{
int pipefd[2];
int pipefd2[2];
int rv= pipe(pipefd);
assert(rv > -1);
int cid = fork();
assert(cid > -1);
int status;
if( cid > 0 ){
//waitpid(cid,NULL,0);
printf("P %d %d\n",getpid(),getppid());
wait(NULL);
printf("Parent \n");
close(0);
dup(pipefd[0]);
close(pipefd[0]);
close(pipefd[1]);
char *const wc_argv[] = {"wc", "-l", NULL};
execvp("wc", wc_argv);
//Parent - Redirect stdout to the write end of the pipe, and execute "ls -l"
}else{
int rv1= pipe(pipefd2);
assert(rv1 > -1);
int cid1 = fork();
assert(cid1 > -1);
if(cid1>0){
printf("C1 %d %d\n",getpid(),getppid());
wait(NULL);
printf("Child1\n");
//Child 1 (parent of child 2)
close(0);
dup(pipefd[0]);
close(1);
dup(pipefd2[1]);
close(pipefd[0]);
close(pipefd[1]);
close(pipefd2[0]);
close(pipefd2[1]);
char *const grep_argv[] = {"grep", "D", NULL};
execvp("grep", grep_argv);
}else{
printf("C2 %d %d\n",getpid(),getppid());
printf("Child2\n");
//Child 2 (child of child 1)
close(1);
dup(pipefd2[1]);
close(pipefd2[0]);
close(pipefd2[1]);
close(pipefd[0]);
close(pipefd[1]);
char *const ls_argv[] = {"ls", "-l", NULL};
execvp("ls", ls_argv);
}
}
}
There are multiple issues with your code. I pointed out some minor matters in comments, but the ones mainly likely to be responsible for the misbehavior you describe are:
Child 1 and the parent both redirect pipefd[0] to their standard inputs. Probably you want child 1 to redirect pipefd2[0] to its standard input instead, but you definitely don't want the two to have the same standard input.
Child 1 redirects its standard output to pipefd2[1], the other end of which pipe will be its standard input once you correct the previous issue. You appear to instead want to redirect to pipefd[1], which presently is not served at all.
Child 1 waits for child 2 before it proceeds. This is non-idiomatic and risky, for you will get a deadlock if child 2 fills the buffer of the second pipe, and therefore blocks before terminating. Pipes are data conduits. Although they do have internal buffers, this should be regarded as an implementation detail. It is incorrect to rely on pipes for buffering. The correct model is that data is consumed from the pipe's read end concurrently with data being written to the pipe's write end.
The parent waits for child 1 before it proceeds. As with child 1's wait, this is risky and non-idiomatic.
Additionally, as #IanAbott remarked in comments, with the way you are arranging the pipes, child 1 waiting for child 2 will reliably produce deadlock. The latter execs a program that will read its standard input to the end, but it will not see EOF on its input until the other ends of the pipe is closed, and that is never closed because child 1 waits on child 2 to finish before it proceeds. I see no necessity for the waits -- neither child 1's nor the parent's -- they could and should just be removed.
OMG!! Thanks ALL
This is my first time posting a question in stack overflow.
I cant believe you guys are so helpful, thank you so much
and i have solved my problem right now.
I can believe I just made a really simple mistake, which is in Child1
close(0);
dup(pipefd2[0]);
close(1);
dup(pipefd[1]);
Original, my program will run parent first, then i want to modify it, then i just thought exchanging the child 2 and parent, everything will be fine, but i forgot to modify the content of child1.
Anyways, you guys are so helpful, and hope you guys stay safe and keep going on our it adventure :)
BTW, I can see there are a lot of recommendations for my code, I will try to digest it :). Thanks all again!!
I have an app that spawns a child process. That child process outputs information about what it's doing by printing to stdout. The parent process does the same (i.e. prints to stdout).
In the child process I can write to stdout with some text prepended, but I have to add that to every single location I print across many source files.
I thought it might be smarter to have the parent process prepend output from the child process that it forks/exec's. I don't want to redirect the output because seeing the output inline with the parent process is beneficial. How do I do this? I'm using fork/exec in the parent.
Do I have to read the output and prepend each line manually or is there a simpler approach?
Update:
Thanks to Barmar. Here is how I'm doing it. I also could read byte by byte in the parent process from the pipe until line end. But I chose not to use that approach for reasons of complexity in my single threaded lua+C app.
// Crude example of output filtering using sed to
// prepend child process output text
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <libgen.h>
#include <string.h>
pid_t spawn(int fd[2], const char* path)
{
printf("Create child\n");
pid_t pid = fork();
switch(pid){
case -1:
printf("Create process failed");
return -1;
case 0:
dup2(fd[1], STDOUT_FILENO);
close(fd[0]);
close(fd[1]);
execl(path, path, NULL);
return 0;
default:
return pid;
}
}
pid_t spawnOutputFilter(int fd[2])
{
printf("Create sed\n");
pid_t pid = fork();
switch(pid){
case -1:
printf("Create sed failed");
return -1;
case 0:
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
close(fd[1]);
execlp("sed", "sed", "s/^/Engine: /", (char *)NULL);
return -1;
default:
return pid;
}
}
int main(int argc, char* argv[])
{
if (argc > 1){
int options;
int fd[2];
pipe(fd);
pid_t pid = spawn(fd, argv[1]);
pid_t sed_pid = spawnOutputFilter(fd);
close(fd[0]);
close(fd[1]);
waitpid(pid, NULL, 0);
waitpid(sed_pid, NULL, 0);
}
return 0;
}
You could create a second child process that performs
execlp("sed", "sed", "s/^/PREFIX: /", (char *)NULL);
Connect the first child's stdout to this process's stdin with a pipe.
I thought it might be smarter to have the parent process prepend output from the child process.
I guess it depends on how you judge "smart". It might be simpler to just make the child prepend the desired text to its outputs.
I don't want to redirect the output because seeing the output inline with the parent process is beneficial. What's the best way to do this?
When two processes share an open file, both access it independently, regardless of the nature of the relationship between those processes. Thus, if your child inherits the parent's stdout, the parent has no mechanism even to notice that the child is sending output, much less to modify that output.
If you want the parent to handle this, you would need to pass the child's output through the parent. You could do that by creating a pipe, and associating the child's stdout with the write end of that pipe. The parent would then need to monitor the read end, and forward suitably-modified outputs to its own stdout. The parent would probably want to create a separate thread for that purpose.
Additionally, if the child sometimes produces multi-line outputs that you want prefixed as a group, rather than per-line, then you'd probably need to build and use some kind of protocol for demarcating message boundaries, which would make the whole parent-moderation idea pretty pointless.
Couldn't you define #define printf(a) printf("your text:" a).
Other alternative I can think of is using dup
You open the same log file in your child process and dup your stdout to new file descriptor.
I am working on an assignment for my Operating System class (Posix & C), building a mini-shell, and I don't know how to solve the following problem:
My mini-shell has to accept two commands, for example ls | grep a. For that I create a pipe of size two and a child. The child closes all that it has to close and opens all that it has to open (standard/pipe's in & out). It then executes "ls," using execvp. I am sure this works well. After that, the parent shuts and opens inputs and outputs (I am sure I do it well), and then executes grep a.
Now, the problem is that the process grep a never finishes. Same for tail -1, e.g.. Yet it does work for head -1. I think that happens because grep and tail, which are executed by the parent, wait for more input, even though the child has finished its operation. ls | grep a produces the right output, displayed on the console (The pipe's output is set as default output), but, as I've said, grep a does not finish.
So, my question is: how can I inform the parent that the pipe has finished writing, so it can finish the execution of grep a for example?
Thank you.
Here's the code:
[fd is the pipe, it is initialized previously in the code. If you can see any incongruous thing, please let me know; I've cleaned the code a bit, and this is only the problematic part, as you can see.]
int fd[2];
pipe(fd);
if ((pid = fork()) != -1){
if(pid == 0){ /*Child executing */
close(fd[0]);
close(1);
dup(fd[1]);
close(fd[1]);
execvp(argvv[0][0], argvv[0]); /* Here's stored the first instruction */
} else{ /* Parent executing */
wait(&status);
close(fd[1]);
close(0);
dup(fd[0]);
close(fd[0]);
execvp(argvv[1][0], argvv[1]); /* Here's stored the second instruction */
}
}
If the grep continues to run after the ls has exited, that indicates that you have not closed all the pipes that you need to close.
In particular, the write end of the pipe whose read end is attached to the grep process is still open in another process. You will need to show your code to know more.
The code you have pasted works correctly (when expanded to a full program, as per the below). Both subprocesses exit just fine.
This means that you've eliminated the code that has the problem when you created your cut-down version here - perhaps you have another fork() between the pipe() call and this fork()?
#include <unistd.h>
#include <sys/wait.h>
int main()
{
pid_t pid;
char *argvv[2][3] = { { "ls", 0, 0}, { "grep", "a", 0 } };
int status;
int fd[2];
pipe(fd);
if ((pid = fork()) != -1) {
if(pid == 0){ /*Child executing */
close(fd[0]);
close(1);
dup(fd[1]);
close(fd[1]);
execvp(argvv[0][0], argvv[0]); /* Here's stored the first instruction */
} else{ /* Parent executing */
wait(&status);
close(fd[1]);
close(0);
dup(fd[0]);
close(fd[0]);
execvp(argvv[1][0], argvv[1]); /* Here's stored the second instruction */
}
}
return 0;
}
I have the main process forking two times and thus creating two children. The two children are piped with each other like this:
ls | more
Now the problem is that the second child never dies. Why is that? When does the last child in a pipe die really?
Removing one wait() call shows the expected result of ls | more but gives some further weird behaviours(stuck terminal etc).
Here is my code:
int main(){
printf("[%d] main\n", getpid());
int pip[2], i;
pipe(pip);
/* CHILDREN*/
for (i=0; i<2; i++){
if (fork()==0){
/* First child */
if (i==0){
printf("[%d] child1\n", getpid());
close(1); dup(pip[1]);
close(pip[0]);
execlp("ls", "ls", NULL);}
/* Second child */
if (i==1){
printf("[%d] child2\n", getpid());
close(0); dup(pip[0]);
close(pip[1]);
execlp("more", "more", NULL);}
}
}
wait(NULL); // wait for first child
wait(NULL); // wait for second child
return 0;
}
The read end of the pipe won't get an EOF mark until the write end has been closed by all its users. The parent of both children still has both ends of the pipe open, so more doesn't see an EOF (a return of 0 from read()), and keeps waiting for more input.
Check with ps axw that it's really the more that isn't dying. Then read the man page for wait. Look at the returned status for wait.
(Hint: look at what causes a 'zombie' process. I don't think the waits are doing what you think they are.)
Hi I'm working on a unix shell and I'm running into two problems. I was wondering if any of you could help me out. My first problem is that the shell is not waiting for the child process to terminate. I can actually go type more commands while the child process is running. My second problems is in the following two lines. I'm not getting any display on the shell.
fprintf(stderr, "Process name is: %s\n", commandArgv[0]);
fprintf(stderr, "Child pid = %d\n", pid);
I have the following method to execute a process entered by the user: i.e. firefox, ls -a, etc
void execute(char *command[], char *file, int descriptor){
pid_t pid;
pid = fork();
if(pid == -1){
printf("error in execute has occurred\n");
}
if(pid == 0){
execvp(*command,command);
fprintf(stderr, "Process name is: %s\n", commandArgv[0]);
fprintf(stderr, "Child pid = %d\n", pid);
wait(&status);
exit(EXIT_SUCCESS);
}
else{
printf("ignore for now\n");
}
}
This is where I call the execute command. It works fine and launches a process, but it doesn't wait for it to finish.
execute(commandArgv, "STANDARD",0);
Do you guys have any idea what I might be doing wrong? Thanks I really appreciate any time you take to help me on this.
Once execvp() runs, it will never return. It replaces in-memory the running app with whatever was provided. So your fprintf() and wait() are in the wrong place.
Other than getting the actual logic worked out correctly (Stéphane's suggestions all good) you might also want to fflush(stderr) after fprintf-ing, to ensure your error messages make it out right away instead of being buffered.
You have a little error in how the process works. After execvp is called, there is no turning back. fork() gives you have the parent and an identical child, but execvp overwrite child image to be the command you are calling.
The execvp returns only when a severe errors occur that prevent overwriting the image. So, you need to print things before its call. So you also may want to change the EXIT_SUCCESS to an EXIT_FAILURE there.
Now there is another mistake using wait: you always want the parent waiting for the child, not the other way around. You cannot ask for the child to wait. She has nothing to wait, she will run and terminate. So, you need to move the wait() call to the else part.
void execute(char *command[], char *file, int descriptor)
{
pid_t pid;
pid = fork();
if(pid == -1)
{
printf("fork() error in execute() has occurred\n");
return; /* return here without running the next else statement*/
}
if(pid == 0)
{
fprintf(stderr, "Process name is: %s\n", commandArgv[0]);
fprintf(stderr, "Child pid = %d\n", getpid());
execvp(*command,command);
fprintf(stderr, "Error! Can't overwrite child's image!\n");
exit(EXIT_FAILURE);
}
else
{
printf("Parent waiting for child pid: %d\n", pid);
wait(&status);
printf("Parent running again\n");
}
}
But reading your question, maybe you actually don't want the parent to wait. If that is the case, just don't use the wait() function.
Take care,
Beco
Edited: some minor mistakes. pid of child is getpid()