Let's say within a program I want to execute two processes, one to execute a ls -al command, then piping the result into the wc command, and displaying the output on the terminal. How can I do this using pipe file descriptors? So far the code I have written:
int main(int argc, char* argv[]) {
int pipefd[2];
int pipefd2[2];
pipe(pipefd2);
if ((fork()) == 0) {
dup2(pipefd2[1], STDOUT_FILENO);
close(pipefd2[0]);
close(pipefd2[1]);
execl("ls", "ls", "-al", NULL);
exit(EXIT_FAILURE);
}
if ((fork()) == 0){
dup2(pipefd2[0], STDIN_FILENO);
close(pipefd2[0]);
close(pipefd2[1]);
execl("/usr/bin/wc", "wc", NULL);
exit(EXIT_FAILURE);
}
close(pipefd[0]);
close(pipefd[1]);
close(pipefd2[0]);
close(pipefd2[1]);
}
An example would be greatly helpful.
Your example code was syntactically and semantically broken (e.g. pipefd2 not decared, confusion between pipefd and pipefd2, etc.) Since this smells like homework, please make sure you understand my annotations below and ask more if you need to. I've omitted error checks on pipe, fork and dup, but they should be there, ideally.
int main(int argc, char *argv[]) {
int pipefd[2];
pid_t ls_pid, wc_pid;
pipe(pipefd);
// this child is generating output to the pipe
//
if ((ls_pid = fork()) == 0) {
// attach stdout to the left side of pipe
// and inherit stdin and stdout from parent
dup2(pipefd[1],STDOUT_FILENO);
close(pipefd[0]); // not using the right side
execl("/bin/ls", "ls","-al", NULL);
perror("exec ls failed");
exit(EXIT_FAILURE);
}
// this child is consuming input from the pipe
//
if ((wc_pid = fork()) == 0) {
// attach stdin to the right side of pipe
// and inherit stdout and stderr from parent
dup2(pipefd[0], STDIN_FILENO);
close(pipefd[1]); // not using the left side
execl("/usr/bin/wc", "wc", NULL);
perror("exec wc failed");
exit(EXIT_FAILURE);
}
// explicitly not waiting for ls_pid here
// wc_pid isn't even my child, it belongs to ls_pid
return EXIT_SUCCESS;
}
Related
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main(int argc, char *argv[])
{
int fd[2];
pid_t pid;
if (pipe(fd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
pid = fork();
if (pid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (pid == 0) {
dup2(fd[1], STDOUT_FILENO);
close(fd[0]);
close(fd[1]);
execlp("ls", "ls", NULL);
perror("execlp");
exit(EXIT_FAILURE);
} else {
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
close(fd[1]);
execlp("more", "more", "-3", NULL);
perror("execlp");
exit(EXIT_FAILURE);
}
}
I dont understand, why does my program when we say if (pid == 0) works like intended where more program display 3 lines and that again 3 lines after the space and so on, and when i say if (pid!=0) ls display everything and more command doesnt work? Why is this?
Why does it matter if parent process does ls and child more command or parent does more and child ls? Either way more wont print anything until ls writes data into the pipe.
Here is the code i tried versus the one above, i tried having parent process calling "ls" command and child process call "more" command. I don't see why this would be an issue? I just changed "responsobilities", i don't see how this changes anything? child process with "more" command comes first, it will attempt to read pipe and it will block since it's empty, parent process with "ls" command will eventually write into the pipe and the child process with "more" will get unblocked, read from the pipe and print it out on the user screen. So i don't understand why is it only displaying the contents of the directory, instead of showing me three piece's of content and then after i click space another three, and so on..
if (pid != 0) {
dup2(fd[1], STDOUT_FILENO);
close(fd[0]);
close(fd[1]);
execlp("ls", "ls", NULL);
perror("execlp");
exit(EXIT_FAILURE);
}
else {
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
close(fd[1]);
execlp("more", "more", "-3", NULL);
perror("execlp");
exit(EXIT_FAILURE);
}
I am working to make a shell like bash, but i have trouble solving heredoc << so i made a test code as simple as possible for this question.
void pipeline()
{
int i = 0;
int fd[2];
pid_t pid;
int fdd = 0;
while (i < 2)
{
pipe(fd);
pid = fork();
if (pid == 0)
{
//dup2(fd[1],1); if i dup in the first pipe cat dont finalize
if (i == 0)
dup2(fd[0],0);
write(fd[1], "hello\nhow\nare\nyou\n", 17);
close(fd[0]);
close(fd[1]);
dup2(fdd, 0);
if (i == 0)
execlp("cat", "cat", NULL);
else
execlp("grep", "grep", "you" , NULL);
perror("error");
exit(1);
}
else
{
close(fd[1]);
fdd = fd[0];
wait(NULL);
i++;
}
}
}
int main(int *argc, char **argv, char **env)
{
pipeline();
}
I know that cat and grep need an EOF to run; what I'm doing is writing in stdin and running cat, but my question is: how do I save stdout for grep without duping stdout on the first pipe?
If I dup on dup2(fd[1],1) cat does not work in the first pipe, could someone help me out to make this code work? And make it as similar to bash heredoc as well if possible.
how do I save stdout for grep without duping stdout on the first pipe?
I'd rearrange the creation of the child processes from rightmost to leftmost - then grep is created first and can output to the initial output descriptor. A necessary change is to run all child processes before waiting on one as well as before writing, so that there's no deadlock even if the pipe buffer wouldn't suffice for the heredoc.
void pipeline()
{
int i = 2; // create children from last to first
int fd[2];
pid_t pid;
int fdd = 1; // output of last child is STDOUT
while (i--)
{
pipe(fd);
pid = fork();
if (pid == 0)
{
dup2(fdd, 1); // child's output
dup2(fd[0], 0);
close(fd[0]);
close(fd[1]);
if (i == 0)
execlp("cat", "cat", "-A", NULL);
else
execlp("grep", "grep", "you" , NULL);
perror("error");
exit(1);
}
if (fdd != 1) close(fdd); // close if a pipe write end
fdd = fd[1]; // preceding child's output is pipe write end
close(fd[0]);
}
write(fd[1], "hello\nhow\nare\nyou\n", 17);
close(fd[1]); // signal EOF to child
while (wait(NULL) > 0) ; // wait for all children
}
I have taken a look at this and also this stack overflow links.
I am having trouble understanding the process for closing write ends of pipes. In the code below, I have 3 processes, one parent, a child of the parent, and a child of the child. I am trying to simulate a pipe for the command - cat xxx | grep 28 | sort. I have written some code for this, and it essentially creates the sorts, "grips"/filters and prints my input, but it hangs at the end. I have to ctrl + c to exit. My code is a little messy, but if you can help me spot the problem that would be nice.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
/**
* Executes the command "cat scores | grep Lakers". In this quick-and-dirty
* implementation the parent doesn't wait for the child to finish and
* so the command prompt may reappear before the child terminates.
*
*/
int main(int argc, char **argv)
{
int pipefd[2];
int pipefd2[2];
int pid;
char *cat_args[] = {"cat", "scores", NULL};
char *grep_args[] = {"grep", "28", NULL};
char *sort_args[] = {"sort", NULL};
// make a pipe (fds go in pipefd[0] and pipefd[1])
if (pipe(pipefd) != 0){
return 1;
}
if (pipe(pipefd2) != 0){
return 1;
}
pid = fork();
if (pid < 0)
{
fprintf(stderr, "fork Failed" );
exit(EXIT_FAILURE);
}
else if (pid == 0)
{
int pid2;
pid2 = fork();
if (pid2 < 0){
fprintf(stderr, "fork Failed" );
return 1;
}
else if (pid2 == 0){
// replace standard input with input part of pipe
// close(0);
// close(pipefd[1]);
// close(pipefd2[1]);
dup2(pipefd2[0], 0);
// close unused hald of pipe
close(pipefd2[0]);
close(pipefd[1]);
close(pipefd2[1]);
close(pipefd[0]);
// execute grep
execvp("sort", sort_args);
close(pipefd[1]);
close(pipefd2[1]);
exit(0);
}else{
// replace standard input with input part of pipe
// close(pipefd[1]);
// close(pipefd2[1]);
dup2(pipefd[0], 0);
dup2(pipefd2[1], 1);
// close unused hald of pipe
close(pipefd[0]);
close(pipefd2[1]);
close(pipefd[1]);
close(pipefd2[0]);
// execute grep
execvp("grep", grep_args);
waitpid(pid2, NULL, 0);
close(pipefd[1]);
close(pipefd[0]);
close(pipefd2[1]);
close(pipefd2[0]);
exit(0);
waitpid(pid2, NULL, 0);
}
}
else
{
// close(pipefd[1]);
// close(pipefd2[1]);
dup2(pipefd[1], 1);
// close unused unput half of pipe
close(pipefd[1]);
close(pipefd[0]);
close(pipefd2[1]);
close(pipefd2[0]);
// execute cat
execvp("cat", cat_args);
exit(0);
waitpid(pid, NULL, 0);
}
close(pipefd[1]);
close(pipefd[0]);
close(pipefd2[1]);
close(pipefd2[0]);
}
here is the output I am getting. Not sure it is relevant but as you can see, the result is sorted by team name. It just doesn't terminate.
Houston 44 28 .611
Indiana 45 28 .616
Oklahoma City 44 28 .611
Utah 44 28 .611
^C
Calling execvp replaces the current process image with a new process image. If no error occured, your code will never reach any line after that, so your close() and waitpid() function calls are useless.
EDIT
Here's a fully functional code to your problem. The comments should be self explanatory. Notice that the command executing order is different and I'm waiting for processes to finish.
Reading from an empty pipe will block until there is some data to read or an error occured, so this is not the only solution.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
static void die (const char *msg) {
perror (msg);
exit (EXIT_FAILURE);
}
int main (int argc, char** argv) {
int pipefd[2];
int pid;
char *cat_args[] = {"cat", "scores", NULL};
char *grep_args[] = {"grep", "28", NULL};
char *sort_args[] = {"sort", NULL};
//make a pipe (file descriptor to read is pipefd[0], fd to write is pipefd[1])
if (pipe (pipefd) < 0)
die ("creating a pipe failed");
pid = fork();
if (pid < 0)
die ("fork");
else if (pid == 0) {
//child process
int pipefd2[2]; //only visible to the affected processes
if (pipe (pipefd2) < 0)
die ("pipe");
int pid2;
pid2 = fork();
if (pid2 < 0)
die ("fork");
else if (pid2 == 0) {
//child of child will execute cat command
close (pipefd2[0]); //we only need to write to the second pipe. close its reading end
//first pipe is for communication between parent and grandparent only
close (pipefd[0]);
close (pipefd[1]);
dup2 (pipefd2[1], STDOUT_FILENO); //write the output to the second pipe instead of the standard output
close (pipefd2[1]); //close writing end of second pipe
execvp("cat", cat_args); //execute cat command
die ("execvp should never return");
}
else {
//child process will execute the grep command
close (pipefd2[1]); // we only need to read from the second pipe. close its writing end
close(pipefd[0]); //we won't read from the first pipe
waitpid (pid2, NULL, 0); //wait for cat command to finish
dup2 (pipefd2[0], STDIN_FILENO); //read from the second pipe instead of the stdin
close (pipefd2[0]); //child finished. close reading end of second pipe
dup2 (pipefd[1], STDOUT_FILENO); //write the results of grep command to first pipe instead of standard output
close (pipefd[1]); //we dup2 the output, close the writing end of first pipe
execvp ("grep", grep_args);
die ("execvp should never return");
}
} else {
//parent process will execute the sort command
close (pipefd[1]); //we won't write to the first pipe
waitpid (pid, NULL, 0); //wait for child to write grep output to the first pipe
dup2 (pipefd[0], STDIN_FILENO); //read from the first pipe instead of stdin
close (pipefd[0]); //child finished. close reading end of first pipe
execvp ("sort", sort_args); //execute command
die ("execvp should never return");
}
//exit (EXIT_SUCCESS); we don't need this. the programm will never reach this line
}
I'm trying to reproduce this command in c language:
ls | wc > output.txt
So, to do that, I wrote the following program:
#include <unistd.h>
#include <stdlib.h>
#include <fcntl.h>
#include <stdio.h>
#include <errno.h>
int main()
{
pid_t lsFork, wcFork;
int tube[2];
pipe(tube);
lsFork = fork();
if(lsFork == 0) // ls command
{
close(tube[0]);
dup2(tube[1], STDOUT_FILENO);
close(tube[1]);
if(execl("/usr/bin/ls", "ls", NULL) == -1)
perror("Cannot execute ls");
}
else
{
wcFork = fork();
if(wcFork == 0) // wc command
{
sleep(1);
int file = open("output.txt", O_WRONLY | O_CREAT);
if(file == -1)
perror("Cannot open output.txt");
close(tube[1]);
dup2(tube[0], STDIN_FILENO);
close(tube[0]);
dup2(file, STDOUT_FILENO);
close(file);
/*char buffer[BUFSIZ];
read(STDIN_FILENO, buffer, BUFSIZ);
write(STDOUT_FILENO, buffer, BUFSIZ);*/
if(execl("/usr/bin/wc", "wc", NULL) == -1)
perror("Cannot execute wc");
close(STDOUT_FILENO);
}
else // parent
{
int status;
waitpid(lsFork, &status, 0);
waitpid(wcFork, &status, 0);
}
}
return EXIT_SUCCESS;
}
But, the program does not exit. According to htop, the wc command is blocking the program. To understand this behaviour, I wrote a piece of code (the lines commented before execl()) and I don't understand what this works and not execl(). Am I forgetting something when calling this function?
The parent process still has the pipe open, so wc is waiting around in case the parent decides to write stuff (which wc would need to count).
Close both ends of the pipe in the parent too:
else // parent
{
int status;
close(tube[0]); // <---
close(tube[1]); // <---
waitpid(lsFork, &status, 0);
waitpid(wcFork, &status, 0);
}
Don't complicate things when you can do it easily..
Try the simpler code below & see if you can understand anything or not.
int main(){
int tube[2];
int fID;
pipe(tube);
if (fork() == 0){
// this is the child process
close(tube[0]); // reading end of the pipe
dup2(tube[1], 1); // stdout ---> pipe writing end
execlp("ls", "ls", NULL);
}else{
if (fork() == 0){
//umask(0022);
fID = open("sample.txt", O_WRONLY | O_CREAT, 0644);
close(tube[1]); // writing end of the pipe
dup2(tube[0], 0); // stdin ----> pipe reading end
dup2(fID, 1);
execlp("wc", "wc", NULL);
}
}
return 0;
}
Note If the purpose of the code is to solely implement the above mentioned piping, then you don't need to implement any waiting mechanisms. The OS will auto-kill all the zombie child, if any. Moreover execlp("wc", "wc", NULL); will auto block the program to end. Hence it will not exit early
You'll need to close the write end of the pipe in the parent too.
I'm creating a small program which contains three processes; a source process, a filter process and a sink process. The stdout of the source process is redirected to the stdin of the filter process, and the filter process' stdout is redirected to the sink process' stdin.
My problem is that no output is printed to stdout from the sink process. Can any of you see the problem in the following tiny snippet of code?
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char** argv)
{
// Pipes
// pipe1 is from source to filter, pipe2 is from filter to sink
int pipe1[2], pipe2[2];
// Create pipes
if (pipe(pipe1) < 0 || pipe(pipe2) < 0)
{
perror("Creating pipes failed!");
}
if (fork() == 0)
{
close(1);
dup(pipe1[1]);
close(pipe1[0]);
close(pipe2[0]);
close(pipe2[1]);
execlp("ls", "ls", NULL);
exit(0);
}
else
{
if (fork() == 0)
{
close(0);
dup(pipe1[0]);
close(pipe1[1]);
close(1);
dup(pipe2[1]);
close(pipe2[0]);
execlp("sort", "sort", NULL);
exit(0);
}
else
{
if (fork() == 0)
{
close(0);
dup(pipe2[0]);
execlp("more", "more", NULL);
exit(0);
}
}
}
wait(NULL);
printf("Done.\n");
return 0;
}
BR
Jacob
I think problem may be, wait will only wait for one process. And when the parent exits after first child returns, I suspect more command also decides to terminate, because it may get SIGHUP (speculation, not sure).
But, check for errors from on all system calls! Also for wait calls which succeeded, print why the child exited (was it signal or normal exit, and if it was normal exit, what was exit code).
Also note, perror does not exit, it only prints.
It is kind of pointless trying to see why some code fails, if it does not have error handling in it...
Some easy way to do pipes for your scenario:
char cmd[MAX_LEN];
sprintf(cmd, "%s | %s | %s", app1, app2, app3); //app123 holds app name + args
system(cmd);
if you want to capture the output of the last app, use popen:
FILE pPipe = popen(cmd, "rt"); /* same access flag as fopen()*/
while (NULL != fget(buf, buf_len, pPipe)) {
// do something with the read line in 'buf'
}