I've been trying to use the pipe() system call to create a shell that supports piping (with an arbitrary number of commands).
Unfortunately, I haven't had much luck using pipe(). After spending a few days looking at various online resources, I decided to put together an oversimplified program that has the same effect as executing ls | sort to see if I could even get a pipe to work for two sibling, child processes. Here's the code:
#include <sys/wait.h>
#include <unistd.h>
void run(char *cmd) {
char *args[2];
args[0] = cmd;
args[1] = NULL;
execvp(cmd, args);
}
int main(void) {
int filedes[2];
pipe(filedes);
pid_t pid_a, pid_b;
if (!(pid_a = fork())) {
dup2(filedes[1], 1);
run("ls");
}
if (!(pid_b = fork())) {
dup2(filedes[0], 0);
run("sort");
}
waitpid(pid_a, NULL, 0);
waitpid(pid_b, NULL, 0);
return 0;
}
The pipe is created in the parent and I know that after the execvp() call, each child process inherits the file descriptors that pipe() creates in the parent. For the ls process, I'm using dup2() to redirect its standard out (1) to the write-end of the pipe and for the sort process, standard in (0) is being redirected to the read-end of the pipe.
Finally, I wait for both processes to finish before exiting.
My intuition tells me this should work, but it doesn't!
Any suggestions?
You have to close the pipes you're not using.
at least sort will read from its stdin until stdin is closed.
In this case, it's stdin is never closed, as you still have 2 open filedescriptors for it.
filedes[0] in the ls child (this likely gets closed when ls finishes)
filedes[0] in the parent program (this never gets closed as you waitpid() for sort to end, but it never will because the parent keeps its stdin open)
Change your program to
if (!(pid_a = fork())) {
dup2(filedes[1], 1);
closepipes(filedes);
run("ls");
}
if (!(pid_b = fork())) {
dup2(filedes[0], 0);
closepipes(filedes);
run("sort");
}
closepipe(filedes);
waitpid(pid_a, NULL, 0);
waitpid(pid_b, NULL, 0);
where closepipes is something like
void closepipes(int *fds)
{
close(fds[0]);
close(fds[1]);
}
Before calling waitpid in the parent process you have to close all file descriptors from the pipe that you don't need. These are:
filedes[0] in pid_a
filedes[1] in pid_b
both filedes[0] and filedes[1] in the parent process
You should also check that pipe() and fork() didn't return -1, which means an error happened.
You need to close (at least) the writing end of the pipe in the parent process. Otherwise, the reading end of the pipe will never read EOF status, and sort will never finish.
This code working properly...
#include <sys/wait.h>
#include <unistd.h>
using namespace std;
void run(char *cmd) {
char *args[2];
args[0] = cmd;
args[1] = NULL;
execvp(cmd, args);
}
void closepipe(int *fds)
{
close(fds[0]);
close(fds[1]);
}
int main(int argc,char *argv[]) {
int filedes[2];
pipe(filedes);
char lss[]="ls";
char sorts[]="sort";
pid_t pid_a, pid_b;
chdir(argv[1]);
if (!(pid_a = fork())) {
dup2(filedes[1], 1);
closepipe(filedes);
run(lss);
}
if (!(pid_b = fork())) {
dup2(filedes[0], 0);
closepipe(filedes);
run(sorts);
}
closepipe(filedes);
waitpid(pid_a, NULL, 0);
waitpid(pid_b, NULL, 0);
return 0;
}
Related
How do I get the output of a program ran by exec(). Let's say I have this code:
int main(int argc, char ** argv) {
int fid = fork();
if(fid == 0) {
execlp("ls", "ls", NULL);
}
wait();
return 0;
}
How can the parent process get the output of the ls command?
The exec family of functions completely replaces the current process. However, they do not close file descriptors unless they marked close-on-exec. Thus, the typical way to do this is to create a pipe where the read side belongs to the parent and the write side belongs to the child.
This would look something like this (error checking omitted and obviously inefficient):
#include <stdint.h>
#include <sys/wait.h>
#include <unistd.h>
int main(int argc, char ** argv) {
int pipefd[2];
int status;
uint8_t buf[256];
pipe(pipefd);
int fid = fork();
if(fid == 0) {
close(pipefd[0]);
dup2(pipefd[1], 1);
close(pipefd[1]);
execlp("ls", "ls", NULL);
}
close(pipefd[1]);
while (read(pipefd[0], buf, 1) > 0)
write(1, buf, 1);
wait(&status);
return 0;
}
Note that to attach the pipe file descriptor to standard output (FD 1), you need to use dup2. You also need to close the ends of the pipe you're not using, or you may never end up reaching end of file.
If you're interested in the exit status, wait (or waitpid) will provide that for you; see the manual page for how to determine if it exited normally and if so, what that status was.
I am trying to execute a linux command by using execvp in c and it does not show anything. I dont quite understand the dup2() function call. Can any one tell me what i am doing wrong here?
Command To be executed: ls | sort
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <stdbool.h>
int main()
{
char *cmd[2], *cmd2[3];
cmd[0] = "ls";
cmd[1] = NULL;
cmd2[0] = "sort";
cmd2[1] = NULL;
pid_t id;
id = fork();
int pipe1[2];
int pipe2[2];
pipe(pipe1);
pipe(pipe2);
if(id==0)
{
printf("child process with pid =%d \r\n", getpid());
//the read end of pipe not needed,
close(pipe1[0]);
// we want the stdout (1) of this process to be connected to 1 of pipe1.
dup2(pipe1[1],1); //or dups(pipe1[1], stdout_fileno)
//close it aswell not needed
//close(pipe1[1]);
execvp("ls",cmd);
return 1;
}
pid_t id2 = fork();
if(id2==0)
{
close(pipe1[1]);
dup2(pipe1[0], 0);
close(pipe1[0]);
execvp("sort",cmd2);
//return 1;
}
close(pipe1[0]);
close(pipe1[1]);
waitpid(id,NULL,0);
waitpid(id2, NULL, 0);
//execvp("sort",cmd2);
//printf("parent process with pid =%d \r\n", getpid());
return 0;
}
You need only one pipe, and in fact you use only one, even though you create two. Except you actually create four, because you create the pipes after the first fork(), thus the parent and first child process create two each.
And that appears to be the problem. If you want the two child processes to communicate over a pipe, then they must use the same pipe. They would do so by inheriting the open file descriptors for the pipe ends from the parent process, which requires that they use a pipe created by a process from which both descend (i.e. the parent process), and which was created before either child was forked. As it stands, only the second child is doing that; the first is using a pipe that it creates itself, and which therefore is not connected to anything else.
Right now, I'm having to start an external process in C. I'm currently using posix_spawn to create the process. It is necessary that I can monitor whether or not the process has terminated. I need to also have a link to the standard out of the process. I've looked at using popen, however, it does not provide an "easy" way of getting the pid. I'm slowly going insane as it can't possibly be this hard to get the stdout of a running process in Linux.
Also, on a further note, I need help deciphering what the file_actions parameter is supposed to mean. man(3) for posix_spawn on this topic says:
If file_actions is not NULL, then the file descriptors open in the child process shall be those open in the calling process as modified by the spawn file actions object pointed to by file_actions and the FD_CLOEXEC flag of each remaining open file descriptor after the spawn file actions have been processed.
If that isn't the definition of a run-on sentence, I have no idea what is.
Since you have the PID (returned from posix_spawn) and you are running Linux, you will find the stdout of the process at /proc/<pid>/fd/1. Just open (or fopen) the file for reading.
The standard way is to use fork though. Use pipe and dup2 to get a file descriptor for reading the child's output, as in this question.
You can use posix_spawn for this, without having to use race-condition-prone, Linux-specific /proc/<pid>/fd/N. You can keep all the benefits of posix_spawn.
You were on the right track thinking about file_actions. Below is an example that prints out the child's stdout in Python-style triple quotes, as well as the child's exit code, from the parent process using posix_spawn and file_actions.
Here is an example of the example output.
child pid: 17468
child exit status: 0
child stdout:
"""Hello World!
"""
Here is the example.
#define _DEFAULT_SOURCE
#include <spawn.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/wait.h>
extern char **environ;
static void dump_child_stdout(int filedes)
{
ssize_t num_read;
char buf[1];
printf("child stdout:\n\"\"\"");
for (;;)
{
num_read = read(filedes, buf, sizeof(buf));
if (num_read > 0)
{
printf("%c", buf[0]);
}
else
{
break;
}
}
printf("\"\"\"\n");
}
int main(int argc, char *argv[])
{
int status;
pid_t pid;
int out[2];
posix_spawn_file_actions_t action;
char *args[] = {"/bin/echo", "Hello World!", NULL };
posix_spawn_file_actions_init(&action);
pipe(out);
posix_spawn_file_actions_adddup2(&action, out[1], STDOUT_FILENO);
posix_spawn_file_actions_addclose(&action, out[0]);
status = posix_spawn(&pid, args[0], &action, NULL, args, environ);
if (status == 0)
{
printf("child pid: %d\n", pid);
if (waitpid(pid, &status, 0) < 0)
{
perror("waitpid");
}
else
{
if (WIFEXITED(status))
{
printf("child exit status: %d\n", WEXITSTATUS(status));
}
else
{
printf("child died an unnatural death.\n");
}
close(out[1]);
dump_child_stdout(out[0]);
}
}
else
{
fprintf(stderr, "posix_spawn: %s\n", strerror(status));
close(out[1]);
}
posix_spawn_file_actions_destroy(&action);
return 0;
}
I'm very new with linux and so. I can't get my script working. I'm just guessing, that the program is getting suspended at executing tr function.
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
int main()
{
int pdesc[2];
pipe(pdesc);
int a = fork();
if (a == 0) // child
{
dup2(pdesc[1],1); // chaning std_out to pipes_out
execlp("ls", "ls", "-l", "-a", NULL);
}
else //parent
{
wait();
int file1 = open("file.txt", O_WRONLY|O_CREAT|O_TRUNC,0777);
dup2(pdesc[0], 0); // chaning std_in to pipes_in
dup2(file1, 1); // chaning std_out to file's stream
execlp("tr", "tr", "a-z", "A-Z", NULL);
}
return 0;
}
Classic mistake, so, good question.
You need to close the unused pipe file descriptors in both the parent and the child.
The process reading from the pipe has (itself) an open pipe write end, so the pipe never gets completely closed, so it never delivers an EOF.
Also, the wait(2) is causing a deadlock, the program doesn't include <sys/wait.h>, and the call to wait(2) is missing a required argument. Because the shell will wait for the parent to finish but not the child, it would be nice, actually, to have a wait(2) call in here somewhere. But in the current two-process design you have no place to put it, because you aren't in control after the parent's execlp(2). One way to fix that would be to have the parent fork() again, and have the original PID do nothing except wait(2) in a loop until all children have finished.
Here is a working version, note also the change to the output file mode.
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
int main()
{
int pdesc[2];
pipe(pdesc);
int a = fork();
if (a == 0) { // child
dup2(pdesc[1],1); // chaining std_out to pipes_out
close(pdesc[1]);
close(pdesc[0]);
execlp("ls", "ls", "-l", "-a", NULL);
} else { //parent
int file1 = open("file.txt", O_WRONLY|O_CREAT|O_TRUNC, 0644);
dup2(pdesc[0], 0); // chaning std_in to pipes_in
dup2(file1, 1); // chaning std_out to file's stream
close(pdesc[0]);
close(pdesc[1]);
close(file1);
execlp("tr", "tr", "a-z", "A-Z", NULL);
}
return 0;
}
I'm trying to run a program with a specific standard input. I succeed by using a file descriptor of a file where there is what I want to put in the stdin, but I fail to write directly on the stdin :
$cat input.test
echo Hello
$
Code C :
int main(int argc, char **argv)
{
int fd = 0;
fd = open("input.test", O_CREAT);
close(STDIN_FILENO);
dup2(fd, STDIN_FILENO);
char *const args[] = { "bash", NULL };
execvp("bash", args);
}
That works :
$./a.out
Hello
$
But if I try to write directly on the STDIN using pipe the program displays nothing and keeps running :
int main(int argc, char **argv)
{
int fds[2];
pipe(fds);
close(STDIN_FILENO);
dup2(fds[1], STDIN_FILENO);
write(fds[1], "echo Hello;", 11); // Résults are identics with fds[0]
char *const args[] = { "bash", NULL };
execvp("bash", args);
}
Thanks for your help
Cordially,
Bastien.
EDIT Problem solved:
Thanks for your answers, here the code which works :
int main(void)
{
int fd[2];
pid_t pid;
if (pipe(fd) < 0)
return EXIT_FAILURE;
if ((pid = fork()) < 0)
return EXIT_FAILURE;
else if (pid != 0) { /* father */
close(fd[1]);
dup2(fd[0], STDIN_FILENO);
execlp("bash", "bash", (char *)0);
} else { /* son */
close(fd[0]);
write(fd[1], "echo hello\n", 11);
}
return EXIT_SUCCESS;
}
You need to dup the read side of the pipe to stdin, not the write side. (And write to the write side, obviously.)
#include <unistd.h>
#include <string.h>
#include <stdio.h>
int main(int argc, char **argv)
{
int fds[2];
char cmd[] = "echo hello\nexit\n";
pipe(fds);
close(STDIN_FILENO);
dup2(fds[0], STDIN_FILENO);
write(fds[1], cmd, strlen(cmd));
char *const args[] = { "bash", NULL };
execvp("bash", args);
return 0;
}
Make sure you check the return values of all those functions though, you'll never manage to debug your code if you don't.
execv and friends replace the current running program with the specified one; they do not return - execution continues at the start of new program instead.
So what you normally do is fork and, in one of the forks, call execv. You then read and write through the pipe from your program continuing in the other fork. There are usually popen functions to do this in most languages; sadly in POSIX the popen() is strictly read or write and not bidirectional.
Luckily, I've made, tested and published a popen3 function. This gives you back three file descriptors - one for stdin to the process, and two for stdout and stderr. You can then use write() on the stdin.
When you call pipe, fd[ 0 ] is open for reading, and fd[ 1 ] is open for writing. You should be dup'ing stdin on the read side ( fd[ 0 ]) and writing to the write side( fd[ 1 ]). Check the return value of write: it is probably -1.
But there is a larger issue. You never close either side of the pipe. bash may block on a read and never do anything until the write side of the pipe is closed. You should close both sides of the pipe after you dup and write. (Or set FD_CLOEXEC).
Also note that doing it the way you do, you're dependent on pipe buffer size. If you write too much, write will be blocked as there's no reader. Do do it reliably, you should fork(), do exec in the child and write to the pipe in the parent. This way the pipe will have a reader and you will be able to write as much data as you want into it.