I am trying to implement piping in Unix, and have been asked to do it recursively. I have a sh program that parses input by the pipe character and then forks a child process to begin the piping. I am going to use the command cat file | grep the | more as an example.
My sh.c program first parses the input string into pipeCmds which is just an array of char * that point at the different parts of the input string. It forks the first child and then begins the recursivePipe() call which should set up all the pipes. After the pipes are set up I exec("more") in this case.
pid = fork();
if(pid == 0){
recursivePipe(pipeCmds[numCmds-2], numCmds-1);
exec(pipeCmds[numCmds-1]); // exec("more")
}else{
pid = wait(getpid());
}
Here is the recursivePipe function that should set up each pipe based on how many commands were in the string (ie. numCmds)
int recursivePipe(char * cmd, int index){
/* cmd = more */
int pid, fd, copy;
int ttyfd;
char tty[64];
if(index < 1){
printf("index is 0... RETURN\n");
return;
}
pipe(pd);
// First fork a new child to stage the WRITING proc
printf("forking to %s from %s\n", cmd, pipeCmds[index]);
pid = fork();
if(pid == 0){
// Child
close(pd[0]); // Close the child's read
//if(index != (numCmds-2)){ // If we are not on the last command, make stdout be the pipe
dup2(pd[1], 1); // Place the WRITE end of the pipe into stdout so anything coming from the pipe
close(pd[1]);
//}
copy = dup(1);
close(1);
gettty(tty);
ttyfd = open(tty, O_WRONLY);
if(ttyfd == 1){
printf("exec(%s) from %s\n", cmd, pipeCmds[index]);
close(1);
dup(copy);
close(copy);
}
/*copy = dup(0);
close(0);
gettty(tty);
ttyfd = open(tty, O_RDONLY);
if(ttyfd == 0){
getc();
close(0);
dup(copy);
close(copy);
}*/
exec(cmd);
}else{
// Parent
printf("in parent: %s[%d]\n", pipeCmds[index], index);
close(pd[1]); // Close the parent's write since more doesn't have to write
//if(index != 0){ // If we are not on the first command, make stdin be the pipe
dup2(pd[0], 0); // Place the READ end of pipe into stdin so that more's stdin comes from the pipe
close(pd[0]);
//}
printf("lets recurse!: %s[%d]\n", pipeCmds[index], index);
// if(index > 0){
recursivePipe(pipeCmds[index-2], index-1); // This will set up all other procs too with correct pipes
//pid = wait(getpid());
// }
printf("done recurse!: %s[%d]\n", pipeCmds[index], index);
}
}
Basically, I attempt to pipe(), then in the child process I close the READ end of the pipe and set stdout to now be the pipe. So in this case, on the first call of recursivePipe(), the parent section is the "more" proc and the child section is the "grep the" part. So "more" closes its stdin and replaces it with the pipe so it reads all output from "grep the"
Based on my printf() inside the function, here is the output of that command:
forking to grep the from more
in parent: more[2]
lets recurse!: more[2]
forking to cat file from grep the
exec( grep the ) from more
exec(cat file) from grep the
in parent: grep the [1]
lets recurse!: grep the[1]
index is 0... RETURN
done recurse!: grep the [1]
done recurse!: more[2]
And then it appears to cat file correctly and send it to more, but the grep program is never used between the two. It is as if cat and more are communicating directly rather than through grep.
Can anyone with knowledge of the unix system help me figure out why my recursion isn't setting up the pipes correctly? Thanks
Related
I'm trying to execute ls | wc -l through a program in C, instead of using the command line.
This is my current working code:
int main() {
int pfds[2];
pipe(pfds);
pid_t pid = fork();
if ( pid == 0 ) { /* The child process*/
close(1);
dup(pfds[1]);
close(pfds[0]);
execlp("ls", "ls", NULL);
} else { /* The parent process*/
close(0);
dup(pfds[0]);
close(pfds[1]);
wait(0);
execlp("wc", "wc", "-l", NULL);
}
return 0;
}
How would I rewrite this code to work with a for-loop?
For example:
for (i=0; i<2; i++) {
// Rewrite the 2-level pipe here
}
Later, I would like to extend the for loop to execute more processes piped together like a | b | c | ...
In order to pipe multiple commands together, you'll need to keep the parent running to keep fork()ing for each command.
Using a for loop, you will need to do this for the first n - 1 commands (the last one will be executed in the main program):
Create a pipe.
Execute fork().
In the child: overwrite standard input with the read end of the previous pipe, and standard output with the write end of the current pipe.
In the child: execute execve().
In the parent: close unneeded pipes and save read end of current pipe to be used in the next iteration.
Then, after the loop ends, overwrite standard input with the read end of the last pipe and execute execve() of the last command.
Below I've written a simple working example that executes:
ls | wc -l | xargs printf "0x%x\n" | cowsay
It should work for any number of commands (including only 1 single command).
NOTE: I did not add error checks in this code apart for execvp() just to make it short, but you should definitely check for errors after each call to pipe(), dup2(), fork() and any other function.
Code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define MAX_ARGC 3
int main(void) {
char *commands[][MAX_ARGC + 1] = {
{"ls", NULL},
{"wc", "-l", NULL},
{"xargs", "printf", "0x%x\n", NULL},
{"cowsay", NULL}
};
size_t i, n;
int prev_pipe, pfds[2];
n = sizeof(commands) / sizeof(*commands);
prev_pipe = STDIN_FILENO;
for (i = 0; i < n - 1; i++) {
pipe(pfds);
if (fork() == 0) {
// Redirect previous pipe to stdin
if (prev_pipe != STDIN_FILENO) {
dup2(prev_pipe, STDIN_FILENO);
close(prev_pipe);
}
// Redirect stdout to current pipe
dup2(pfds[1], STDOUT_FILENO);
close(pfds[1]);
// Start command
execvp(commands[i][0], commands[i]);
perror("execvp failed");
exit(1);
}
// Close read end of previous pipe (not needed in the parent)
close(prev_pipe);
// Close write end of current pipe (not needed in the parent)
close(pfds[1]);
// Save read end of current pipe to use in next iteration
prev_pipe = pfds[0];
}
// Get stdin from last pipe
if (prev_pipe != STDIN_FILENO) {
dup2(prev_pipe, STDIN_FILENO);
close(prev_pipe);
}
// Start last command
execvp(commands[i][0], commands[i]);
perror("execvp failed");
exit(1);
}
Output on my machine (since ls returned 41 == 0x29 lines):
______
< 0x29 >
------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
I'm trying to execute ls | wc -l through a program in C, instead of using the command line.
This is my current working code:
int main() {
int pfds[2];
pipe(pfds);
pid_t pid = fork();
if ( pid == 0 ) { /* The child process*/
close(1);
dup(pfds[1]);
close(pfds[0]);
execlp("ls", "ls", NULL);
} else { /* The parent process*/
close(0);
dup(pfds[0]);
close(pfds[1]);
wait(0);
execlp("wc", "wc", "-l", NULL);
}
return 0;
}
How would I rewrite this code to work with a for-loop?
For example:
for (i=0; i<2; i++) {
// Rewrite the 2-level pipe here
}
Later, I would like to extend the for loop to execute more processes piped together like a | b | c | ...
In order to pipe multiple commands together, you'll need to keep the parent running to keep fork()ing for each command.
Using a for loop, you will need to do this for the first n - 1 commands (the last one will be executed in the main program):
Create a pipe.
Execute fork().
In the child: overwrite standard input with the read end of the previous pipe, and standard output with the write end of the current pipe.
In the child: execute execve().
In the parent: close unneeded pipes and save read end of current pipe to be used in the next iteration.
Then, after the loop ends, overwrite standard input with the read end of the last pipe and execute execve() of the last command.
Below I've written a simple working example that executes:
ls | wc -l | xargs printf "0x%x\n" | cowsay
It should work for any number of commands (including only 1 single command).
NOTE: I did not add error checks in this code apart for execvp() just to make it short, but you should definitely check for errors after each call to pipe(), dup2(), fork() and any other function.
Code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define MAX_ARGC 3
int main(void) {
char *commands[][MAX_ARGC + 1] = {
{"ls", NULL},
{"wc", "-l", NULL},
{"xargs", "printf", "0x%x\n", NULL},
{"cowsay", NULL}
};
size_t i, n;
int prev_pipe, pfds[2];
n = sizeof(commands) / sizeof(*commands);
prev_pipe = STDIN_FILENO;
for (i = 0; i < n - 1; i++) {
pipe(pfds);
if (fork() == 0) {
// Redirect previous pipe to stdin
if (prev_pipe != STDIN_FILENO) {
dup2(prev_pipe, STDIN_FILENO);
close(prev_pipe);
}
// Redirect stdout to current pipe
dup2(pfds[1], STDOUT_FILENO);
close(pfds[1]);
// Start command
execvp(commands[i][0], commands[i]);
perror("execvp failed");
exit(1);
}
// Close read end of previous pipe (not needed in the parent)
close(prev_pipe);
// Close write end of current pipe (not needed in the parent)
close(pfds[1]);
// Save read end of current pipe to use in next iteration
prev_pipe = pfds[0];
}
// Get stdin from last pipe
if (prev_pipe != STDIN_FILENO) {
dup2(prev_pipe, STDIN_FILENO);
close(prev_pipe);
}
// Start last command
execvp(commands[i][0], commands[i]);
perror("execvp failed");
exit(1);
}
Output on my machine (since ls returned 41 == 0x29 lines):
______
< 0x29 >
------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
I've been trying to write a program that will send and receive commands to a bash shell (/bin/sh). Like a wrapper program around a bash shell. So, I could write to stdin "cd ~/Desktop", then write again "ls" and I will receive a listing of the files on the desktop. I can't get it working though. On the second write command in this code, it will echo back whatever I wrote to stdin. I've also tried using popen() but that only provides output, not allowing me to write to stdin. Could someone please help solve this problem? Thanks
void main()
{
// Create a pipe and fork
//
int fd[2];
int p = pipe(fd);
pid_t pid = fork();
if (pid > 0)
{
// Read from the pipe and output the result
//
//close(fd[1]);
char buf[1024] = { 0 };
read(fd[0], buf, sizeof(buf));
printf("1 - %s\n", buf);
write (fd[1], "ifconfig", strlen ("ifconfig") );
// problem is here, read is returning echo'd bytes from write()
read(fd[0], buf, sizeof(buf));
printf("2 - %s\n", buf);
// Wait for child to terminate
int status;
wait(&status);
}
else if (pid == 0)
{
// Redirect stdout and stderr to the pipe and execute the shell
// command
//
dup2(fd[0], STDIN_FILENO);
dup2(fd[1], STDOUT_FILENO);
dup2(fd[1], STDERR_FILENO);
//close(fd[0]);
execl("/bin/sh", "exec sh", "-c", "ls", (char*) NULL );
}
}
EDIT - Updated code per 1st answer, now there is no output from the 2nd read() call
void main()
{
// Create a pipe and fork
//
int fd[2];
int ChildToParent[2], ParentToChild[2];
pipe (ParentToChild);
pipe (ChildToParent);
pid_t pid = fork();
if (pid > 0)
{
// In parent process
// Read the output of the child from child_to_parent[0]
// We don't need child_to_parent[1] so close it
close(ChildToParent[1]);
// Write output to the child using parent_to_child[1]
// We don't need parent_to_child[0] so close it
close(ParentToChild[0]);
// Read from and write to the child process...
char buf[1024] = { 0 };
read(ChildToParent[0], buf, sizeof(buf));
printf("1 - %s\n", buf);
write(ParentToChild[1], "whoami", strlen ("whoami") );
memset (buf, 0, 1024);
// this call to read returns nothing
read(ChildToParent[0], buf, sizeof(buf));
printf("2 - %s\n", buf);
}
else if (pid == 0)
{
// Redirect stdout and stderr to the pipe and execute the shell
// command
//
// child_to_parent[1] is were we write output, it's the
// new standard output, child_to_parent[0] can be closed
dup2 (ChildToParent[1], STDOUT_FILENO);
close(ChildToParent[0]);
// parent_to_child[0] is where we read input from, it's the
// new standard input, parent_to_child[1] can be closed
dup2 (ParentToChild[0], STDIN_FILENO);
close(ParentToChild[1]);
//close(fd[0]);
execl("/bin/sh", "exec sh", "-c", "ls", (char*) NULL );
}
}
Remember that pipes are a one-way communication stream. You can't use it for two-way communication between two processes. For that you need two pipes, one in each direction.
Perhaps something like this simple example:
// Pipe for the child process to write to the parent process
int child_to_parent[2];
// Pipe for the parent process to write to the child process
int parent_to_child[2];
// Create the TWO pipes
pipe(child_to_parent);
pipe(parent_to_child);
pid_t pid = fork();
if (pid > 0)
{
// In parent process
// Read the output of the child from child_to_parent[0]
// We don't need child_to_parent[1] so close it
close(child_to_parent[1]);
// Write output to the child using parent_to_child[1]
// We don't need parent_to_child[0] so close it
close(parent_to_child[0]);
// Read from and write to the child process...
}
else if (pid == 0)
{
// In child process
// child_to_parent[1] is were we write output, it's the
// new standard output, child_to_parent[0] can be closed
dup2(child_to_parent[1], STDOUT_FILENO);
close(child_to_parent[0]);
// parent_to_child[0] is where we read input from, it's the
// new standard input, parent_to_child[1] can be closed
dup2(parent_to_child[0], STDIN_FILENO);
close(parent_to_child[1]);
// Do whatever the child is supposed to do
}
I am trying to run ls|wc using execvp. So I create a pipe and then fork to create a child. I close the appropriate(read./write) end in parent/child and then map the other end to stdout/stdin. Then I run the ls in parent using execvp and wc in child. When I run the program it says
wc:standard input:bad file descriptor.
0 0 0
wc: -:Bad file descriptor
Here is my code:
int main()
{
//int nbBytes = 0; //stream length
int pfd_1[2]; //file descriptor
//char buffer[MAX_FILE_LENGTH];
char* arg[MAX_FILE_LENGTH];
pid_t processPid;
//Create a pipe
if(pipe(pfd_1) == -1)
{
printf("Error in creating pipe");
return 0;
}
//Create a child
processPid = fork();
if(processPid == -1)
{
printf("Erro in fork");
exit(1);
}
else if(processPid == 0) //Child
{
//redirect read end file descriptor to standard input
dup2(pfd_1[0],0);
//Close the write end
if(close(pfd_1[1] == -1))
{
printf("Error in closing the write end file descriptor");
exit(1);
}
arg[0] = "wc";
//arg[1] = "-l";
arg[1] = '\0';
if(execvp(arg[0],arg) == -1)
{
printf("Error in executing ls");
}
}
else //Parent
{
//redirect standard output to the file descriptor
dup2(pfd_1[1],1);
//Close the read end
if(close(pfd_1[0] == -1))
{
printf("Error in closing the read end from parent");
exit(1);
}
//Command
arg[0] = "ls";
arg[1] = "/proc/1/status";
arg[2] = '\0';
if(execvp(arg[0],arg) == -1)
{
printf("Error in executing ls");
}
}
}
Any idea what might be wrong? Why would it consider standard input as bad file descriptor? My understanding was since the stdin and read end file descriptor are aliases so the wc -l would read whatever the output is from the parent process. Do I need to do scanf to read from the stdin?
The problem is in this line:
if(close(pfd_1[1] == -1))
You are closing the result of pfd_1[1] == -1, which is by necessity equal to 0 (as they will never be equal). The correct line would probably be:
if (close(pfd_1[1]) == -1)
Note that you do this again later in attempting to close the read end in the parent process.
If you're going to fork children, you have to call wait() in the parent process in order to avoid "zombie" child processes. So you don't want to overlay the parent process that did the original process forking with another executable via exec.
One quick way to setup a series of pipes in the way you want would be to fork a child for each executable you want to run, and read that data back into a buffer in the parent. Then feed that data from the first child into a new child process that the parent forks off. So each child is fed data from the parent, processes the data, and writes the data back to the parent process, which stores the transformed data in a buffer. That buffer is then fed to the next child, etc., etc. The final results of the data in the buffer are the final output of the pipe.
Here's a little pseudo-code:
//allocate buffer
unsigned char buffer[SIZE];
for (each executable to run in pipeline)
{
pipes[2];
pipe(pipes);
pid_t pid = fork();
if (pid == 0)
{
//setup the pipe in the child process
//call exec
}
else
{
//setup the pipe in the parent process
if (child executable is not the first in the pipeline)
{
//write contents of buffer to child process
}
//read from the pipe until the child exits
//store the results in buffer
//call wait, and maybe also check the return value to make sure the
//child returned successfully
wait(NULL);
//clean up the pipe
}
}
I have a homework to do that says the following:
Write a program in C that creates a child who will also create a child, make a pipe between the three processes, the fist process(father) will connect the second(child) and the child will connect with the third (child of the child). Our program should display the total number of system users who use bash as default shell. The result of the program should be identical to the "cat / etc / passwd | grep" / bin / bash $ "| wc-l"
I am confused with the first child and the method that we close the first pipe and open the second in the same time. If you reply me with the right code I 'll undestand it right once.
Thank you.
Here is what I 've wrote so far:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
main()
{
int pid, pid2;
int fd[2];
int fd2[2];
char *arg[3];
char *arg2[3];
char *arg3[3];
if (pipe(fd) == -1)
{
perror("pipe");
exit(1);
}
pid = fork();
if (pid == -1)
{
perror("fork");
exit(2);
}
if (pid == 0)
{
if (pipe(fd2) == -1)
{
perror("pipe");
exit(11);
}
pid2=fork();
if(pid2 == -1)
{
perror("fork 2");
exit(22);
}
if (pid2 == 0)
{
//i am child 2 (child of the child)
close (fd2[1]);
dup2 (fd2[0],0);
close (fd2[0]);
arg3[0] = "wc";
arg3[1] = "-l";
arg3[2] = NULL;
execvp("wc", arg3);
perror("execvp second child");
}
else
{
//i am child 1
close (fd[1]);
dup2(fd[0],0);
close (fd[0]);
close (fd2[0]);
dup2(fd2[1],1);
close (fd2[1]);
arg2[0] = "grep";
arg2[1] = "/bin/bash$";
arg2[2] = NULL;
execvp("grep", arg2);
perror("execvp first child");
}
}
else
{
//i 'm the father
close (fd[0]);
dup2(fd[1],1);
close (fd[1]);
arg[0] = "cat";
arg[1] = "/etc/passwd";
arg[2] = NULL;
execvp("cat", arg);
perror("execvp father");
}
}
Your program very nearly works. What's missing is
//i am child 2 (child of the child)
close (fd[1]);
close (fd[0]);
The pipe you called fd is for communicating between 'cat' and 'grep'. What's happening in your current code is that cat dumps the file and exits, closing its output. Grep reads all of that and waits for the EOF on its input. Since "child 2" still has the input side of the pipe open (it inherited it via fork), grep waits forever. If run your program and then type ps you should see a grep and a wc hanging around waiting to finish.
The other thing you would normally do when constructing a pipeline like this is arrange it so that the final task (in this case wc) is the one that the shell is waiting for. As written, when your program is run from the shell it will appear to finish when cat finishes, and the output of wc will print as if from a background task. If you arrange the pipe so that wc is under "i am child 1" then the shell will be waiting for wc instead.
Alternatively you could fork all of the three processes off and "child 1" would invoke wait() to wait for all of them before exiting. That waiting process would be like your own tiny shell.