I've been searching and reading manuals, but still can't get the pipe mechanism. I'm making a program, which should do the following:
Parent process creates pipe, two child proccesses and waits.
First child generates pairs of random numbers and passes them with space between to the second process via pipe. It goes on till
gets signal from parent.
Second child redirects it input so it is the output of the first child and redirects output to the out.txt file. Then it executes
already compiled program, calculating GCD of numbers from (1);
Parent closes pipe and kills child.
So I got this C code (I reduced it so the post can fit the rules):
const int PIPE_READEND=0;
const int PIPE_WRITEEND=1;
(...)
if (child1 == 0) {
//Child1 code here
close(fd[1]);
struct sigaction sa;
sa.sa_handler = sigHandler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
if (sigaction(SIGUSR1,&sa,NULL) == -1){ //Handling SIGUSR1 signal
perror("Signal handling unexpected error");
exit(errno);
}
int a,b;
srand(time(&t));
if (dup2(fd[PIPE_READEND],1) < 0){ //Redirecting stdout to the pipe fd.
perror("In Child1 Redirecting stdout to pipe error");
exit(errno);
}
close(fd[0]);
while(1){
a = rand();
b = rand();
printf("%d %d\n", a, b);
sleep(1);
}
(...)
if ((child2 = fork()) < 0){
perror("Fork error in Child2 process");
exit(errno);
} else if (child2 == 0){
//Child2 code here
close(fd[PIPE_READEND]);
FILE *outfile = fopen("out.txt","w");
dup2(fd[PIPE_WRITEEND],0);
dup2(outfile,1);
close(fd[PIPE_WRITEEND]);
execl("c1/main","main",(char *)NULL);
The problem is, after executing it, out.txt stays empty. I'm disappointed about the pipe array indices, which one is used for what.
You write and read from wrong indexes of pipe. You need to change them:
This redirects stdout to pipe input.
close(fd[0]);
dup2(fd[1], STDOUT_FILENO);
This redirects pipe output to stdin.
close(fd[1]);
dup2(fd[0], STDIN_FILENO);
Also dup2 takes integer, not pointer, so you should do:
f = fopen("out.txt", "w");
dup2(fileno(f), STDOUT_FILENO);
FILE *outfile = fopen("out.txt","w");
dup2(fd[PIPE_WRITEEND],0);
dup2(outfile,1);
This makes no sense. The dup2 function doesn't take a FILE * as a parameter. Use open, not fopen.
Related
I'm writing a function that echo an input to a sed and then another sed. I thinck i used all my wait signal in the right way but the last print i can get is before the call to dup2() in my first child process in the echo.
void sendbc (char * str_ ) {
int fd[2];
int fd1[2];
int pid,pid1;
char* echo[] = {"echo", str_,NULL};
char* sed1[] = {"sed","s/[^:]*;"" " "//",NULL};
char* sed2[] = {"sed","s/[^:]*."" " "//",NULL};
int status,er;
FILE *f;
if(pipe(fd) < 0){
exit(100);
}
if(pipe(fd1) < 0){
exit(100);
}
pid = fork();
if (pid == 0) {
dup2(fd[1], 1) //last command before blocking
close(fd[1]);
close(fd[0]);
execvp(echo[0], echo);
printf("Error in execvp1\n");
}else{
wait(&status);
pid = fork();
if (pid == 0){
dup2(fd[0], 0);
dup2(fd1[1], 1);
dup2(fd1[1], 2);
close(fd[1]);
close(fd[0]);
close(fd1[1]);
close(fd1[0]);
execvp(sed1[0],sed1);
printf("Error in execvp2\n");
}else{
wait(&status);
dup2(fd1[0],0);
dup2(1,2);
//dup2(1,1);
close(fd1[1]);
close(fd1[0]);
execvp(sed2[0],sed2);
printf("Error in execvp3\n");
}
}
if(pid!=0)
wait(&status);
close(fd[0]);
close(fd[1]);
close(fd1[1]);
close(fd1[0]);
}
I can imagine 2 possibilities... dup2 is blocking or i need to create more process because it end process on call, but this sounds not right after a quick read on his man page... what could it be?
General Problem
You aren't closing enough file descriptors in the various processes.
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD or F_DUPFD_CLOEXEC.
If the parent process will not communicate with any of its children via
the pipe, it must ensure that it closes both ends of the pipe early
enough (before waiting, for example) so that its children can receive
EOF indications on read (or get SIGPIPE signals or write errors on
write), rather than blocking indefinitely.
Even if the parent uses the pipe without using dup2(), it should
normally close at least one end of the pipe — it is extremely rare for
a program to read and write on both ends of a single pipe.
Note that the O_CLOEXEC option to
open(),
and the FD_CLOEXEC and F_DUPFD_CLOEXEC options to fcntl() can also factor
into this discussion.
If you use
posix_spawn()
and its extensive family of support functions (21 functions in total),
you will need to review how to close file descriptors in the spawned process
(posix_spawn_file_actions_addclose(),
etc.).
Note that using dup2(a, b) is safer than using close(b); dup(a);
for a variety of reasons.
One is that if you want to force the file descriptor to a larger than
usual number, dup2() is the only sensible way to do that.
Another is that if a is the same as b (e.g. both 0), then dup2()
handles it correctly (it doesn't close b before duplicating a)
whereas the separate close() and dup() fails horribly.
This is an unlikely, but not impossible, circumstance.
Specific Issues
You aren't closing enough file descriptors for safety.
Your regexes are dubious.
You should not make processes in a pipeline wait for each other.
Pet peeve: I prefer to use fd1 and fd2 when I have two closely related variables like the pairs of pipe file descriptors; I find fd and fd1 and the like silly. You may, however, choose to ignore this.
Working Code
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
static void dump_argv(char **argv)
{
printf("%d:\n", getpid());
while (*argv != NULL)
{
printf("%d: <<%s>>\n", getpid(), *argv++);
}
}
static void sendbc(char *str)
{
int fd1[2];
int fd2[2];
int pid;
char *echo[] = {"echo", str, NULL};
char *sed1[] = {"sed", "s/[^:]*[;]//", NULL};
char *sed2[] = {"sed", "s/[^:]*[.]//", NULL};
if (pipe(fd1) < 0)
exit(100);
if (pipe(fd2) < 0)
exit(101);
printf("%d: at work\n", getpid());
pid = fork();
if (pid < 0)
exit(102);
else if (pid == 0)
{
printf("%d: child 1 - echo\n", getpid());
dump_argv(echo);
dup2(fd1[1], 1);
close(fd1[1]);
close(fd1[0]);
close(fd2[0]);
close(fd2[1]);
execvp(echo[0], echo);
fprintf(stderr, "Error in execvp1\n");
exit(103);
}
else
{
printf("%d: parent - before second fork\n", getpid());
pid = fork();
if (pid == 0)
{
printf("%d: child 2 - sed 1\n", getpid());
dump_argv(sed1);
dup2(fd1[0], 0);
dup2(fd2[1], 1);
close(fd1[1]);
close(fd1[0]);
close(fd2[1]);
close(fd2[0]);
execvp(sed1[0], sed1);
fprintf(stderr, "Error in execvp2\n");
exit(104);
}
else
{
printf("%d: parent - sed 2\n", getpid());
dump_argv(sed1);
dup2(fd2[0], 0);
close(fd1[1]);
close(fd1[0]);
close(fd2[1]);
close(fd2[0]);
execvp(sed2[0], sed2);
fprintf(stderr, "Error in execvp3\n");
exit(105);
}
}
fprintf(stderr, "Reached unexpectedly\n");
exit(106);
}
int main(void)
{
char message[] =
"This is the first line\n"
"and this is the second - with a semicolon ; here before a :\n"
"and the third line has a colon : before the semicolon ;\n"
"but the fourth line has a dot . before the colon\n"
"whereas the fifth line has a colon : before the dot .\n"
;
sendbc(message);
return 0;
}
Example output
$ ./pipe29
74829: at work
74829: parent - before second fork
74829: parent - sed 2
74829:
74829: <<sed>>
74829: <<s/[^:]*[;]//>>
74830: child 1 - echo
74830:
74830: <<echo>>
74830: <<This is the first line
and this is the second - with a semicolon ; here before a :
and the third line has a colon : before the semicolon ;
but the fourth line has a dot . before the colon
whereas the fifth line has a colon : before the dot .
>>
74831: child 2 - sed 1
74831:
74831: <<sed>>
74831: <<s/[^:]*[;]//>>
This is the first line
here before a :
and the third line has a colon :
before the colon
whereas the fifth line has a colon :
$
Apart from the diagnostic printing, the primary differences are that this code rigorously closes all the unused ends of the pipes and it contains no calls to wait() or its relatives — they are not needed and in general are harmful when they block concurrent execution of the processes in the pipeline.
I've been trying to write a really simple program in which the parent process passes 100 lines to a child process through a pipe. The child should then use the generated lines and execute the command line program more over those lines.
However, when I try to run the program, it just freezes. I was careful to close all descriptors not being used by both processes but I don't really understand what may be causing it.
Code:
int main(void){
int fd[2];
if (pipe(fd) == -1){
perror("Error creating pipe");
return 1;
}
dup2(fd[1], STDOUT_FILENO);
int i;
for (i = 1; i <= 100; i++){
printf("Line %d\n", i);
}
close(fd[1]);
pid_t pid = fork();
if(pid == 0) {
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
execlp("more", "more",(char*) NULL);
fprintf(stderr, "Failed to execute 'more'\n");
exit(1);
}
wait(NULL);
return 0;
}
I was careful to close all descriptors not being used by both processes
Not really.
dup2(fd[1], STDOUT_FILENO);
Here you make stdout a copy of fd[1].
close(fd[1]);
Here you close fd[1], but stdout is still open.
Then you fork. At this point both processes have access to the write end of the pipe via stdout.
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
In the child process you copy fd[0] to stdin and close fd[0].
Then, when you exec more, it still has access to both ends of the pipe (via stdin / stdout).
At the same time your parent process has access to both ends of the pipe (via fd[0] / stdout).
In effect you've closed nothing.
There's a second issue: Your parent process writes to stdout, which is bound to the write end of the pipe, without anyone reading it. Depending on how much you write, whether stdout is line buffered or block buffered, how big the stdout buffer is, and how much your pipe itself can store, this itself can deadlock. If the pipe runs full and there's no one around to read from it, printf will just block.
To fix this, don't dup2 in the parent process and don't write to the pipe before the child process has started.
int main(void){
int fd[2];
if (pipe(fd) == -1){
perror("Error creating pipe");
return 1;
}
pid_t pid = fork();
if (pid == -1) {
perror("Error spawning process");
return 2;
}
if (pid == 0) {
close(fd[1]); /* close write end of the pipe in the child */
dup2(fd[0], STDIN_FILENO);
close(fd[0]);
execlp("more", "more", (char*)NULL);
fprintf(stderr, "Failed to execute 'more'\n");
exit(1);
}
close(fd[0]); /* close read end of the pipe in the parent */
FILE *fp = fdopen(fd[1], "w");
if (!fp) {
perror("Error opening file handle");
return 3;
}
for (int i = 1; i <= 100; i++){
fprintf(fp, "Line %d\n", i);
}
fclose(fp); /* flush and close write end of the pipe in the parent */
wait(NULL);
return 0;
}
I've been trying to write a program that will send and receive commands to a bash shell (/bin/sh). Like a wrapper program around a bash shell. So, I could write to stdin "cd ~/Desktop", then write again "ls" and I will receive a listing of the files on the desktop. I can't get it working though. On the second write command in this code, it will echo back whatever I wrote to stdin. I've also tried using popen() but that only provides output, not allowing me to write to stdin. Could someone please help solve this problem? Thanks
void main()
{
// Create a pipe and fork
//
int fd[2];
int p = pipe(fd);
pid_t pid = fork();
if (pid > 0)
{
// Read from the pipe and output the result
//
//close(fd[1]);
char buf[1024] = { 0 };
read(fd[0], buf, sizeof(buf));
printf("1 - %s\n", buf);
write (fd[1], "ifconfig", strlen ("ifconfig") );
// problem is here, read is returning echo'd bytes from write()
read(fd[0], buf, sizeof(buf));
printf("2 - %s\n", buf);
// Wait for child to terminate
int status;
wait(&status);
}
else if (pid == 0)
{
// Redirect stdout and stderr to the pipe and execute the shell
// command
//
dup2(fd[0], STDIN_FILENO);
dup2(fd[1], STDOUT_FILENO);
dup2(fd[1], STDERR_FILENO);
//close(fd[0]);
execl("/bin/sh", "exec sh", "-c", "ls", (char*) NULL );
}
}
EDIT - Updated code per 1st answer, now there is no output from the 2nd read() call
void main()
{
// Create a pipe and fork
//
int fd[2];
int ChildToParent[2], ParentToChild[2];
pipe (ParentToChild);
pipe (ChildToParent);
pid_t pid = fork();
if (pid > 0)
{
// In parent process
// Read the output of the child from child_to_parent[0]
// We don't need child_to_parent[1] so close it
close(ChildToParent[1]);
// Write output to the child using parent_to_child[1]
// We don't need parent_to_child[0] so close it
close(ParentToChild[0]);
// Read from and write to the child process...
char buf[1024] = { 0 };
read(ChildToParent[0], buf, sizeof(buf));
printf("1 - %s\n", buf);
write(ParentToChild[1], "whoami", strlen ("whoami") );
memset (buf, 0, 1024);
// this call to read returns nothing
read(ChildToParent[0], buf, sizeof(buf));
printf("2 - %s\n", buf);
}
else if (pid == 0)
{
// Redirect stdout and stderr to the pipe and execute the shell
// command
//
// child_to_parent[1] is were we write output, it's the
// new standard output, child_to_parent[0] can be closed
dup2 (ChildToParent[1], STDOUT_FILENO);
close(ChildToParent[0]);
// parent_to_child[0] is where we read input from, it's the
// new standard input, parent_to_child[1] can be closed
dup2 (ParentToChild[0], STDIN_FILENO);
close(ParentToChild[1]);
//close(fd[0]);
execl("/bin/sh", "exec sh", "-c", "ls", (char*) NULL );
}
}
Remember that pipes are a one-way communication stream. You can't use it for two-way communication between two processes. For that you need two pipes, one in each direction.
Perhaps something like this simple example:
// Pipe for the child process to write to the parent process
int child_to_parent[2];
// Pipe for the parent process to write to the child process
int parent_to_child[2];
// Create the TWO pipes
pipe(child_to_parent);
pipe(parent_to_child);
pid_t pid = fork();
if (pid > 0)
{
// In parent process
// Read the output of the child from child_to_parent[0]
// We don't need child_to_parent[1] so close it
close(child_to_parent[1]);
// Write output to the child using parent_to_child[1]
// We don't need parent_to_child[0] so close it
close(parent_to_child[0]);
// Read from and write to the child process...
}
else if (pid == 0)
{
// In child process
// child_to_parent[1] is were we write output, it's the
// new standard output, child_to_parent[0] can be closed
dup2(child_to_parent[1], STDOUT_FILENO);
close(child_to_parent[0]);
// parent_to_child[0] is where we read input from, it's the
// new standard input, parent_to_child[1] can be closed
dup2(parent_to_child[0], STDIN_FILENO);
close(parent_to_child[1]);
// Do whatever the child is supposed to do
}
I have been making simple shell which performs pipe.
Here is some code for operating pipe syntax.
int fd[2];
int stdin_copy;
int stdout_copy;
int status;
char * msg;
if (pipe(fd) == -1) {
perror("pipe");
exit(1);
}
// fd[0] : process read from fd[0]
// fd[1] : process write to fd[1]
if (execok(pr_words) == 0) { /* is it executable? */
status = fork(); /* yes; create a new process */
if (status == -1) { /* verify fork succeeded */
perror("fork");
exit(1);
} else if (status == 0) { /* in the child process... */
stdout_copy = dup(1);
close(1); // close standard output
dup(fd[1]);
close(fd[0]);
close(fd[1]); // close and fd[1] will be stdout
pr_words[l_nwds] = NULL; /* mark end of argument array */
status = execve(path, pr_words, environ); /* try to execute it */
perror("execve"); /* we only get here if */
exit(0); /* execve failed... */
}
/*------------------------------------------------*/
/* The parent process (the shell) continues here. */
/*------------------------------------------------*/
else if (status > 0) { // in the parent process....
wait( & status); /* wait for process to end */
if (execok(af_words) == 0) {
if (pipe(fd2) == -1) {
perror("pipe");
exit(1);
}
status = fork();
if (status == -1) {
perror("fork");
exit(1);
} else if (status == 0) { // in the child process...
stdin_copy = dup(0);
close(0);
dup(fd[0]);
close(fd[1]);
close(fd[0]);
read(fd[0], readbuffer, sizeof(readbuffer));
af_words[r_nwds] = NULL; /* mark end of argument array */
status = execve(path, af_words, environ); /* try to execute it */
} else if (status > 0) {
wait( & status);
msg = "over";
write(2, msg, strlen(msg));
close(fd[0]);
close(fd[1]);
dup2(stdin_copy, 0);
dup2(stdout_copy, 1);
close(stdin_copy);
close(stdout_copy);
printf("%s", "hi");
}
} else {
/*----------------------------------------------------------*/
/* Command cannot be executed. Display appropriate message. */
/*----------------------------------------------------------*/
msg = "*** ERROR: '";
write(2, msg, strlen(msg));
write(2, af_words[0], strlen(af_words[0]));
msg = "' cannot be executed.\n";
write(2, msg, strlen(msg));
}
}
} else {
/*----------------------------------------------------------*/
/* Command cannot be executed. Display appropriate message. */
/*----------------------------------------------------------*/
msg = "*** ERROR: '";
write(2, msg, strlen(msg));
write(2, pr_words[0], strlen(pr_words[0]));
msg = "' cannot be executed.\n";
write(2, msg, strlen(msg));
}
pr_words and af_words is two-dimensional pointer containing command, right-side and left-side of pipe. (ex. ls | cat -> pr_words = "ls\0" , af_words = "cat\0")
And, first I make child process using fork() and register fd[1] for standard output. (and also save stdin file descriptor before closing stdin) And after execute left side of command, make other child process for handling right side of command.
Similarly, I saved stdout file descriptor before close stdout and made fd[0] standard input. By using input from first outcome of execve function, I thought every outcome would be saved in fd[1]. (Because this was currently registered as std output).
And, finally, restore pipe input and output to standard output. (I don't want to use dup2 but I have no choice because of my lack of knowledge )
However, in execution of this code, after I enter the 'ls | cat', there is no output. Furthermore, I set every entry of terminal will print '#'. (which means that '# ls' or '# cat' ...) But, after enter above pipe command, that program even does not print '#'.
I guess input and output stream of this program are completely twisted after dealing with pipe command.
How can I fix it? I mean, I want save outcome of first execve into fd[1] and after using this fd[1] for performing second execve, make final outcome will be printed through stdout file description.
I see a few issues with your code at least:
First off, you shouldn't wait() on the first process before starting the second one. A pipe only has a few KB of buffer in it, after which your shell will hang if the first child process tries to continue to write there. You need to start both children before wait()ing for each of them. Just move the first wait(&status) call down next to the other one. You'll probably want to use waitpid or something later so you know which one finished first and which status goes to which, but you can address that once you get the basics working.
Secondly, all variables and file descriptor mappings in your program are copied when you fork(). Therefore, you don't need to save stdin or stdout in either child process, because none of the changes you make in the child processes will affect the parent. Furthermore, because you only initialize stdin_copy and stdout_copy in the child processes, the versions of those variables you use in the parent process after the second fork() are uninitialized. This is what's causing the parent shell's I/O to be messed up after executing this code. You don't actually need to do anything in the parent after forking the second time to maintain the original stdin and stdout there -- you never change them in that process before that point. You probably want to remove all of this from the post-fork parent code:
close(fd[0]);
close(fd[1]);
dup2(stdin_copy, 0);
dup2(stdout_copy, 1);
close(stdin_copy);
close(stdout_copy);
Thirdly, why are you reading from the pipe before calling execve() in the second child? That's just going to strip data out of the pipe that your exec'd child will never see. That's probably what's causing the pipe itself to appear not to work. You probably want to remove this:
read(fd[0], readbuffer, sizeof(readbuffer));
Lastly, this line probably needs to go before the execok() call (and similarly for the other similar one):
pr_words[l_nwds] = NULL; /* mark end of argument array */
The skeleton of the code should look about like this, leaving off error handling and execok checks, and demonstrating the use of waitpid() if you want to know which status code is for which child:
int child_pid[2];
child_pid[0] = fork();
if (child_pid[0] == 0) {
// first child, close stdout and replace with pipe, then exec
} else {
child_pid[1] = fork();
if (child_pid[1] == 0) {
// second child, close stdin and replace with pipe, then exec
} else {
// parent, and now we have the pids of the children
waitpid(child_pid[0], &status, 0); // wait for first child
waitpid(child_pid[1], &status, 0); // wait for second child
// *do not* mess with stdin/stdout, they are okay here
}
}
I am trying to run ls|wc using execvp. So I create a pipe and then fork to create a child. I close the appropriate(read./write) end in parent/child and then map the other end to stdout/stdin. Then I run the ls in parent using execvp and wc in child. When I run the program it says
wc:standard input:bad file descriptor.
0 0 0
wc: -:Bad file descriptor
Here is my code:
int main()
{
//int nbBytes = 0; //stream length
int pfd_1[2]; //file descriptor
//char buffer[MAX_FILE_LENGTH];
char* arg[MAX_FILE_LENGTH];
pid_t processPid;
//Create a pipe
if(pipe(pfd_1) == -1)
{
printf("Error in creating pipe");
return 0;
}
//Create a child
processPid = fork();
if(processPid == -1)
{
printf("Erro in fork");
exit(1);
}
else if(processPid == 0) //Child
{
//redirect read end file descriptor to standard input
dup2(pfd_1[0],0);
//Close the write end
if(close(pfd_1[1] == -1))
{
printf("Error in closing the write end file descriptor");
exit(1);
}
arg[0] = "wc";
//arg[1] = "-l";
arg[1] = '\0';
if(execvp(arg[0],arg) == -1)
{
printf("Error in executing ls");
}
}
else //Parent
{
//redirect standard output to the file descriptor
dup2(pfd_1[1],1);
//Close the read end
if(close(pfd_1[0] == -1))
{
printf("Error in closing the read end from parent");
exit(1);
}
//Command
arg[0] = "ls";
arg[1] = "/proc/1/status";
arg[2] = '\0';
if(execvp(arg[0],arg) == -1)
{
printf("Error in executing ls");
}
}
}
Any idea what might be wrong? Why would it consider standard input as bad file descriptor? My understanding was since the stdin and read end file descriptor are aliases so the wc -l would read whatever the output is from the parent process. Do I need to do scanf to read from the stdin?
The problem is in this line:
if(close(pfd_1[1] == -1))
You are closing the result of pfd_1[1] == -1, which is by necessity equal to 0 (as they will never be equal). The correct line would probably be:
if (close(pfd_1[1]) == -1)
Note that you do this again later in attempting to close the read end in the parent process.
If you're going to fork children, you have to call wait() in the parent process in order to avoid "zombie" child processes. So you don't want to overlay the parent process that did the original process forking with another executable via exec.
One quick way to setup a series of pipes in the way you want would be to fork a child for each executable you want to run, and read that data back into a buffer in the parent. Then feed that data from the first child into a new child process that the parent forks off. So each child is fed data from the parent, processes the data, and writes the data back to the parent process, which stores the transformed data in a buffer. That buffer is then fed to the next child, etc., etc. The final results of the data in the buffer are the final output of the pipe.
Here's a little pseudo-code:
//allocate buffer
unsigned char buffer[SIZE];
for (each executable to run in pipeline)
{
pipes[2];
pipe(pipes);
pid_t pid = fork();
if (pid == 0)
{
//setup the pipe in the child process
//call exec
}
else
{
//setup the pipe in the parent process
if (child executable is not the first in the pipeline)
{
//write contents of buffer to child process
}
//read from the pipe until the child exits
//store the results in buffer
//call wait, and maybe also check the return value to make sure the
//child returned successfully
wait(NULL);
//clean up the pipe
}
}