EDIT: THE QUESTION IS ANSWERED IN COMMENTS
So, i'm studying pipes. Long story short, i have two programs:
first program creates a pipe and two forks: first fork closes read descriptor and writes some stuff to write one (then closes it), second fork closes write one, dup2 on read side of pipe to standard input (end closes read side itself) and execl second program, giving a size of a text the first fork writes as an argument; the parent closes both pipe sides and waitpids for child that wasexecld (second one).
second program just reads from its standard input (pipe side) the stuff and writes it out to standard output, then closes pipe side just in case.
In such a setup everything works as I intended, but when I delete waitpid in first program (or just wait for the first child that writes instead of the second one), the second one behaves weirdly - it executes till the end, passing through all the IO (that is, the printf before exit got executed), and then doesn't give me the prompt back. That is, the terminal looks like the program awaits for an input from standard input. If i execute the first program without execl, then everything works fine, If I execute just the second one with one argument, then it waits only until input is provided to standard input (as it should as it is not a part of a pipe in this case).
As i know, when parent terminates, the child is "inherited" by init and got waited. But even if it wasn't, that is, even if it remained as a zombie, then it still would be weird - why i cannot get my prompt back until i wait explicitly?
The code is below (of the setup that works correctly):
First program
/* headers */
int main(void)
{
int fildes[2];
pid_t p1, p2;
int status;
char mess[] = "written from execved program!\n";
int buf = strlen(mess);
if(pipe(fildes) == -1) {
perror("pipe in main");
exit(EXIT_FAILURE);
}
p1 = fork();
if(p1 == -1) {
perror("fork p1 in main");
exit(EXIT_FAILURE);
}
else if (p1 == 0) {
printf("Child 1!\n");
close(fildes[0]);
write(fildes[1], mess, buf);
close(fildes[1]);
printf("Before exit in child 1!\n");
exit(EXIT_SUCCESS);
}
p2 = fork();
if(p2 == -1) {
perror("fork p2 in main");
exit(EXIT_FAILURE);
}
else if (p2 == 0) {
printf("Child 2!\n");
dup2(fildes[0], 0);
close(fildes[0]);
close(fildes[1]);
char s_buf[30];
sprintf(s_buf, "%d", buf);
execl("./pipe2slave", "pipe2slave", s_buf, (char *) 0);
perror("execl have returned");
exit(EXIT_FAILURE);
}
close(fildes[0]);
close(fildes[1]);
/*
below if I wait for, say, p1, or don't wait it all,
the weird behavior described in my question happens
*/
if(waitpid(p2, &status, 0) == -1) {
perror("waitpid in main");
exit(EXIT_FAILURE);
}
if(WIFEXITED(status))
printf("pipe2slave exit status is %d\n", WEXITSTATUS(status));
printf("End of main in pipe2!\n");
exit(EXIT_SUCCESS);
}
Second program
/* headers */
int main(int argc, char **argv)
{
if (argc != 2) {
perror("pipe2slave - not enough args");
exit(EXIT_FAILURE);
}
printf("program name is %s\n", argv[0]);
int buf = atoi(argv[1]);
printf("%d\n", buf);
char mess_in[buf];
read(0, mess_in, buf);
write(1, mess_in, buf);
fsync(1);
close(0);
printf("end of slave!\n");
exit(EXIT_SUCCESS);
}
Thank you in advance!
Related
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
int main(void){
//Variables, p[2] for each end of the pipe. nbytes to read pipe return value SUCCESS or FAILURE. pid_t to hold pid of fork process.
// buffer to hold response from the child process.
int p[2], nbytes;
pid_t childpid;
char string[] = "Hello, World!\n";
char buffer[80];
//Declaration of pipe
pipe(p);
//Error handling.
if(((childpid = fork()) == -1) || (pipe(p) == -1))
{
perror("fork");
exit(1);
}
//Child process sends message to paprent.
if(childpid == 0)
{
/* Child process closes up input side of pipe */
close(p[0]);
/* Send "string" through the output side of pipe */
write(p[1], string, (strlen(string)+1));
exit(0);
}
else
{
/* Parent process closes up output side of pipe */
close(p[1]);
/* Read in a string from the pipe */
nbytes = read(p[0], buffer, sizeof(buffer));
printf("Received string: %s", buffer);
}
return(0);
}
Output > Received string: #�=zJ
The point of the exercise is to have a child process send a message through a pipe to the parent process and the parent returns the result. This exact code worked the first time I ran it, but then when I tried to run it a second time it started to return seemingly random characters each time. I tried to copy my buffer to another variable but then it was empty. Is the pipe actually not function the way I think it is? What am I doing wrong?
You first create a pipe with pipe(p); and then you create another with ... || (pipe(p) == -1)) Is that deliberate?
2nd Pipe was causing an issue.
You have:
pipe(p);
//Error handling.
if(((childpid = fork()) == -1) || (pipe(p) == -1))
{
perror("fork");
exit(1);
}
This creates two pipes — one in the line pipe(p); and the second in the condition if(((childpid = fork()) == -1) || (pipe(p) == -1)). This is wasteful at best. Moreover, the second pipe is after the fork(), so the parent and child processes don't access the same pipe any more — you overwrote the one created before the fork() which they do share. Test the result of pipe() before calling fork() and remove the extra condition in the if test:
if (pipe(p) != 0)
{
perror("pipe");
exit(1);
}
if ((childpid = fork()) < 0)
{
perror("fork");
exit(1);
}
Get used to testing for errors and writing appropriate code to handle them. It will be a major part of your life as a C programmer.
Later on in the code, you have:
{
/* Parent process closes up output side of pipe */
close(p[1]);
/* Read in a string from the pipe */
nbytes = read(p[0], buffer, sizeof(buffer));
printf("Received string: %s", buffer);
}
You need to heed the value of nbytes. Since it is an int, you could use:
printf("Received %d bytes: [%.*s]\n", nbytes, nbytes, buffer);
This limits the output to what was read, and reports 0 if that's what it gets. I suppose you should also check for -1 in nbytes before using it in the printf() statement:
if (nbytes < 0)
{
fprintf(stderr, "failed to read from pipe descriptor %d\n", p[0]);
// Or perror("read");
// Should you exit here with a non-zero status?
}
else
printf("Received %d bytes: [%.*s]\n", nbytes, nbytes, buffer);
Note: errors are reported on stderr; perror() does that automatically.
The problem is that you create two pipes when you really only need to check the first for errors:
// Declaration of pipe
if(pipe(p) == -1) { // check for error here
perror("pipe");
exit(1);
}
// Error handling.
if((childpid = fork()) == -1) { // and don't create another pipe here
perror("fork");
exit(1);
}
You should also check the return values from write and read. They may not write or read the full string in one go.
I'm writing a function that echo an input to a sed and then another sed. I thinck i used all my wait signal in the right way but the last print i can get is before the call to dup2() in my first child process in the echo.
void sendbc (char * str_ ) {
int fd[2];
int fd1[2];
int pid,pid1;
char* echo[] = {"echo", str_,NULL};
char* sed1[] = {"sed","s/[^:]*;"" " "//",NULL};
char* sed2[] = {"sed","s/[^:]*."" " "//",NULL};
int status,er;
FILE *f;
if(pipe(fd) < 0){
exit(100);
}
if(pipe(fd1) < 0){
exit(100);
}
pid = fork();
if (pid == 0) {
dup2(fd[1], 1) //last command before blocking
close(fd[1]);
close(fd[0]);
execvp(echo[0], echo);
printf("Error in execvp1\n");
}else{
wait(&status);
pid = fork();
if (pid == 0){
dup2(fd[0], 0);
dup2(fd1[1], 1);
dup2(fd1[1], 2);
close(fd[1]);
close(fd[0]);
close(fd1[1]);
close(fd1[0]);
execvp(sed1[0],sed1);
printf("Error in execvp2\n");
}else{
wait(&status);
dup2(fd1[0],0);
dup2(1,2);
//dup2(1,1);
close(fd1[1]);
close(fd1[0]);
execvp(sed2[0],sed2);
printf("Error in execvp3\n");
}
}
if(pid!=0)
wait(&status);
close(fd[0]);
close(fd[1]);
close(fd1[1]);
close(fd1[0]);
}
I can imagine 2 possibilities... dup2 is blocking or i need to create more process because it end process on call, but this sounds not right after a quick read on his man page... what could it be?
General Problem
You aren't closing enough file descriptors in the various processes.
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD or F_DUPFD_CLOEXEC.
If the parent process will not communicate with any of its children via
the pipe, it must ensure that it closes both ends of the pipe early
enough (before waiting, for example) so that its children can receive
EOF indications on read (or get SIGPIPE signals or write errors on
write), rather than blocking indefinitely.
Even if the parent uses the pipe without using dup2(), it should
normally close at least one end of the pipe — it is extremely rare for
a program to read and write on both ends of a single pipe.
Note that the O_CLOEXEC option to
open(),
and the FD_CLOEXEC and F_DUPFD_CLOEXEC options to fcntl() can also factor
into this discussion.
If you use
posix_spawn()
and its extensive family of support functions (21 functions in total),
you will need to review how to close file descriptors in the spawned process
(posix_spawn_file_actions_addclose(),
etc.).
Note that using dup2(a, b) is safer than using close(b); dup(a);
for a variety of reasons.
One is that if you want to force the file descriptor to a larger than
usual number, dup2() is the only sensible way to do that.
Another is that if a is the same as b (e.g. both 0), then dup2()
handles it correctly (it doesn't close b before duplicating a)
whereas the separate close() and dup() fails horribly.
This is an unlikely, but not impossible, circumstance.
Specific Issues
You aren't closing enough file descriptors for safety.
Your regexes are dubious.
You should not make processes in a pipeline wait for each other.
Pet peeve: I prefer to use fd1 and fd2 when I have two closely related variables like the pairs of pipe file descriptors; I find fd and fd1 and the like silly. You may, however, choose to ignore this.
Working Code
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
static void dump_argv(char **argv)
{
printf("%d:\n", getpid());
while (*argv != NULL)
{
printf("%d: <<%s>>\n", getpid(), *argv++);
}
}
static void sendbc(char *str)
{
int fd1[2];
int fd2[2];
int pid;
char *echo[] = {"echo", str, NULL};
char *sed1[] = {"sed", "s/[^:]*[;]//", NULL};
char *sed2[] = {"sed", "s/[^:]*[.]//", NULL};
if (pipe(fd1) < 0)
exit(100);
if (pipe(fd2) < 0)
exit(101);
printf("%d: at work\n", getpid());
pid = fork();
if (pid < 0)
exit(102);
else if (pid == 0)
{
printf("%d: child 1 - echo\n", getpid());
dump_argv(echo);
dup2(fd1[1], 1);
close(fd1[1]);
close(fd1[0]);
close(fd2[0]);
close(fd2[1]);
execvp(echo[0], echo);
fprintf(stderr, "Error in execvp1\n");
exit(103);
}
else
{
printf("%d: parent - before second fork\n", getpid());
pid = fork();
if (pid == 0)
{
printf("%d: child 2 - sed 1\n", getpid());
dump_argv(sed1);
dup2(fd1[0], 0);
dup2(fd2[1], 1);
close(fd1[1]);
close(fd1[0]);
close(fd2[1]);
close(fd2[0]);
execvp(sed1[0], sed1);
fprintf(stderr, "Error in execvp2\n");
exit(104);
}
else
{
printf("%d: parent - sed 2\n", getpid());
dump_argv(sed1);
dup2(fd2[0], 0);
close(fd1[1]);
close(fd1[0]);
close(fd2[1]);
close(fd2[0]);
execvp(sed2[0], sed2);
fprintf(stderr, "Error in execvp3\n");
exit(105);
}
}
fprintf(stderr, "Reached unexpectedly\n");
exit(106);
}
int main(void)
{
char message[] =
"This is the first line\n"
"and this is the second - with a semicolon ; here before a :\n"
"and the third line has a colon : before the semicolon ;\n"
"but the fourth line has a dot . before the colon\n"
"whereas the fifth line has a colon : before the dot .\n"
;
sendbc(message);
return 0;
}
Example output
$ ./pipe29
74829: at work
74829: parent - before second fork
74829: parent - sed 2
74829:
74829: <<sed>>
74829: <<s/[^:]*[;]//>>
74830: child 1 - echo
74830:
74830: <<echo>>
74830: <<This is the first line
and this is the second - with a semicolon ; here before a :
and the third line has a colon : before the semicolon ;
but the fourth line has a dot . before the colon
whereas the fifth line has a colon : before the dot .
>>
74831: child 2 - sed 1
74831:
74831: <<sed>>
74831: <<s/[^:]*[;]//>>
This is the first line
here before a :
and the third line has a colon :
before the colon
whereas the fifth line has a colon :
$
Apart from the diagnostic printing, the primary differences are that this code rigorously closes all the unused ends of the pipes and it contains no calls to wait() or its relatives — they are not needed and in general are harmful when they block concurrent execution of the processes in the pipeline.
To learn how Pipe IPC mechanism works, I wrote a simple program that creates two child processes which share data using a pipe. The first child process has to read data from a file and pass it to the pipe.
Afterwards, the second child process has to read it, convert it to uppercase and write it to another file. The read system call in the second child process returns -1 when reading from the pipe. Also when I execute the program, in some cases printf in the first child does not print anything and in other cases printf in the second child does not print, too. Could you please point the mistakes in the program which are causing the problems?
int main(int args[], char * argv[]) {
int fd[2];
long length;
char buff1[250];
char buff2[250];
FILE * fptr1;
FILE * fptr2;
pid_t A, B;
pipe(fd);
A = fork();
if (A == -1) {
printf("error in fork of A\n");
exit(1);
}
if (A == 0) {
fptr1 = fopen(argv[1], "r"); // program receives file names as argument
if (fptr1 == NULL) {
printf("Erro in file open1\n");
exit(1);
}
fseek(fptr1, 0 L, SEEK_END);
length = ftell(fptr1);
fseek(fptr1, 0 L, SEEK_SET);
close(fd[0]);
fread(buff1, length, 1, fptr1);
buff1[length] = '\0';
printf("buff1 = %s", buff1);
write(fd[1], buff1, length);
fclose(fptr1);
exit(0);
} else {
B = fork();
if (B == -1) {
printf("Error in forking child B");
exit(1);
}
if (B == 0) {
fptr2 = fopen(argv[2], "w");
if (fptr2 == NULL) {
printf("Error in file open2\n");
exit(1);
}
close(fd[1]);
int n = read(fd[0], buff2, length);
printf("n = %d\n", n);
upper_string(buff2); // converts characters to uppecase
fwrite(buff2, 1, length, fptr2);
fclose(fptr2);
}
}
return 0;
}
There are few things to take into account here. First thing i would like to point is that you do not need to use two fork() calls. In that case you have three processes working in parallel (parent process and two child process, one per each fork() call).
One important point to take into account when you work with processes working in parallel is synchronism. In your code you are creating two processes. Parent process does not wait for any of its child, so it finishes its execution, and if child processes have not finished, they will become child of init process. But appart from that, you have the typical producer consumer problem. One of your child produce something and the other consume it, but how they work in parallel, consumer need to know that the product is ready to be consumed. So, in this case, i think the easiest way to do this job is to use just one fork(), so child become the producer and the parent process (the consumer) wait until its child finish the job.
So I have this problem, I need to create 3 processes (each handle a different task). The first process sends information over to the second (the first waits for an acknowledgement from the second). The second then sends information to the third (the second waits for an acknowledgement from the third). Then the third processes the final information... This process is supposed to loop over and over until process one analyzes an entire text file. So far, I tried writing the communication between the 3 processes with pipes. I'm not sure how I send an acknowledgment from process 2 to process 1 and process 3 to process 2. I'm also not entirely sure how to loop it. Thanks!
I have to use a stop and wait protocol... I'm not sure how that is done.
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
int c = 0, t = 0;
int fd1[2], fd2[2];
char *theFile = "/Users/Desktop/thefile";
FILE *file = fopen (theFile, "r");
if (file == NULL) {
perror("FILE DOES NOT EXIST");
exit(1);
}
while (c == 0) {
int status;
char readbuffer[80];
char readbuffer2[80];
int tTemp = 0;
pipe(fd1);
pipe(fd2);
pid_t pid = fork();
if (pid < 0) {
perror("Pipe Error");
exit(1);
}
if (pid == 0) {
//Child 1
close(fd1[0]);
close(fd2[0]);
close(fd2[1]);
char line [80];
int c2 = 0;
file = fopen (theFile, "r");
while (fgets(line, sizeof(line), file) != NULL){
if (c2 == t) {
printf("Line: %s\n", line);
break;
}
c2++;
}
if (t != c2) {
c = 1;
} else {
write(fd1[1], line, (strlen(line)+1));
}
t++;
exit(1);
}
pid_t pid2 = fork();
if (pid2 < 0) {
perror("Pipe Error");
exit(1);
}
if (pid2 == 0) {
//Child 2
close(fd1[1]);
close(fd2[0]);
read(fd1[0], readbuffer, sizeof(readbuffer));
printf("2nd Child string: %s\n", readbuffer);
char string2[80] = "asdfasdf";
write(fd2[1], string2, (strlen(string2)+1));
exit(1);
}
pid_t pid3 = fork();
if (pid3 < 0) {
perror("Pipe Error");
exit(1);
}
if (pid3 == 0) {
//Child 3
close(fd2[1]);
close(fd1[0]);
close(fd1[1]);
read(fd2[0], readbuffer2, sizeof(readbuffer2));
exit(1);
}
waitpid(pid, &status, 0);
waitpid(pid2, &status, 0);
waitpid(pid3, &status, 0);
}
fclose(file);
return 0;
}
As I have come to understand the problem via the comments, you are asking about two distinct requirements:
implementing a "stop & wait" protocol between each pair of processes, and
using waitpid() to collect child processes that finish
The latter is pretty straightforward; what you already have seems fine. The former is what you seem mostly to be stuck on.
There are a couple of things here. One is a question of semantics: stop & wait, in the form we are discussing it, requires the recipient of a message to acknowledge to the sender that a message was successfully received before the sender proceeds. There is a significant difference between characterizing the acknowledgment that way, and characterizing it as a signal for the receiver of the acknowledgment to perform any particular action. What the receiver does in response to the acknowledgment is its own concern, not inherent in the acknowledgment itself.
As to communications, then, I recommend establishing two pipes between each pair of processes, one for communication in each direction. To wait for an acknowledgment, then, a process just performs a blocking read on the appropriate pipe.
As for looping, each process must loop separately, but the loops will all take about the same form:
Read the next line (process 1 gets lines from a file; the others get lines from pipes connected to the previous process)
Terminate if no line is available
Except for process 1, write a one-byte acknowledgment message on the pipe to the previous process
Process the line
Write the line to the appropriate terminus (process 3 writes lines to an unspecified terminus -- maybe stdout -- the others write lines to pipes connected to the next process).
Except for process 3, perform a blocking read to receive an acknowledgment from the next process.
Go to (1)
Be sure to check the result codes of all the functions that provide them. Since that's most library functions and syscalls it can get tedious, so I suggest a macro to help you out there.
For clarity and readability, I suggest writing the work of each of the three processes as a separate function. After fork() to create each process just handle file descriptor mangling as needed and call the appropriate function.
I'm creating a small program which contains three processes; a source process, a filter process and a sink process. The stdout of the source process is redirected to the stdin of the filter process, and the filter process' stdout is redirected to the sink process' stdin.
My problem is that no output is printed to stdout from the sink process. Can any of you see the problem in the following tiny snippet of code?
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char** argv)
{
// Pipes
// pipe1 is from source to filter, pipe2 is from filter to sink
int pipe1[2], pipe2[2];
// Create pipes
if (pipe(pipe1) < 0 || pipe(pipe2) < 0)
{
perror("Creating pipes failed!");
}
if (fork() == 0)
{
close(1);
dup(pipe1[1]);
close(pipe1[0]);
close(pipe2[0]);
close(pipe2[1]);
execlp("ls", "ls", NULL);
exit(0);
}
else
{
if (fork() == 0)
{
close(0);
dup(pipe1[0]);
close(pipe1[1]);
close(1);
dup(pipe2[1]);
close(pipe2[0]);
execlp("sort", "sort", NULL);
exit(0);
}
else
{
if (fork() == 0)
{
close(0);
dup(pipe2[0]);
execlp("more", "more", NULL);
exit(0);
}
}
}
wait(NULL);
printf("Done.\n");
return 0;
}
BR
Jacob
I think problem may be, wait will only wait for one process. And when the parent exits after first child returns, I suspect more command also decides to terminate, because it may get SIGHUP (speculation, not sure).
But, check for errors from on all system calls! Also for wait calls which succeeded, print why the child exited (was it signal or normal exit, and if it was normal exit, what was exit code).
Also note, perror does not exit, it only prints.
It is kind of pointless trying to see why some code fails, if it does not have error handling in it...
Some easy way to do pipes for your scenario:
char cmd[MAX_LEN];
sprintf(cmd, "%s | %s | %s", app1, app2, app3); //app123 holds app name + args
system(cmd);
if you want to capture the output of the last app, use popen:
FILE pPipe = popen(cmd, "rt"); /* same access flag as fopen()*/
while (NULL != fget(buf, buf_len, pPipe)) {
// do something with the read line in 'buf'
}