dup2() is blocking with child processes? C - c

I'm writing a function that echo an input to a sed and then another sed. I thinck i used all my wait signal in the right way but the last print i can get is before the call to dup2() in my first child process in the echo.
void sendbc (char * str_ ) {
int fd[2];
int fd1[2];
int pid,pid1;
char* echo[] = {"echo", str_,NULL};
char* sed1[] = {"sed","s/[^:]*;"" " "//",NULL};
char* sed2[] = {"sed","s/[^:]*."" " "//",NULL};
int status,er;
FILE *f;
if(pipe(fd) < 0){
exit(100);
}
if(pipe(fd1) < 0){
exit(100);
}
pid = fork();
if (pid == 0) {
dup2(fd[1], 1) //last command before blocking
close(fd[1]);
close(fd[0]);
execvp(echo[0], echo);
printf("Error in execvp1\n");
}else{
wait(&status);
pid = fork();
if (pid == 0){
dup2(fd[0], 0);
dup2(fd1[1], 1);
dup2(fd1[1], 2);
close(fd[1]);
close(fd[0]);
close(fd1[1]);
close(fd1[0]);
execvp(sed1[0],sed1);
printf("Error in execvp2\n");
}else{
wait(&status);
dup2(fd1[0],0);
dup2(1,2);
//dup2(1,1);
close(fd1[1]);
close(fd1[0]);
execvp(sed2[0],sed2);
printf("Error in execvp3\n");
}
}
if(pid!=0)
wait(&status);
close(fd[0]);
close(fd[1]);
close(fd1[1]);
close(fd1[0]);
}
I can imagine 2 possibilities... dup2 is blocking or i need to create more process because it end process on call, but this sounds not right after a quick read on his man page... what could it be?

General Problem
You aren't closing enough file descriptors in the various processes.
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD or F_DUPFD_CLOEXEC.
If the parent process will not communicate with any of its children via
the pipe, it must ensure that it closes both ends of the pipe early
enough (before waiting, for example) so that its children can receive
EOF indications on read (or get SIGPIPE signals or write errors on
write), rather than blocking indefinitely.
Even if the parent uses the pipe without using dup2(), it should
normally close at least one end of the pipe — it is extremely rare for
a program to read and write on both ends of a single pipe.
Note that the O_CLOEXEC option to
open(),
and the FD_CLOEXEC and F_DUPFD_CLOEXEC options to fcntl() can also factor
into this discussion.
If you use
posix_spawn()
and its extensive family of support functions (21 functions in total),
you will need to review how to close file descriptors in the spawned process
(posix_spawn_file_actions_addclose(),
etc.).
Note that using dup2(a, b) is safer than using close(b); dup(a);
for a variety of reasons.
One is that if you want to force the file descriptor to a larger than
usual number, dup2() is the only sensible way to do that.
Another is that if a is the same as b (e.g. both 0), then dup2()
handles it correctly (it doesn't close b before duplicating a)
whereas the separate close() and dup() fails horribly.
This is an unlikely, but not impossible, circumstance.
Specific Issues
You aren't closing enough file descriptors for safety.
Your regexes are dubious.
You should not make processes in a pipeline wait for each other.
Pet peeve: I prefer to use fd1 and fd2 when I have two closely related variables like the pairs of pipe file descriptors; I find fd and fd1 and the like silly. You may, however, choose to ignore this.
Working Code
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
static void dump_argv(char **argv)
{
printf("%d:\n", getpid());
while (*argv != NULL)
{
printf("%d: <<%s>>\n", getpid(), *argv++);
}
}
static void sendbc(char *str)
{
int fd1[2];
int fd2[2];
int pid;
char *echo[] = {"echo", str, NULL};
char *sed1[] = {"sed", "s/[^:]*[;]//", NULL};
char *sed2[] = {"sed", "s/[^:]*[.]//", NULL};
if (pipe(fd1) < 0)
exit(100);
if (pipe(fd2) < 0)
exit(101);
printf("%d: at work\n", getpid());
pid = fork();
if (pid < 0)
exit(102);
else if (pid == 0)
{
printf("%d: child 1 - echo\n", getpid());
dump_argv(echo);
dup2(fd1[1], 1);
close(fd1[1]);
close(fd1[0]);
close(fd2[0]);
close(fd2[1]);
execvp(echo[0], echo);
fprintf(stderr, "Error in execvp1\n");
exit(103);
}
else
{
printf("%d: parent - before second fork\n", getpid());
pid = fork();
if (pid == 0)
{
printf("%d: child 2 - sed 1\n", getpid());
dump_argv(sed1);
dup2(fd1[0], 0);
dup2(fd2[1], 1);
close(fd1[1]);
close(fd1[0]);
close(fd2[1]);
close(fd2[0]);
execvp(sed1[0], sed1);
fprintf(stderr, "Error in execvp2\n");
exit(104);
}
else
{
printf("%d: parent - sed 2\n", getpid());
dump_argv(sed1);
dup2(fd2[0], 0);
close(fd1[1]);
close(fd1[0]);
close(fd2[1]);
close(fd2[0]);
execvp(sed2[0], sed2);
fprintf(stderr, "Error in execvp3\n");
exit(105);
}
}
fprintf(stderr, "Reached unexpectedly\n");
exit(106);
}
int main(void)
{
char message[] =
"This is the first line\n"
"and this is the second - with a semicolon ; here before a :\n"
"and the third line has a colon : before the semicolon ;\n"
"but the fourth line has a dot . before the colon\n"
"whereas the fifth line has a colon : before the dot .\n"
;
sendbc(message);
return 0;
}
Example output
$ ./pipe29
74829: at work
74829: parent - before second fork
74829: parent - sed 2
74829:
74829: <<sed>>
74829: <<s/[^:]*[;]//>>
74830: child 1 - echo
74830:
74830: <<echo>>
74830: <<This is the first line
and this is the second - with a semicolon ; here before a :
and the third line has a colon : before the semicolon ;
but the fourth line has a dot . before the colon
whereas the fifth line has a colon : before the dot .
>>
74831: child 2 - sed 1
74831:
74831: <<sed>>
74831: <<s/[^:]*[;]//>>
This is the first line
here before a :
and the third line has a colon :
before the colon
whereas the fifth line has a colon :
$
Apart from the diagnostic printing, the primary differences are that this code rigorously closes all the unused ends of the pipes and it contains no calls to wait() or its relatives — they are not needed and in general are harmful when they block concurrent execution of the processes in the pipeline.

Related

Can't read from a pipe after dup2() and fork. C

I'm writing a code that echo a string and sed it two times. My output is correct, but when I try to place that string on an array it blocks on read and goes on with the other calls.
Here's the code:
#include <unistd.h>
#include <sys/types.h>
#include <dirent.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
char **sendout=NULL;
int send_i=0;
void sender2(char* str_) {
int fd[2];
int fd1[2];
int fd2[2];
int pid;
char* echo[] = {"echo", str_, NULL};
char* sed[] = {"sed", "regex1", NULL};
char* sed2[] = {"sed", "regex2", NULL};
int status;
if (pipe(fd) < 0) {
exit(100);
}
pid = fork();
if (pid == 0) {
close(fd[0]);
dup2(fd[1], 1);
close(fd[1]);
execvp(echo[0], echo);
printf("Error in execvp1\n");
}
if (pipe(fd1) < 0) {
exit(100);
}
pid = fork();
if (pid == 0) {
close(fd[1]);
close(fd1[0]);
dup2(fd[0], 0);
dup2(fd1[1], 1);
dup2(fd1[1], 2);
close(fd[0]);
close(fd1[1]);
execvp(sed2[0], sed2);
printf("Error in execvp2\n");
}
if (pipe(fd2) < 0) {
exit(100);
}
pid = fork();
if (pid == 0) {
close(fd1[1]);
close(fd2[0]);
dup2(fd1[0], 0);
dup2(fd2[1], 1);
dup2(fd2[1], 2);
close(fd2[1]);
close(fd1[0]);
execvp(sed[0], sed);
}
pid = fork();
if (pid == 0) {
close(fd2[1]);
char* line = NULL;
size_t len = 0;
ssize_t read_;
FILE* f_pipe;
f_pipe = fdopen(fd2[0], "r");
printf("1\n");
while ((read_ = getline(&line, &len, f_pipe)) != -1) {
printf("2\n");
sendout = realloc(sendout, sizeof(char*) * (send_i + 1));
sendout[send_i] = strdup(line);
send_i++;
printf("%s\n", line);
}
fclose(f_pipe);
close(fd2[0]);
return;
}
close(fd[1]);
close(fd[0]);
close(fd1[1]);
close(fd1[0]);
close(fd2[1]);
close(fd2[0]);
if (pid != 0) {
wait(&status);
}
}
int main() {
sender2("hello");
}
Like I said it all worked until the read. If I pass 3 string to the function the output is like:
1
1
1
If I don't dup to the last pipe it prints pretty well what I need, I also used return in the last fork because it's the only child process that isn't killed from execvp. But it doesn't even reach the first print. I even tried opening the pipe as a file or with a classic open, so it goes that I tried open and also fopen, as you can see. I'm failing because it can't read anything. That would be a time problem.
Fork and File Descriptors
When you fork a process, copies of all file descriptors are inherited. Since those are copies, the descriptors must be closed in both the child and the parent. You should always close them as soon as possible. This is especially true if you fork several times.
It's very easy to miss something here. It is therefore best to check very carefully that all file descriptors have been closed.
Minimum Amount of Changes
So the minimum number of changes for your code to get a result would be as follows.
If the first fork in line 41 is successful then in the parent you need to close the pipe file descriptors fd[0] and fd[1], e.g. in line 56.
pid = fork();
if (pid == 0) {
...
}
close(fd[0]); //<-- add these two lines
close(fd[1]);
if (pipe(fd2) < 0) {
...
Likewise you need to do the same after the second fork for fd1, so:
pid = fork();
if (pid == 0) {
...
}
close(fd1[0]); //<-- add these two lines
close(fd1[1]);
pid = fork();
When you now run your code you would already get as output:
1
2
hello
Better Test Case
This would not yet verify that both sed commands would run correctly. For a test case change the call in main to:
sender2("hello mars");
and change your sed commands to:
char* sed[] = {"sed", "s/moon/world/", NULL};
char* sed2[] = {"sed", "s/mars/moon/", NULL};
(sed2 command is executed before sed in your code, it would make the code a bit easier to understand if sed is executed before sed2)
This gives as output then:
1
2
hello world
So both sed commands are executed.
Additional Remarks
Below are some remarks in no particular order, mainly concerning error handling.
A call to fork returns pid_t and not int. So you should change your definition of the variable pid to: pid_t pid;.
If execvp fails one should print the error cause and exit with an error status, e.g. something like this:
perror("execvp of command xyz failed");
exit(EXIT_FAILURE);
If opening a pipe fails, also print a descriptive message on stderr.
Also fork calls can fail, this should also be handled. In this case fork returns -1. Same as above, print error message on stderr and return an error status.
In main you should return a success or failure state (e.g. return EXIT_SUCCESS;).
You don't use the the variable read_. Then the variable can be removed.
If fdopen fails it returns NULL. This error case should be handled.
The memory allocated with realloc is never released.

C - using different pipe ends in different processes

I've been searching and reading manuals, but still can't get the pipe mechanism. I'm making a program, which should do the following:
Parent process creates pipe, two child proccesses and waits.
First child generates pairs of random numbers and passes them with space between to the second process via pipe. It goes on till
gets signal from parent.
Second child redirects it input so it is the output of the first child and redirects output to the out.txt file. Then it executes
already compiled program, calculating GCD of numbers from (1);
Parent closes pipe and kills child.
So I got this C code (I reduced it so the post can fit the rules):
const int PIPE_READEND=0;
const int PIPE_WRITEEND=1;
(...)
if (child1 == 0) {
//Child1 code here
close(fd[1]);
struct sigaction sa;
sa.sa_handler = sigHandler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
if (sigaction(SIGUSR1,&sa,NULL) == -1){ //Handling SIGUSR1 signal
perror("Signal handling unexpected error");
exit(errno);
}
int a,b;
srand(time(&t));
if (dup2(fd[PIPE_READEND],1) < 0){ //Redirecting stdout to the pipe fd.
perror("In Child1 Redirecting stdout to pipe error");
exit(errno);
}
close(fd[0]);
while(1){
a = rand();
b = rand();
printf("%d %d\n", a, b);
sleep(1);
}
(...)
if ((child2 = fork()) < 0){
perror("Fork error in Child2 process");
exit(errno);
} else if (child2 == 0){
//Child2 code here
close(fd[PIPE_READEND]);
FILE *outfile = fopen("out.txt","w");
dup2(fd[PIPE_WRITEEND],0);
dup2(outfile,1);
close(fd[PIPE_WRITEEND]);
execl("c1/main","main",(char *)NULL);
The problem is, after executing it, out.txt stays empty. I'm disappointed about the pipe array indices, which one is used for what.
You write and read from wrong indexes of pipe. You need to change them:
This redirects stdout to pipe input.
close(fd[0]);
dup2(fd[1], STDOUT_FILENO);
This redirects pipe output to stdin.
close(fd[1]);
dup2(fd[0], STDIN_FILENO);
Also dup2 takes integer, not pointer, so you should do:
f = fopen("out.txt", "w");
dup2(fileno(f), STDOUT_FILENO);
FILE *outfile = fopen("out.txt","w");
dup2(fd[PIPE_WRITEEND],0);
dup2(outfile,1);
This makes no sense. The dup2 function doesn't take a FILE * as a parameter. Use open, not fopen.

C Programming: Fork() and IPC with Pipes

So I have this problem, I need to create 3 processes (each handle a different task). The first process sends information over to the second (the first waits for an acknowledgement from the second). The second then sends information to the third (the second waits for an acknowledgement from the third). Then the third processes the final information... This process is supposed to loop over and over until process one analyzes an entire text file. So far, I tried writing the communication between the 3 processes with pipes. I'm not sure how I send an acknowledgment from process 2 to process 1 and process 3 to process 2. I'm also not entirely sure how to loop it. Thanks!
I have to use a stop and wait protocol... I'm not sure how that is done.
#include <stdio.h>
#include <string.h>
#include <sys/wait.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
int c = 0, t = 0;
int fd1[2], fd2[2];
char *theFile = "/Users/Desktop/thefile";
FILE *file = fopen (theFile, "r");
if (file == NULL) {
perror("FILE DOES NOT EXIST");
exit(1);
}
while (c == 0) {
int status;
char readbuffer[80];
char readbuffer2[80];
int tTemp = 0;
pipe(fd1);
pipe(fd2);
pid_t pid = fork();
if (pid < 0) {
perror("Pipe Error");
exit(1);
}
if (pid == 0) {
//Child 1
close(fd1[0]);
close(fd2[0]);
close(fd2[1]);
char line [80];
int c2 = 0;
file = fopen (theFile, "r");
while (fgets(line, sizeof(line), file) != NULL){
if (c2 == t) {
printf("Line: %s\n", line);
break;
}
c2++;
}
if (t != c2) {
c = 1;
} else {
write(fd1[1], line, (strlen(line)+1));
}
t++;
exit(1);
}
pid_t pid2 = fork();
if (pid2 < 0) {
perror("Pipe Error");
exit(1);
}
if (pid2 == 0) {
//Child 2
close(fd1[1]);
close(fd2[0]);
read(fd1[0], readbuffer, sizeof(readbuffer));
printf("2nd Child string: %s\n", readbuffer);
char string2[80] = "asdfasdf";
write(fd2[1], string2, (strlen(string2)+1));
exit(1);
}
pid_t pid3 = fork();
if (pid3 < 0) {
perror("Pipe Error");
exit(1);
}
if (pid3 == 0) {
//Child 3
close(fd2[1]);
close(fd1[0]);
close(fd1[1]);
read(fd2[0], readbuffer2, sizeof(readbuffer2));
exit(1);
}
waitpid(pid, &status, 0);
waitpid(pid2, &status, 0);
waitpid(pid3, &status, 0);
}
fclose(file);
return 0;
}
As I have come to understand the problem via the comments, you are asking about two distinct requirements:
implementing a "stop & wait" protocol between each pair of processes, and
using waitpid() to collect child processes that finish
The latter is pretty straightforward; what you already have seems fine. The former is what you seem mostly to be stuck on.
There are a couple of things here. One is a question of semantics: stop & wait, in the form we are discussing it, requires the recipient of a message to acknowledge to the sender that a message was successfully received before the sender proceeds. There is a significant difference between characterizing the acknowledgment that way, and characterizing it as a signal for the receiver of the acknowledgment to perform any particular action. What the receiver does in response to the acknowledgment is its own concern, not inherent in the acknowledgment itself.
As to communications, then, I recommend establishing two pipes between each pair of processes, one for communication in each direction. To wait for an acknowledgment, then, a process just performs a blocking read on the appropriate pipe.
As for looping, each process must loop separately, but the loops will all take about the same form:
Read the next line (process 1 gets lines from a file; the others get lines from pipes connected to the previous process)
Terminate if no line is available
Except for process 1, write a one-byte acknowledgment message on the pipe to the previous process
Process the line
Write the line to the appropriate terminus (process 3 writes lines to an unspecified terminus -- maybe stdout -- the others write lines to pipes connected to the next process).
Except for process 3, perform a blocking read to receive an acknowledgment from the next process.
Go to (1)
Be sure to check the result codes of all the functions that provide them. Since that's most library functions and syscalls it can get tedious, so I suggest a macro to help you out there.
For clarity and readability, I suggest writing the work of each of the three processes as a separate function. After fork() to create each process just handle file descriptor mangling as needed and call the appropriate function.

How can the child process read stdout from the pipe and the parent process write stdin to the pipe?

// This code is pasted from
// http://linux.die.net/man/2/pipe
#include <sys/wait.h>
#include <unistd.h>
int main(int argc, char *argv[])
{
int pipefd[2];
pid_t cpid;
char buf;
if (argc != 2) {
fprintf(stderr, "Usage: %s <string>\n", argv[0]);
exit(EXIT_FAILURE);
}
if (pipe(pipefd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child reads from pipe */ <----- Point A
close(pipefd[1]); /* Close unused write end */
while (read(pipefd[0], &buf, 1) > 0)
write(STDOUT_FILENO, &buf, 1);
write(STDOUT_FILENO, "\n", 1);
close(pipefd[0]);
_exit(EXIT_SUCCESS);
} else { /* Parent writes argv[1] to pipe */ <----- Point B
close(pipefd[0]); /* Close unused read end */
write(pipefd[1], argv[1], strlen(argv[1]));
close(pipefd[1]); /* Reader will see EOF */
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
}
As what I understood,
if (...)
............; ---+
else |---> " Only ONE of them can be reached! "
............; ---+
So, how can the child process read from the pipe AND the parent process write to the pipe in this code?
The result of fork() is that one process becomes two (by asexual reproduction). So while it is still the case that exactly one branch of the if/else block will be taken in a process, there are two processes, and one path will be taken by each.
More specifically, look at what fork() returns: a PID to the parent, and 0 to the new child. Apart from that the two processes are almost identical. So the if (cpid == 0) check is a common pattern after fork() so that you can proceed with distinct logic in each process. In your case, that's reading in one process and writing in the other.
The system call fork() returns twice. Both in the parent process and the child process. The moment you call fork(), two exact copies of your program are running. The SINGLE difference is the return value of fork().
So your "if else only one" rule is still valid when you consider each process in isolation.
Check this resource for a description of the fork call return value:
On success, the PID of the child process is returned in
the parent, and 0 is returned in the child. On failure, -1 is returned
in the parent, no child process is created, and errno is set
appropriately.
So the line that contains cpid = fork(); is executed by both process after the fork, where the parent receives the new process' PID and the child receives 0 as PID. Hence the distinction between parent and child.

two forks and the use of wait

Currently am doing two forks to pipeline two process, but I think am doing my wait(&status) wrong because after the command my shell just hangs and does not return to my prompt. I know my pipe is working because I can see the result if I remove the wait.
Any tips?
pipe(mypipe);
pid1=fork();
if(pid1==0)
{
pid2=fork();
if(pid2==0)
{
close(0);
dup(mypipe[0]);
close(mypipe[1]);
execv(foundnode2->path_dir,arv2);
exit(0);
}
close(1);
dup(mypipe[1]);
close(mypipe[0]);
pid2 = wait(&status2);
execv(foundnode1->path_dir,arv1);
exit(0);
}
pid1 = wait(&status2);
Rule of Thumb: if you use dup() or dup2() to map one end of a pipe to standard input or standard output, you should close() both ends of the pipe itself. You're not doing that; your waits are waiting for the programs to finish but the programs will not finish because there is still a proess with the pipe open that could write to the pipe. Also, the process which created the pipe needs to close both ends of the pipe since it is not, itself, using the pipe (the child processes are using it). See also C MiniShell — Adding Pipelines.
Also, you should not be waiting for the first child to finish before launching the second (so the pid2 = wait(&status2); line is a bad idea). Pipes have a fairly small capacity; if the total data to be transferred is too large, the writing child may block waiting for the reading child to read, but the reading child hasn't started yet because it is waiting for the writing child to exit (and it takes a long time for this deadlock to resolve itself). You're seeing the output appear without the wait() calls because the second part of the pipeline executes and processes the data from the first part of the pipeline, but it is still waiting for more data to come from the shell.
Taking those tips into account, you might end up with:
pipe(mypipe);
pid1 = fork();
if (pid1 == 0)
{
pid2 = fork();
if (pid2 == 0)
{
close(0);
dup(mypipe[0]);
close(mypipe[1]);
close(mypipe[0]);
execv(foundnode2->path_dir, arv2);
fprintf(stderr, "Failed to exec %s\n", foundnode2->path_dir);
exit(1);
}
close(1);
dup(mypipe[1]);
close(mypipe[0]);
close(mypipe[1]);
execv(foundnode1->path_dir, arv1);
fprintf(stderr, "Failed to exec %s\n", foundnode1->path_dir);
exit(1);
}
close(mypipe[0]);
close(mypipe[1]);
pid1 = wait(&status1);
Notice the error reporting to standard error when the commands fail to execv(). Also, the exit status of 0 should be reserved for success; 1 is a convenient error exit status, or you can use EXIT_FAILURE from <stdlib.h>.
There is a lot of error checking omitted still; the fork() operations could fail; the pipe() might fail. One consequence is that if the second fork() fails, you still launch the second child (identified by foundnode1->path_dir).
And I note that you could save yourself a little work by moving the pipe creation into the first child process (the parent then does not need to — indeed, cannot — close the pipe):
int pid1 = fork();
if (pid1 == 0)
{
int mypipe[2];
pipe(mypipe);
int pid2 = fork();
if (pid2 == 0)
{
close(0);
dup(mypipe[0]);
close(mypipe[1]);
close(mypipe[0]);
execv(foundnode2->path_dir, arv2);
fprintf(stderr, "Failed to exec %s\n", foundnode2->path_dir);
exit(1);
}
close(1);
dup(mypipe[1]);
close(mypipe[0]);
close(mypipe[1]);
execv(foundnode1->path_dir, arv1);
fprintf(stderr, "Failed to exec %s\n", foundnode1->path_dir);
exit(1);
}
pid1 = wait(&status1);
If it's just a pipe with two processes, I wouldn't wait at all. Just fork and do an exec in parent and child.
int fd[2];
pipe(fd);
int pid = fork();
if (pid == -1) {
/* error handling */
} else if (pid == 0) {
dup2(fd[0], 0);
close(fd[1]);
execv(foundnode2->path_dir,arv2);
/* error handling for failed exec */
exit(1);
} else {
dup2(fd[1], 1);
close(fd[0]);
execv(foundnode1->path_dir,arv1);
/* error handling for failed exec */
exit(1);
}

Resources