A simple multi-process program in linux.
Input some numbers like ./findPrime 10 20 30.
The program will create 3 child processes to find out all primes between 2-10, 10-20, 20-30.
Once a child process find a prime, it will write "2 is prime" through a pipe and send to the parent. Parent will print it on the screen.
THE PROBLEM here is that, I use a while loop to write message into the pipe and use another while loop on the parent side to receive the message, but with the code below, it only display the first message, so I am wondering what`s going on, how can i keep reading from that pipe? Did I miss someting? Thanks very much!
char readBuffer[100];
char outBuffer[15];
int pids[argc];
int fd[2];
pipe(fd);
for(i = 0; i < argc; i++)
{
if( i == 0)
bottom = 2;
else
bottom = args[i - 1];
top = args[i];
pids[i] = fork();
if(pids[i] == 0)
{
printf("Child %d: bottom=%d, top=%d\n", getpid(), bottom, top);
close(fd[0]);
j = bottom;
while(j <= top)
{
int res = findPrime(j);
if(res == 1)
{
sprintf(outBuffer, "%d is prime", j);
write(fd[1], outBuffer, (strlen(outBuffer)+1));
}
j++;
}
exit(0x47);
}
else if(pids[i] < 0)
{
fprintf(stderr, "fork failed! errno = %i\n", errno);
break;
}
else
{
close(fd[1]);
while((nbytes = read(fd[0], readBuffer, sizeof(readBuffer))) > 0 )
printf("%s\n", readBuffer);
int status = 0;
pid = waitpid(pids[i], &status, 0);
if(pid >= 0)
printf("Child %d exited cleanly\n", pid);
}
}
And these child process should run in the order that they were created, like when Process 1 is done, then Process 2 will run, and process 3 will after 2.
I also want the parent process display the message immediately when it receives one.
Parent/children share their file descriptors (as they presently are) at the time of the fork. Your immediate problem is that you close fd[1] in the parent. When the first child ends the fact that the process ends means that fd[1] will be automatically closed in the child. As the OS no longer has any valid references to the file descriptor it becomes invalid. So your pipe writes fail in all subsequent children.
So just don't close fd[1] in the parent.
But then you have other problems too. One that jumps out is that if one of your child processes doesn't find a prime it will never write to the pipe. The parent, however, will block forever waiting for something to read that is never going to arrive. By not closing fd[1] in the parent you won't see EOF - i.e. read() == 0 in the parent. So one solution is to pass a "done" message back via the pipe and have the parent parse that stuff out.
A better solution yet is to consider a redesign. Count the number of processes you are going to need by parsing the command line arguments right at the beginning of the program. Then dynamically allocate the space for the number of pipe descriptors you are going to need and give each process its own pair. That could avoid everything altogether and is a more standard way of doing things.
Related
I'm working on a C shell and am having trouble with getting an arbitrary amount of pipes to work. When I run the shell, it hangs on any piping. For some reason, when I do ls -la | sort, it hangs on the sort until I enter stuff and hit Ctrl+D. I know it has something to do with a pipe not closing, but the print statements show that pipes 3,4,5 all get closed in both the parent and child. I've been at this for a few hours and don't know why this doesn't work. Any help would be much appreciated.
Original Code:
char *current_command;
current_command = strtok_r(cmdline_copy, "|", &cmdline_copy);
char *commands[100][MAX_ARGS]; //Max 100 piped commands with each having MAX_ARGS arguments
int i = 0;
while (current_command != NULL) { //Go through each command and add it to the array
char *copy = malloc(strlen(current_command)*sizeof(char)); //Copy of curretn command
strcpy(copy, current_command);
char *args_t[MAX_ARGS];
int nargs_t = get_args(copy, args_t);
memcpy(commands[i], args_t, sizeof(args_t)*nargs_t); //Copy the command and it's arguments to the 2d array
i++;
current_command = strtok_r(NULL, "|\n", &cmdline_copy); //Use reentrant version of strtok to prevent fighting with get_args function
}
int fd[2*(i-1)]; //Set up the pipes i.e fd[0,1] is first pipe, fd[1,2] second pipe, etc.
for (int j = 0; j < i*2; j+=2) {
pipe(fd+j);
}
//Here is where we do the commands
for (int j = 0; j < i; j++) {
pid = fork(); //Fork
if (pid == 0) { //Child process
if (j == 0) { //First process
printf("Child Closed %d\n", fd[0]);
close(fd[0]);
dup2(fd[1], fileno(stdout));
}
else if (j == i -1) { //Last process
dup2(fd[j], fileno(stdin));
printf("Child closed %d\n", fd[j]);
printf("Child closed %d\n", fd[j+1]);
close(fd[j+1]);
close(fd[j]);
}
else { //Middle processes
dup2(fd[j], fileno(stdin));
dup2(fd[j+1], fileno(stdout));
printf("Child closed %d\n", fd[j]);
close(fd[j]);
}
execvp(commands[j][0], commands[j]);
}
else if (pid > 0) { //Parent
printf("Parent closed %d\n", fd[j]);
close(fd[j]);
printf("Parent closed %d\n", fd[j+1]);
close(fd[j+1]);
waitpid(pid, NULL, 0); //Wait for the process
}
else {
perror("Error with fork");
exit(1);
}
}
Final Code:
char *current_command;
current_command = strtok_r(cmdline_copy, "|", &cmdline_copy);
char *commands[100][MAX_ARGS]; //Max 100 piped commands with each having MAX_ARGS arguments
int command_count = 0;
while (current_command != NULL) { //Go through each command and add it to the array
char *copy = malloc(strlen(current_command)*sizeof(char)); //Copy of curretn command because get_args uses strtok
strcpy(copy, current_command);
char *args_t[MAX_ARGS];
int nargs_t = get_args(copy, args_t);
memcpy(commands[command_count], args_t, sizeof(args_t)*nargs_t); //Copy the command and it's arguments to the 2d array
command_count++;
current_command = strtok_r(NULL, "|\n", &cmdline_copy); //Use reentrant version of strtok to prevent fighting with get_args function
}
int fd[command_count*2-1];
pid_t pids[command_count];
for (int j = 0; j < command_count*2; j+=2) { //Open up a pair of pipes for every command
pipe(fd+j);
}
for (int j = 0; j < command_count; j++) {
pids[j] = fork();
if (pids[j] == 0) { //Child process
if (j == 0) { //Duplicate only stdout pipe for first pipe
dup2(fd[1], fileno(stdout));
}
else if (j == (command_count-1)) { //Duplicate only stdin for last pipe
up2(fd[2*(command_count-1)-2], fileno(stdin));
}
else { //Duplicate both stdin and stdout
dup2(fd[2*(j-1)], fileno(stdin));
dup2(fd[2*j+1], fileno(stdout));
}
for (int k = 0; k < j*2; k++) { //Close all fds
close(fd[k]);
}
execvp(commands[j][0], commands[j]); //Exec the command
}
else if (pids[j] < 0) {
perror("Error forking");
}
}
for (int k = 0; k < command_count*2; k++) { //Parent closes all fds
close(fd[k]);
}
waitpid(pids[command_count-1], NULL, 0); //Wait for only the last process;
You aren't closing enough file descriptors in the children (or, in this case, in the parent).
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD
In your code, you create all the pipes before you fork any children; therefore, each child needs to close all the pipe file descriptors after duplicating the one or two that it is going to use for input or output.
The parent process must also close all the pipe descriptors.
Also, the parent should not wait for children to complete until after launching all the children. In general, children will block with full pipe buffers if you make them run sequentially. You also defeat the benefits of parallelism. Note, however, that the parent must keep the pipes open until it has launched all the children — it must not close them after it launches each child.
For your code, the outline operation should be:
Create N pipes
For each of N (or N+1) children:
Fork.
Child duplicates standard input and output pipes
Child closes all of the pipe file descriptors
Child executes process (and reports error and exits if it fails)
Parent records child PID.
Parent goes on to next iteration; no waiting, no closing.
Parent now closes N pipes.
Parent now waits for the appropriate children to die.
There are other ways of organizing this, of greater or lesser complexity. The alternatives typically avoid opening all the pipes up front, which reduces the number of pipes to be closed.
'Appropriate children' means there are various ways of deciding when a pipeline (sequence of commands connected by pipes) is 'done'.
One option is to wait for the last command in the sequence to exit. This has advantages — and is the traditional way to do it. Another advantage is that the parent process can launch the last child; the child can launch its predecessor in the pipeline, back to the first process in the pipeline. In this scenario, the parent never creates a pipe, so it doesn't have to close any pipes. It also only has one child to wait for; the other processes in the pipeline are descendents of the one child.
Another option is to wait for all the processes to die(1). This is more or less what Bash does. This allows Bash to know the exit status of each element of the pipeline; the alternative does not permit that — which is relevant to set -o pipefail and the PIPEFAIL array.
Can you help me understand why the dup2 statement for the middle pipes is dup2(fd[(2*j)+1], fileno(stdout)) and dup2(fd[2*(j-1)], fileno(stdin))? I got it off Google and it works, but I'm unsure why.
fileno(stdout) is 1.
fileno(stdin) is 0.
The read end of a pipe is file descriptor 0 (analogous to standard input).
The write end of a pipe is file descriptor 1 (analogous to standard output).
You have an array int fd[2*N]; for some value of N > 1, and you get a pair of file descriptors for each pipe.
For an integer k, fd[k*2+0] is the read descriptor of a pipe, and fd[k*2+1] is the read descriptor.
When j is neither 0 nor (N-1), you want it to read from the previous pipe and to write to its pipe:
fd[(2*j)+1] is the write descriptor of pipe j — which gets connected to stdout.
fd[2*(j-1)] is the read descriptor of pipe j-1 — which gets connected to stdin.
So, the two dup2() calls connect the the correct pipe file descriptors to standard input and standard output of process j in the pipeline.
(1)
There can be obscure scenarios where this leaves the parent hung indefinitely. I emphasize obscure; it requires something like a process that hangs around as a daemon without forking.
To learn how Pipe IPC mechanism works, I wrote a simple program that creates two child processes which share data using a pipe. The first child process has to read data from a file and pass it to the pipe.
Afterwards, the second child process has to read it, convert it to uppercase and write it to another file. The read system call in the second child process returns -1 when reading from the pipe. Also when I execute the program, in some cases printf in the first child does not print anything and in other cases printf in the second child does not print, too. Could you please point the mistakes in the program which are causing the problems?
int main(int args[], char * argv[]) {
int fd[2];
long length;
char buff1[250];
char buff2[250];
FILE * fptr1;
FILE * fptr2;
pid_t A, B;
pipe(fd);
A = fork();
if (A == -1) {
printf("error in fork of A\n");
exit(1);
}
if (A == 0) {
fptr1 = fopen(argv[1], "r"); // program receives file names as argument
if (fptr1 == NULL) {
printf("Erro in file open1\n");
exit(1);
}
fseek(fptr1, 0 L, SEEK_END);
length = ftell(fptr1);
fseek(fptr1, 0 L, SEEK_SET);
close(fd[0]);
fread(buff1, length, 1, fptr1);
buff1[length] = '\0';
printf("buff1 = %s", buff1);
write(fd[1], buff1, length);
fclose(fptr1);
exit(0);
} else {
B = fork();
if (B == -1) {
printf("Error in forking child B");
exit(1);
}
if (B == 0) {
fptr2 = fopen(argv[2], "w");
if (fptr2 == NULL) {
printf("Error in file open2\n");
exit(1);
}
close(fd[1]);
int n = read(fd[0], buff2, length);
printf("n = %d\n", n);
upper_string(buff2); // converts characters to uppecase
fwrite(buff2, 1, length, fptr2);
fclose(fptr2);
}
}
return 0;
}
There are few things to take into account here. First thing i would like to point is that you do not need to use two fork() calls. In that case you have three processes working in parallel (parent process and two child process, one per each fork() call).
One important point to take into account when you work with processes working in parallel is synchronism. In your code you are creating two processes. Parent process does not wait for any of its child, so it finishes its execution, and if child processes have not finished, they will become child of init process. But appart from that, you have the typical producer consumer problem. One of your child produce something and the other consume it, but how they work in parallel, consumer need to know that the product is ready to be consumed. So, in this case, i think the easiest way to do this job is to use just one fork(), so child become the producer and the parent process (the consumer) wait until its child finish the job.
I have parent process that creates two child processes. First child will write to pipe and second child will read from the pipe. After 5 seconds parent will terminate first child.(so its write end should be automatically closed, isn't it?). I need second child to terminate automatically, because it uses pipe and after first child is terminated, the pipe should be terminated too. My problem: how can i force child 2 to die immediately when child 1 i killed?(i don't need child 2 to print something after child 1 is dead, even if he still has any information in the pipe buffer to read). Should i use pipe2 instead of simple pipe?
void do_close(int fd){
if(close(fd) != 0){
perror("close");
exit(2);
}
}
void signalhandler(int signum){
fprintf(stderr, "f1 terminated!\n");
exit(1);
}
int main(){
pid_t f1 = -1, f2 = -1;
int pipefd[2];
if(pipe(pipefd) == -1){
perror("pipe");
exit(2);
}
f1 = fork();
if(f1 > 0){
f2 = fork();
}
if(f1 > 0 && f2 > 0){
do_close(pipefd[0]);
do_close(pipefd[1]);
sleep(5);
kill(f1, SIGTERM);
waitpid(f1, NULL, 0);
waitpid(f2, NULL, 0);
}
else if(f1 == 0){
signal(SIGTERM, signalhandler);
do_close(pipefd[0]);
while(1){
fflush(stdout);
write(pipefd[1], "Hello world!\n", 13);
sleep(1);
}
}
else if(f2 == 0){
do_close(pipefd[1]);
char *buf = (char*)malloc(13);
while(read(pipefd[0], buf, 13) > 0){
for(int i = 0; i < 13; i++){
printf("%c", buf[i]);
}
sleep(3);
}
}
return 0;
}
"Terminated" isn't the usual terminology for a pipe, and it might be causing a slight misunderstanding. The termination of the writing process isn't immediately "felt" by the reader if there is still data in the buffer.
To summarize the program, you have one process that writes to a pipe at a rate of 1 line per second for 5 seconds, then dies. Another process reads from the pipe at a rate of 1 line every 3 seconds until EOF or error, then exits.
The lines are small and fixed-size so there's no chance of reading an incomplete line.
Nothing in the program should cause an error on the pipe, so the second child process will read until EOF.
EOF on a pipe occurs when 2 conditions are met: there are no writers, and the buffer is empty. The death of the first child process accomplishes the first condition. The second condition is not immediately true, because at the 5 second mark, there have been 5 lines written to the pipe, and only 2 of them have been read (one at the start, and one after 3 seconds).
The second child process keeps reading, pulling in the remaining lines, and eventually exits after about 5*3=15 seconds.
(The timing isn't infinitely precise, so you aren't guaranteed to get exactly 5 lines written to the pipe. When I ran it I got 6.)
In the code below, do the forks actually run in parallel or one after another?
What is the meaning of wait(NULL) ?
(The program creates an n number of child processes, n is supplied via command line)
int main ( int argc, char *argv[] ) {
int i, pid;
for(i = 0; i < atoi(argv[1]); i++) {
pid = fork();
if(pid < 0) {
printf("Error occured");
exit(1);
} else if (pid == 0) {
printf("Child (%d): %d\n", i + 1, getpid());
exit(0);
} else {
wait(NULL);
}
}
}
They do run in parallel, up until the point that one of them waits.
wait(NULL) or more accurately wait(0) means wait until a state change in the child process. To find out about Unix / Linux C api calls, type man <function Name> on the command line. You'll need to get used to reading those pages, so better start now.
In your case
man wait
would have given you what you needed.
Since you've only fork(...)ed once, you have only one child. The parent waits until it changes state, and since the child's state during the fork is the same as the parent's state prior to the fork (that of running), the likely outcome is that the parent waits until the child dies. Then the parent will continue executing, but since it doesn't have much to do after the wait(...) call, it will quickly exit too.
Learning to use the fork() command and how to pipe data between a parent and it's children. I am currently trying to write a simple program to test how the fork and pipe functions work. My problem seems to be the correct use/placement of the wait function. I want the parent to wait for both of its children to finish processing. Here is the code I have so far:
int main(void)
{
int n, fd1[2], fd2[2];
pid_t pid;
char line[100];
if (pipe(fd1) < 0 || pipe(fd2) < 0)
{
printf("Pipe error\n");
return 1;
}
// create the first child
pid = fork();
if (pid < 0)
printf("Fork Error\n");
else if (pid == 0) // child segment
{
close(fd1[1]); // close write end
read(fd1[0], line, 17); // read from pipe
printf("Child reads the message: %s", line);
return 0;
}
else // parent segment
{
close(fd1[0]); // close read end
write(fd1[1], "\nHello 1st World\n", 17); // write to pipe
// fork a second child
pid = fork();
if (pid < 0 )
printf("Fork Error\n");
else if (pid == 0) // child gets return value 0 and executes this block
// this code is processed by the child process only
{
close(fd2[1]); // close write end
read(fd2[0], line, 17); // read from pipe
printf("\nChild reads the message: %s", line);
}
else
{
close(fd2[0]); // close read end
write(fd2[1], "\nHello 2nd World\n", 17); // write to pipe
if (wait(0) != pid)
printf("Wait error\n");
}
if (wait(0) != pid)
printf("Wait error\n");
}
// code executed by both parent and child
return 0;
} // end main
Currently my output looks something along the lines of:
./fork2
Child reads the message: Hello 1st World
Wait error
Child reads the message: Hello 2nd World
Wait error
Where is the appropriate place to make the parent wait?
Thanks,
Tomek
That seems mostly ok (I didn't run it, mind you). Your logic error is in assuming that the children will end in some particular order; don't check the results of wait(0) against a particular pid unless you're sure you know which one you're going to get back!
Edit:
I ran your program; you do have at least one bug, your second child process calls wait(), which you probably didn't want to do. I recommend breaking some of your code out into functions, so you can more clearly see the order of operations you're performing without all the clutter.
i think its better to use something like this, in order to wait for all the childrens.
int stat;
while (wait(&stat) > 0)
{}