I am trying to make a program that makes a number of children defined by the user. The parent must use named pipes (it is a requirement) to send information back and forth with his children. So, I need to create a number of named pipes equal to the amount of children I am forking. How can I do this efficiently and have every child know what his pipe is named?
pid_t childpid;
for(i = 0; i < numWorker; i++){
// char *pipeName = "somename";
// change the pipeName to reflect the child by adding a suffix
// mkfifo(pipeName, 0666);
childpid = fork();
if(childpid < 0){
perror("fork\n");
}
else if(childpid == 0){
signal(SIGCONT, handleSignalChild);
// how can I open the fifo here and then carry on reading and writing
//inside the while() below?
break; // child exits the creation loop.
}
}
// Main program execution begins here
while(1){
if(childpid == 0){
// read and write to the already opened pipes.
//code to handle child execution.
}
else{
// open all fifo pipes and get ready to read and write stuff.
//code to handle parent execution.
}
}
EDIT: Reworded the question to make more sense.
Related
I want to create a program in C, where I use fork() to create multiple children, then wait for all of them to finish and do parent code (only once).
I tried using for loop and two forks but I there is a problem: either parent code isn't running at the end or children are not running parallel.
//Number of processes I want to create
int processes = 6;
pid_t *main_fork = fork();
if(main_fork ==0){
for(int i=0;i<processes;i++){
pid_t *child_fork = fork();
if(child_fork ==0){
// child code
exit(0);
}
else if(child_fork >0){
//And here is the problem, with the wait: children don't
//run parallel and if I delete it, the main parent code doesn't run
wait(NULL);
}else{
// Child fork failed
printf("fork() failed!\n");
return 1;
}
}
}else if(main_fork >0){
wait(NULL);
//Main parent code - here I want to do something only once after all
//children are done
}else{
// Main fork failed
printf("fork() failed!\n");
return 1;
}
If somebody could could fix my code, or write a better solution to this problem I would be so grateful!
If you want all the children to run in parallel, you have to do the wait after all the children has been started. Otherwise you start a child, wait for it to finish, start a new one, wait for that to finish, start a third one, wait for the third one to finish, and so on...
So what you typically want to do is to start all the children and put all the pid_t in an array, and when you are finished you may call wait() for each pid_t
This is the simple, and good enough solution for your case.
Here is a sample code that you can fit to your problem:
pid_t children[processes];
for(int i=0; i<processes; i++)
{
pid_t child = fork();
if(child == 0)
{
// child code
....
// We call _exit() rather than exit() since we don't want to clean up
// data structures inherited from parent
_exit(0);
}
else if (child == -1)
{
// Child fork failed
fprintf(stderr, "myprog: fork failed, %s", strerror(errno));
// Do real cleanup on failure is to complicated for this example, so we
// just exit
exit(EXIT_FAILURE);
}
children[i] = child;
}
// Do something if you want to do something before you expect the children to exit
....
for(int i=0; i<processes; i++)
{
pid_t child = children[i];
int status;
waitpid(child, &status, );
// Do something with status
}
Naturally this is not a complete example that fits any situation. Sometimes you have to tell the children when they should exit. Other times the children are not started/stopped in one go, and you have to play with asynchronous events, and so on...
I'm working on a C shell and am having trouble with getting an arbitrary amount of pipes to work. When I run the shell, it hangs on any piping. For some reason, when I do ls -la | sort, it hangs on the sort until I enter stuff and hit Ctrl+D. I know it has something to do with a pipe not closing, but the print statements show that pipes 3,4,5 all get closed in both the parent and child. I've been at this for a few hours and don't know why this doesn't work. Any help would be much appreciated.
Original Code:
char *current_command;
current_command = strtok_r(cmdline_copy, "|", &cmdline_copy);
char *commands[100][MAX_ARGS]; //Max 100 piped commands with each having MAX_ARGS arguments
int i = 0;
while (current_command != NULL) { //Go through each command and add it to the array
char *copy = malloc(strlen(current_command)*sizeof(char)); //Copy of curretn command
strcpy(copy, current_command);
char *args_t[MAX_ARGS];
int nargs_t = get_args(copy, args_t);
memcpy(commands[i], args_t, sizeof(args_t)*nargs_t); //Copy the command and it's arguments to the 2d array
i++;
current_command = strtok_r(NULL, "|\n", &cmdline_copy); //Use reentrant version of strtok to prevent fighting with get_args function
}
int fd[2*(i-1)]; //Set up the pipes i.e fd[0,1] is first pipe, fd[1,2] second pipe, etc.
for (int j = 0; j < i*2; j+=2) {
pipe(fd+j);
}
//Here is where we do the commands
for (int j = 0; j < i; j++) {
pid = fork(); //Fork
if (pid == 0) { //Child process
if (j == 0) { //First process
printf("Child Closed %d\n", fd[0]);
close(fd[0]);
dup2(fd[1], fileno(stdout));
}
else if (j == i -1) { //Last process
dup2(fd[j], fileno(stdin));
printf("Child closed %d\n", fd[j]);
printf("Child closed %d\n", fd[j+1]);
close(fd[j+1]);
close(fd[j]);
}
else { //Middle processes
dup2(fd[j], fileno(stdin));
dup2(fd[j+1], fileno(stdout));
printf("Child closed %d\n", fd[j]);
close(fd[j]);
}
execvp(commands[j][0], commands[j]);
}
else if (pid > 0) { //Parent
printf("Parent closed %d\n", fd[j]);
close(fd[j]);
printf("Parent closed %d\n", fd[j+1]);
close(fd[j+1]);
waitpid(pid, NULL, 0); //Wait for the process
}
else {
perror("Error with fork");
exit(1);
}
}
Final Code:
char *current_command;
current_command = strtok_r(cmdline_copy, "|", &cmdline_copy);
char *commands[100][MAX_ARGS]; //Max 100 piped commands with each having MAX_ARGS arguments
int command_count = 0;
while (current_command != NULL) { //Go through each command and add it to the array
char *copy = malloc(strlen(current_command)*sizeof(char)); //Copy of curretn command because get_args uses strtok
strcpy(copy, current_command);
char *args_t[MAX_ARGS];
int nargs_t = get_args(copy, args_t);
memcpy(commands[command_count], args_t, sizeof(args_t)*nargs_t); //Copy the command and it's arguments to the 2d array
command_count++;
current_command = strtok_r(NULL, "|\n", &cmdline_copy); //Use reentrant version of strtok to prevent fighting with get_args function
}
int fd[command_count*2-1];
pid_t pids[command_count];
for (int j = 0; j < command_count*2; j+=2) { //Open up a pair of pipes for every command
pipe(fd+j);
}
for (int j = 0; j < command_count; j++) {
pids[j] = fork();
if (pids[j] == 0) { //Child process
if (j == 0) { //Duplicate only stdout pipe for first pipe
dup2(fd[1], fileno(stdout));
}
else if (j == (command_count-1)) { //Duplicate only stdin for last pipe
up2(fd[2*(command_count-1)-2], fileno(stdin));
}
else { //Duplicate both stdin and stdout
dup2(fd[2*(j-1)], fileno(stdin));
dup2(fd[2*j+1], fileno(stdout));
}
for (int k = 0; k < j*2; k++) { //Close all fds
close(fd[k]);
}
execvp(commands[j][0], commands[j]); //Exec the command
}
else if (pids[j] < 0) {
perror("Error forking");
}
}
for (int k = 0; k < command_count*2; k++) { //Parent closes all fds
close(fd[k]);
}
waitpid(pids[command_count-1], NULL, 0); //Wait for only the last process;
You aren't closing enough file descriptors in the children (or, in this case, in the parent).
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD
In your code, you create all the pipes before you fork any children; therefore, each child needs to close all the pipe file descriptors after duplicating the one or two that it is going to use for input or output.
The parent process must also close all the pipe descriptors.
Also, the parent should not wait for children to complete until after launching all the children. In general, children will block with full pipe buffers if you make them run sequentially. You also defeat the benefits of parallelism. Note, however, that the parent must keep the pipes open until it has launched all the children — it must not close them after it launches each child.
For your code, the outline operation should be:
Create N pipes
For each of N (or N+1) children:
Fork.
Child duplicates standard input and output pipes
Child closes all of the pipe file descriptors
Child executes process (and reports error and exits if it fails)
Parent records child PID.
Parent goes on to next iteration; no waiting, no closing.
Parent now closes N pipes.
Parent now waits for the appropriate children to die.
There are other ways of organizing this, of greater or lesser complexity. The alternatives typically avoid opening all the pipes up front, which reduces the number of pipes to be closed.
'Appropriate children' means there are various ways of deciding when a pipeline (sequence of commands connected by pipes) is 'done'.
One option is to wait for the last command in the sequence to exit. This has advantages — and is the traditional way to do it. Another advantage is that the parent process can launch the last child; the child can launch its predecessor in the pipeline, back to the first process in the pipeline. In this scenario, the parent never creates a pipe, so it doesn't have to close any pipes. It also only has one child to wait for; the other processes in the pipeline are descendents of the one child.
Another option is to wait for all the processes to die(1). This is more or less what Bash does. This allows Bash to know the exit status of each element of the pipeline; the alternative does not permit that — which is relevant to set -o pipefail and the PIPEFAIL array.
Can you help me understand why the dup2 statement for the middle pipes is dup2(fd[(2*j)+1], fileno(stdout)) and dup2(fd[2*(j-1)], fileno(stdin))? I got it off Google and it works, but I'm unsure why.
fileno(stdout) is 1.
fileno(stdin) is 0.
The read end of a pipe is file descriptor 0 (analogous to standard input).
The write end of a pipe is file descriptor 1 (analogous to standard output).
You have an array int fd[2*N]; for some value of N > 1, and you get a pair of file descriptors for each pipe.
For an integer k, fd[k*2+0] is the read descriptor of a pipe, and fd[k*2+1] is the read descriptor.
When j is neither 0 nor (N-1), you want it to read from the previous pipe and to write to its pipe:
fd[(2*j)+1] is the write descriptor of pipe j — which gets connected to stdout.
fd[2*(j-1)] is the read descriptor of pipe j-1 — which gets connected to stdin.
So, the two dup2() calls connect the the correct pipe file descriptors to standard input and standard output of process j in the pipeline.
(1)
There can be obscure scenarios where this leaves the parent hung indefinitely. I emphasize obscure; it requires something like a process that hangs around as a daemon without forking.
I am learning Linux and piping that kind stuff for my system programming course right now, I am having a hard time understanding closing file descriptors in an array of pipes now.
// write the code to loop over the command line arguments (remember to skip the executable name)
for (int i = 1; i < argc; i++) {
// call pipe before we fork
if ((pipe(pipe_fd[i-1])) == -1) {
perror("pipe");
exit(1);
}
// call fork
int result = fork();
if (result < 0) { // case: a system call error
// handle the error
perror("fork");
exit(1);
} else if (result == 0) { // case: a child process
// child does their work here
// child only writes to the pipe so close reading end
if (close(pipe_fd[i-1][0]) == -1) {
perror("close reading end from inside child");
exit(1);
}
// before we forked the parent had open the reading ends to
// all previously forked children -- so close those
int child_no;
for (child_no = 0; child_no < i-1; child_no++) {
if (close(pipe_fd[child_no][0]) == -1) {
perror("close reading ends of previously forked children");
exit(1);
}
}
int len = strlen(argv[i]);
// write len to the pipe as an integer
if (write(pipe_fd[i-1][1], &len, sizeof(int)) != sizeof(int)) {
perror("write from child to pipe");
exit(1);
}
// I'm done with the pipe so close it
if (close(pipe_fd[i-1][1]) == -1) {
perror("close pipe after writing");
exit(1);
}
// exit so I don't fork my own children on next loop iteration
exit(0);
} else {
// in the parent but before doing the next loop iteration
// close the end of the pipe that I don't want open
if (close(pipe_fd[i-1][1]) == -1) {
perror("close writing end of pipe in parent");
exit(1);
}
}
}
I will give a list of what I understand right now:
I understand parent and child process need to close those fds that they don't need to use, in this case child is writing to parent, so parent needs to close writing port and child needs to close reading port.
I understand file descriptors are shared among parent process and children process.
The above code is given from my lecture slide, I feel confused by one thing specifically.
In the loop, I observe that each child is closing its reading port once this child is created by fork, and the code that does this action is:
else if (result == 0) { // case: a child process
// child does their work here
// child only writes to the pipe so close reading end
if (close(pipe_fd[i-1][0]) == -1) {
perror("close reading end from inside child");
exit(1);
}
From what I understand at this point is that, each child is going to close its own reading port after being given birth by fork, and I think the latter children created SHOULD NOT worry about closing previous children's reading port.
But my understanding seems not correct after I read this code:
// before we forked the parent had open the reading ends to
// all previously forked children -- so close those
int child_no;
for (child_no = 0; child_no < i-1; child_no++) {
if (close(pipe_fd[child_no][0]) == -1) {
perror("close reading ends of previously forked children");
exit(1);
}
}
I don't understand why the latter children should go to close previous children's reading port, aren't those reading port already be closed once those children are created?
Thanks for helping me out. :)
A descriptor isn't really closed until all processes that have it open close it. Since each child inherits all the pipe descriptors from the previous process, they should close all the ones they're not using.
The main reason to close reading ports is so that the writing process will get an error or signal if it tries to write to the pipe after the reader has exited. If the other children kept all the reading ports opened, this wouldn't happen until all subsequent children exit.
I have 8 children, and am trying to use 8 pairs of nonduplex unnamed pipes to communicate with them. Thus, I have 2 pipes for each child and 16 pipes in total (one for childRead_ParentWrite and the other for parentRead_ChildWrite).
Anyway, my main question is when to close the pipes. I was taught to initially close the sides that are not being used by the process, and then when the process is finished with its side of the pipe, to close it off. However, I am brand new to the subject and am having some trouble. Here is my code:
// The 16 pipes
int fd_childReads_ParentWrites[8][2]; // Parent closes 0, Child closes 1
int fd_parentReads_ChildWrites[8][2]; // Child closes 0, Parent closes 1
// The 16 buffers
char buf_ChildReads_ParentWrites[8][80];
char buf_ParentReads_ChildWrites[8][80];
// CREATE THE PIPES
// FORK THE CHILDREN
for(playerNumber = 0; playerNumber < NUM_PLAYERS; playerNumber++)
{
pid = fork();
if (pid < 0) // Error occurred
{
printf("Fork Failed\n");
exit(1);
}
else if (pid == 0) // Child
{
break;
}
}
// MANAGE PROCESSES
if (pid == 0) // CHILD
{
printf("I am the child: %d\n", getpid());
// Close the appropriate pipe ends
close(fd_childReads_ParentWrites[playerNumber][1]);
close(fd_parentReads_ChildWrites[playerNumber][0]);
// CHILD DOES STUFF WITH PIPES
// When finished, close the working child pipe ends
close(fd_childReads_ParentWrites[playerNumber][0]);
close(fd_parentReads_ChildWrites[playerNumber][1]);
}
else // PARENT
{
printf("I am the parent: %d\n", getpid()); // NOT BEING PRINTED
// Close the appropriate pipe ends
for (i = 0; i < NUM_PLAYERS; i++)
{
close(fd_childReads_ParentWrites[i][0]);
close(fd_parentReads_ChildWrites[i][1]);
}
// PARENT DOES STUFF WITH PIPES
// Finally, close the working parent pipe ends
for (i = 0; NUM_PLAYERS < 8; i++)
{
close(fd_childReads_ParentWrites[i][1]);
close(fd_parentReads_ChildWrites[i][0]);
}
// Wait for the children
for (playerNumber = 0; playerNumber < NUM_PLAYERS; playerNumber++)
{
wait(NULL);
}
}
I must be doing something wrong. The program prints out the correct number of children, but the parent's printf() line is never printed. When I take out all of the close() functions it prints, but even taking out solely the children's close()'s doesn't print he parent line.
If someone could explain to me the correct way to close nonduplex unnamed pipes in a situation like this, that would be awesome.
A simple multi-process program in linux.
Input some numbers like ./findPrime 10 20 30.
The program will create 3 child processes to find out all primes between 2-10, 10-20, 20-30.
Once a child process find a prime, it will write "2 is prime" through a pipe and send to the parent. Parent will print it on the screen.
THE PROBLEM here is that, I use a while loop to write message into the pipe and use another while loop on the parent side to receive the message, but with the code below, it only display the first message, so I am wondering what`s going on, how can i keep reading from that pipe? Did I miss someting? Thanks very much!
char readBuffer[100];
char outBuffer[15];
int pids[argc];
int fd[2];
pipe(fd);
for(i = 0; i < argc; i++)
{
if( i == 0)
bottom = 2;
else
bottom = args[i - 1];
top = args[i];
pids[i] = fork();
if(pids[i] == 0)
{
printf("Child %d: bottom=%d, top=%d\n", getpid(), bottom, top);
close(fd[0]);
j = bottom;
while(j <= top)
{
int res = findPrime(j);
if(res == 1)
{
sprintf(outBuffer, "%d is prime", j);
write(fd[1], outBuffer, (strlen(outBuffer)+1));
}
j++;
}
exit(0x47);
}
else if(pids[i] < 0)
{
fprintf(stderr, "fork failed! errno = %i\n", errno);
break;
}
else
{
close(fd[1]);
while((nbytes = read(fd[0], readBuffer, sizeof(readBuffer))) > 0 )
printf("%s\n", readBuffer);
int status = 0;
pid = waitpid(pids[i], &status, 0);
if(pid >= 0)
printf("Child %d exited cleanly\n", pid);
}
}
And these child process should run in the order that they were created, like when Process 1 is done, then Process 2 will run, and process 3 will after 2.
I also want the parent process display the message immediately when it receives one.
Parent/children share their file descriptors (as they presently are) at the time of the fork. Your immediate problem is that you close fd[1] in the parent. When the first child ends the fact that the process ends means that fd[1] will be automatically closed in the child. As the OS no longer has any valid references to the file descriptor it becomes invalid. So your pipe writes fail in all subsequent children.
So just don't close fd[1] in the parent.
But then you have other problems too. One that jumps out is that if one of your child processes doesn't find a prime it will never write to the pipe. The parent, however, will block forever waiting for something to read that is never going to arrive. By not closing fd[1] in the parent you won't see EOF - i.e. read() == 0 in the parent. So one solution is to pass a "done" message back via the pipe and have the parent parse that stuff out.
A better solution yet is to consider a redesign. Count the number of processes you are going to need by parsing the command line arguments right at the beginning of the program. Then dynamically allocate the space for the number of pipe descriptors you are going to need and give each process its own pair. That could avoid everything altogether and is a more standard way of doing things.