How to create custom multiple processes in C? - c

I would like to ask you guys some help with C programming. Basically Im having issues with fork() system call.
Here's my question:
We have a Manager Process which has to create POP_SIZE Student processes. Manager Process and Student Processes itself cannot do anything else until all Student Processes have been created.
Every Student Process is identified by:
1) its identification number (6-digit integer)
2) grade obtained in specific exam (integer)
Here's the code I managed to write:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <sys/types.h>
#define POP_SIZE 10
int main(int argc, char *argv[]){
pid_t firstFork;
int *status;
int numStudents = 0;
pid_t managerChild, managerParent;
pid_t students[POP_SIZE];
int studentStatus[POP_SIZE];
switch(firstFork = fork()){
case -1:
perror("Something wrong with fork()\n");
break;
case 0:
managerChild = getpid();
printf("Manager Child Process %d started\n", managerChild);
printf("I have to create %d Student Processes\n", POP_SIZE);
for(int i = 0; i < POP_SIZE; i++){
switch(students[i] = fork()){
case -1:
perror("Something wrong with FORK in Manager Child Process\n");
break;
case 0:
printf("Created first Student Process PID: %d\n", getpid());
numStudents++;
break;
default:
printf("Haven't created all Student Processes\n");
waitpid(managerChild, status, WUNTRACED | WNOHANG);
printf("%d Student Processes succesfully created\n", numStudents);
break;
}
}
break;
default:
for(int i = 0; i < POP_SIZE; i++)
wait(NULL);
}
}
I'd need some help in understanding where to put wait(*status) or waitpid(pid, *status, __options) functions in my code in order to achieve my requirements specified above?
Moreover, how can I assign and keep storing of variables for every single process?
Thank you very much

Since you will be creating many child processes, it is best to start by creating a function that creates the child process, and has it execute a function specified by the caller. Let's assume both the ID number and grade are ints. Then,
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <string.h>
#include <errno.h>
#include <stdio.h>
/* Run func(id, grade) in a child process.
Returns the child process PID if success,
or -1 with errno set in case an error occurs.
*/
pid_t run_child(int id, int grade,
int (*func)(int id, int grade))
{
pid_t p;
p = fork();
if (p == -1) {
/* fork() failed; it set errno to indicate the error. */
return -1;
} else
if (!p) {
/* Run child process function. When it returns,
have the child exit with that exit status. */
exit(func(id, grade));
} else {
/* Parent process. p is positive. */
return p;
}
}
Note that the third parameter is a function pointer. We specify it using the function name. That function must take two int parameters (the ID and the grade, respectively), and return an int. For example:
/* Each child process runs this function.
*/
int child_process(int id, int grade)
{
printf("Child: id = %d, grade = %d, PID = %d.\n", id, grade, (int)getpid());
return EXIT_SUCCESS;
}
We can create a child process that runs that function using child_pid = run_child(123456, 5, child_process);. Note how the name of the function can be used to specify a function pointer. The standard C qsort() function uses the exact same mechanism to allow one to quicksort anything; the caller just needs to specify a function that can compare two elements in the array to be sorted.
We will be creating several children, and reaping them at once. That means it makes sense to write a function that reaps all child processes, essentially blocking until they all exit. We are likely interested in the exit statuses of at least some of them, so let's pass the interesting child processes PIDs, ints to save the status to, and the number of processes in those arrays, as parameters:
/* Reap all child processes.
If child_count > 0, child processes with PID in child_pid[]
will have child_pid[] negated when reaped, with exit status saved
in child_status.
The function returns the number of child processes reaped.
*/
size_t reap_children(pid_t *child_pid, int *child_status, size_t child_count)
{
size_t reaped = 0;
size_t i;
int status;
pid_t p;
while (1) {
/* Reap a child process, if any. */
p = wait(&status);
if (p == -1) {
/* errno == EINTR is not an error; it occurs when a
signal is delivered to a hander installed without
SA_RESTART flag. This will not occur in this program,
but it is good practice to handle that case gracefully. */
if (errno == EINTR)
continue;
/* errno set by wait(). */
return reaped;
}
/* Another child process was reaped. */
reaped++;
/* If the reaped child was one of the interesting ones,
negate its pid and save the exit status. */
for (i = 0; i < child_count; i++) {
if (child_pid[i] == p) {
child_pid[i] = -p;
child_status[i] = status;
break;
}
}
}
}
Note that p = wait(&status) reaps a child process. This means that if one or more child processes have already exited, it picks one of them, and returns its PID, with exit status saved to &status. If all child processes left are still running, the call will wait until at least one of them exits. If there are no more child processes, it returns -1 with errno set to ECHILD.
If signal handlers were used, wait() can also return -1 with errno set to EINTR, if a signal was delivered to a signal handler that was installed without the SA_RESTART flag with sigaction(). Many programmers forgo this check (because "it'll never happen"), but I do like to include that check because it is easy, and makes sure adding signal handling to my code won't bite me in the butt later on. I very often do, too. (Add signal handling, I mean.)
The reason we negate the pids when the respective child process is reaped, is simple: it allows us to easily detect which child processes were reaped. (POSIX says all process IDs are positive, and pid_t is a signed type. Negating a PID is a commonly used technique, too; just see e.g. waitpid().)
If we wanted to reap a specific child process, we'd use waitpid(). For example,
pid_t child, p; /* wait for 'child'. */
int status;
do {
p = waitpid(child, &status, 0);
if (p == -1) {
if (errno == EINTR)
continue;
break;
}
} while (p != child);
if (p == child) {
/* Reaped 'child', status in 'status'. */
} else {
/* Error: failed to reap 'child'. See 'strerror(errno)'. */
}
Do note that in POSIX/Unix terminology 'child process' refers to processes created by this process only; not "grandchildren", processes created by child processes.
I prefer to write my processes to take in parameters from the command line. If no parameters are specified, or -h or --help is specified, a short help ("usage") is displayed; this is extremely common in POSIX and Unix command-line tools, and therefore very intuitive.
The following main() takes one or more ID:grade as command-line parameters. For each one, it creates a child process, and has it run the child_process() function with the specified ID and grade. The main program will then reap them all, and describe the exit status of each child process.
int main(int argc, char *argv[])
{
pid_t child_pid[argc];
int child_status[argc];
int count, i, n, arg, id, grade, status;
char dummy;
if (argc < 2 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
fprintf(stderr, "\n");
fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
fprintf(stderr, " %s ID:GRADE [ ID:GRADE ]*\n", argv[0]);
fprintf(stderr, "\n");
return EXIT_SUCCESS;
}
status = EXIT_SUCCESS;
count = 0;
for (arg = 1; arg < argc; arg++) {
if (sscanf(argv[arg], "%d:%d %c", &id, &grade, &dummy) == 2) {
child_pid[count] = run_child(id, grade, child_process);
if (child_pid[count] == -1) {
fprintf(stderr, "Cannot fork a child process: %s.\n", strerror(errno));
status = EXIT_FAILURE;
} else
count++;
} else {
fprintf(stderr, "%s: Not a valid ID:GRADE specification.\n", argv[arg]);
status = EXIT_FAILURE;
}
}
if (count < 0) {
fprintf(stderr, "No running child processes.\n");
return EXIT_FAILURE;
}
n = reap_children(child_pid, child_status, count);
printf("Reaped %d child processes.\n", n);
for (i = 0; i < count; i++) {
if (child_pid[i] < 0) {
printf("Child process %d (%d of %d)", (int)(-child_pid[i]), i + 1, count);
if (WIFEXITED(child_status[i])) {
if (WEXITSTATUS(child_status[i]) == EXIT_SUCCESS)
printf(" exited with success (EXIT_SUCCESS), %d.\n", EXIT_SUCCESS);
else
if (WEXITSTATUS(child_status[i]) == EXIT_FAILURE)
printf(" exited with failure (EXIT_FAILURE), %d.\n", EXIT_FAILURE);
else
printf(" exited with status %d.\n", WEXITSTATUS(child_status[i]));
} else
if (WIFSIGNALED(child_status[i])) {
printf(" died from signal %d.\n", WTERMSIG(child_status[i]));
} else {
printf(" died from unknown causes.\n");
}
} else {
printf("Child process %d (%d of %d) was lost!\n", (int)child_pid[i], i + 1, count);
}
}
return status;
}
If you save the above as example.c, you can compile it to example using e.g.
gcc -Wall -O2 example.c -o example
If you then run say
./example 100001:1 100002:5 100003:3 21532:4
the output will be something like
Child: id = 100002, grade = 5, PID = 1260.
Child: id = 100001, grade = 1, PID = 1259.
Child: id = 100003, grade = 3, PID = 1261.
Child: id = 21532, grade = 4, PID = 1262.
Reaped 4 child processes.
Child process 1259 (1 of 4) exited with success (EXIT_SUCCESS), 0.
Child process 1260 (2 of 4) exited with success (EXIT_SUCCESS), 0.
Child process 1261 (3 of 4) exited with success (EXIT_SUCCESS), 0.
Child process 1262 (4 of 4) exited with success (EXIT_SUCCESS), 0.
Note that the initial Child: lines can be in any order, because the child processes run essentially in parallel. Each child process runs as soon as it is started, so this example is not a copy-and-paste answer to OP's requirements.
If you want to experiment with complex process hierarchies, I recommend using Graphviz to visualize them. For example, dot-kids.c:
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
static void reap_all(void)
{
pid_t p;
int status;
while (1) {
p = wait(&status);
if (p == -1) {
if (errno == EINTR)
continue;
if (errno == ECHILD)
return;
fprintf(stderr, "Process %d: reap_all(): %s.\n", (int)getpid(), strerror(errno));
return;
}
printf(" \"%d\" -> \"%d\" [ color=\"#ff0000\" ];\n", (int)p, (int)getpid());
if (WIFEXITED(status)) {
if (WEXITSTATUS(status) == EXIT_SUCCESS)
printf(" \"%d\" [ label=\"%d\" ];\n", (int)p, (int)p);
else
printf(" \"%d\" [ label=\"%d (exit %d)\" ];\n", (int)p, (int)p, WEXITSTATUS(status));
} else
if (WIFSIGNALED(status))
printf(" \"%d\" [ label=\"%d (signal %d)\" ];\n", (int)p, (int)p, WTERMSIG(status));
else
printf(" \"%d\" [ label=\"%d (lost)\" ];\n", (int)p, (int)p);
fflush(stdout);
}
}
static pid_t run_child(int (*child)(int depth, int width), int depth, int width)
{
pid_t p;
fflush(stdout);
fflush(stderr);
p = fork();
if (p == -1) {
fprintf(stderr, "Process %d: Cannot fork: %s.\n", (int)getpid(), strerror(errno));
return -1;
} else
if (!p) {
exit(child(depth, width));
} else {
printf(" \"%d\" -> \"%d\" [ color=\"#0000ff\" ];\n", (int)getpid(), (int)p);
fflush(stdout);
return p;
}
}
int child(int depth, int width)
{
if (depth > 0) {
while (width > 0)
run_child(child, depth - 1, width--);
reap_all();
}
return EXIT_SUCCESS;
}
int main(int argc, char *argv[])
{
int depth, width, i;
char dummy;
if (argc != 3 || !strcmp(argv[1], "-h") || !strcmp(argv[2], "--help")) {
fprintf(stderr, "\n");
fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv[0]);
fprintf(stderr, " %s depth width | dot -Tx11\n", argv[0]);
fprintf(stderr, "\n");
return EXIT_SUCCESS;
}
if (sscanf(argv[1], " %d %c", &depth, &dummy) != 1 || depth < 0) {
fprintf(stderr, "%s: Invalid depth.\n", argv[1]);
return EXIT_FAILURE;
}
if (sscanf(argv[2], " %d %c", &width, &dummy) != 1 || width < 1) {
fprintf(stderr, "%s: Invalid width.\n", argv[2]);
return EXIT_FAILURE;
}
printf("digraph {\n");
printf(" \"%d\" [ shape=\"box\", label=\"%d\" ];\n", (int)getpid(), (int)getpid());
fflush(stdout);
for (i = 0; i < width; i++)
run_child(child, depth, width - 1);
reap_all();
printf("}\n");
return EXIT_SUCCESS;
}
Compile it using e.g.
gcc -Wall -O2 dot-kids.c -o dot-kids
and run using e.g.
./dot-kids 1 3 | dot -Tx11
to see a process graph similar to
where the numbers are process IDs, blue arrows show which process created which, and red arrows show which process reaped which.

I think there are some mistakes in your code. The output I get is something like:
5 Student Processes succesfully created
Haven't created all Student Processes
Haven't created all Student Processes
3 Student Processes succesfully created
4 Student Processes succesfully created
Created first Student Process PID: 11436
Created first Student Process PID: 11438
Created first Student Process PID: 11437
Haven't created all Student Processes
4 Student Processes succesfully created
Haven't created all Student Processes
3 Student Processes succesfully created
Created first Student Process PID: 11439
Haven't created all Student Processes
3 Student Processes succesfully created
Created first Student Process PID: 11440
Haven't created all Student Processes
3 Student Processes succesfully created
Created first Student Process PID: 11441
Haven't created all Student Processes
2 Student Processes succesfully created
Created first Student Process PID: 11442
Created first Student Process PID: 11443
You see there are too much children executing, so this should make you suspicious (paricularly note that sometimes the number for the student processes seems decreasing from print to print). The parent will continue executing the for loop. However the child continues executing from the point where the fork is called and being it inside a loop, it will fork too creating another child and so on and so on. To avoid that you need a break from the for loop for the children processes.
You can try something like the following. I added a variable jj that if <0 means it is a child process executing. Before next loop iteration the variable is checked and if <0 it breaks from the for loop.
It is not the most elegant solution but seems ok.
#include<stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <sys/types.h>
#define POP_SIZE 10
int main(int argc, char *argv[]){
pid_t firstFork;
int *status;
int numStudents = 0;
pid_t managerChild, managerParent;
pid_t students[POP_SIZE];
int studentStatus[POP_SIZE];
switch(firstFork = fork()){
case -1:
printf("Something wrong with fork()\n");
break;
case 0:
managerChild = getpid();
printf("Manager Child Process %d started\n", managerChild);
printf("I have to create %d Student Processes\n", POP_SIZE);
int jj = 0;
for(int i = 0; i < POP_SIZE; i++){
switch(students[i] = fork()){
case -1:
printf("Something wrong with FORK in Manager Child Process\n");
jj = -1;
break;
case 0:
printf("Created first Student Process PID: %d\n", getpid());
numStudents++;
jj = -1;
break;
default:
printf("Haven't created all Student Processes\n");
waitpid(managerChild, status, WUNTRACED | WNOHANG);
printf("%d Student Processes succesfully created\n", numStudents);
break;
}
if (jj<0) break;
}
break;
default:
for(int i = 0; i < POP_SIZE; i++)
wait(NULL);
}
}

Related

Fork(), wait for all child process to finish vs wait for one child to finish

I don't understand as to where should I use the wait(NULL) or while(pid = wait(&status))>0). I am very confused and I get weird results please help!.
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main (int argc, char *argv[]) {
pid_t childpid = 0;
int i, n;
if (argc != 2){ /* check for valid number of command-line arguments */
fprintf(stderr, "Usage: %s processes\n", argv[0]);
return 1;
}
n = atoi(argv[1]);
for (i = 1; i < n; i++)
if ((childpid = fork()) <= 0)
break;
fprintf(stderr, "i:%d process ID:%ld parent ID:%ld child ID:%ld\n",
i, (long)getpid(), (long)getppid(), (long)childpid);
return 0;
}
How would you modify this code so that the original process prints out its information after all children have exited?
How would you modify this code so that a process prints its information after its child process has exited?
The same modification can do both jobs.
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
int main (int argc, char *argv[])
{
pid_t childpid = 0;
int i, n;
if (argc != 2)
{ /* check for valid number of command-line arguments */
fprintf(stderr, "Usage: %s processes\n", argv[0]);
return 1;
}
n = atoi(argv[1]);
for (i = 1; i < n; i++)
{
if ((childpid = fork()) <= 0)
break;
}
int corpse;
int status;
while ((corpse = wait(&status)) > 0)
{
fprintf(stderr, "%d: child %d exited with status 0x%.4X\n",
(int)getpid(), corpse, status);
}
fprintf(stderr, "i:%d process ID:%ld parent ID:%ld child ID:%ld\n",
i, (long)getpid(), (long)getppid(), (long)childpid);
return 0;
}
When a child process executes wait(), it immediately fails since there are no grandchildren to be waited for. Therefore, the child prints its output after "its children" (all zero of them) have exited.
When the parent process executes the wait() loop, it reports on each child as the information is made available, and prints its own information afterwards.
You've got more than one child in general, so the second question is a bit ambiguous. However, if you want it to wait for one child (and you don't care which), then remove the loop from around the wait(). If you care which child, use waitpid() instead of just wait().
You can make the output more interesting by returning i or a number calculated from i in the child processes. Note that before the forking loop runs from 1 rather than 0, so if you specify 4 on the command line, you get 3 child processes plus the original parent process.

exit() the program from parent before child process has terminated

I have a C server. This server has to handle multiple connections and user's input (through a simple ncurses GUI). So I created two childs.
My problem comes when from the main menu of the user interface, I need to exit the program (then terminate the second child process -which handles the connections- from the first child process).
I'll try to explain myself with a little example:
int main(){
pid_t pid;
int status1, status2;
if((pid = fork()) < 0){
perror("main fork failure:");
exit(1);
}
if(pid == 0){
pid = fork();
if(pid == 0){
/*
some stuff the second child does while
the first child is already running
*/
}
/* this is the first child */
int choice;
choice = menu();
switch(choice){
case 1:
break;
case 2:
/*
HERE I have to exit (from the first child first,
and from the program then): how can I kill the
second child that is running to prevent
zombie processes?
*/
// kill() which pid?
exit(2);
break;
}
wait(&status2);
}
wait(&status1);
return 0;
}
So, how can I kill it if I don't know the second child pid from the first child?
In your code, you reuse the variable pid, but fortunately, the non-zero pid is the one you need to signal.
Hence:
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
extern int menu(void);
static void wait_for_pid(int pid)
{
int status;
int corpse;
while ((corpse = wait(&status)) >= 0 && corpse != pid)
printf("Unexpected child %d exited with status 0x%.4X\n", corpse, status);
if (corpse == pid)
printf("Child %d exited with status 0x%.4X\n", corpse, status);
else
printf("Child %d died without its death being tracked\n", pid);
}
int main(void)
{
pid_t pid;
if ((pid = fork()) < 0)
{
perror("main fork failure:");
exit(1);
}
if (pid == 0)
{
if ((pid = fork()) < 0)
{
perror("child fork failure:");
exit(1);
}
if (pid == 0)
{
pause(); /* Do nothing until signalled */
exit(0);
}
/* this is the first child */
int choice = menu();
switch (choice)
{
case 1:
/* action 1 */
break;
case 2:
kill(pid, SIGTERM);
exit(2);
/*NOTREACHED*/
}
wait_for_pid(pid);
exit(0);
}
wait_for_pid(pid);
return 0;
}
The loop in the wait_for_pid() function should be overkill for the child, but the parent process could have children it doesn't know about under some circumstances — unlikely but not impossible circumstances.
The use of pause() in the second child is simply writing some code; it is not useful and would not therefore be what you'd write there. Writing the comment /* action 1 */ is likewise dummy code; you'd replace it with code that does something useful. I'd probably have functions to call for the first child and the second child, rather than embedding much code in main(). I assume that it's written as shown to create an MCVE (Minimal, Complete, Verifiable Example); thank you for keeping the code small.
The code above was untested because there was no menu() function. The code below has a menu function — not that it is very interactive.
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
extern int menu(void);
int menu(void)
{
printf("Dozing...\n");
sleep(1);
printf("Menu option 2 chosen\n");
return 2;
}
static void wait_for_pid(int pid)
{
int status;
int corpse;
int curpid = getpid();
printf("%d: waiting for children to die\n", curpid);
while ((corpse = wait(&status)) >= 0 && corpse != pid)
printf("%d: Unexpected child %d exited with status 0x%.4X\n", curpid, corpse, status);
if (corpse == pid)
printf("%d: Child %d exited with status 0x%.4X\n", curpid, corpse, status);
else
printf("%d: Child %d died without its death being tracked\n", curpid, pid);
}
int main(void)
{
pid_t pid;
if ((pid = fork()) < 0)
{
perror("main fork failure:");
exit(1);
}
if (pid == 0)
{
if ((pid = fork()) < 0)
{
perror("child fork failure:");
exit(1);
}
if (pid == 0)
{
printf("Second child (%d) - pausing\n", (int)getpid());
pause(); /* Do nothing until signalled */
printf("Second child (%d) - awake despite no signal handling\n", (int)getpid());
exit(0);
}
/* this is the first child */
printf("First child (%d) - menuing\n", (int)getpid());
int choice = menu();
switch (choice)
{
case 1:
/* action 1 */
break;
case 2:
printf("kill(%d, SIGTERM)\n", pid);
kill(pid, SIGTERM);
wait_for_pid(pid);
exit(2);
/*NOTREACHED*/
}
/* Reached on menu choices != 2 */
/* Probably needs a loop around the menu() - end loop before wait_for_pid() */
wait_for_pid(pid);
exit(0);
}
wait_for_pid(pid);
return 0;
}
When run, a sample output sequence was:
19489: waiting for children to die
First child (19490) - menuing
Dozing...
Second child (19491) - pausing
Menu option 2 chosen
kill(19491, SIGTERM)
19490: waiting for children to die
19490: Child 19491 exited with status 0x000F
19489: Child 19490 exited with status 0x0200
All of which looks as would be expected. You can see the death from SIGTERM in the status 0x000F (SIGTERM is normally 15, and is 15 on macOS Sierra, though AFAIK no standard demands that it is 15). You can see the first child exited normally with status 2 from the 0x0200. You can see that the parent started waiting before the children did anything. And you can see the debugging techniques — copious printing and including the PID most of the time.

traversing directories using fork()

I try to go in folders and read files using fork(). I use file tree walk function to go in the folders recursively. The basic idea is that there will be children number of files and directory in a directory. The children read each file seperately and concurrently. But, if there are directories the children will be parents of the children to read files.
static int soner_each_time(const char *filepath, const struct stat *info,
int typeflag, struct FTW *ftwinfo)
{
pid_t pid = 0;
char buf[BUFSIZE];
int status;
int i = 0;
/* The variables are related to functions of reading file */
int totalLines;
char arr[TOTALNUMBEROFLINES][BUFSIZE];
int retval;
const char *const filename = filepath + ftwinfo->base;
if (( pid = fork()) < 0) {
const int cause = errno;
fprintf(stderr, "Fork error: %s\n", strerror(cause));
errno = cause;
return -1;
}
else if( pid > 0 ) // parent
{
if (typeflag == FTW_DP || typeflag == FTW_D)
{
sprintf(buf, "%*s%s\n\n", ftwinfo->level * 4, "", filepath);
write(1, buf, strlen(buf));
pid = wait(&status);
if (pid == -1)
perror("Failed to wait for child");
else if (WIFEXITED(status) && !WEXITSTATUS(status))
printf("parent [%d] reaped child [%d]\n", getpid(), pid);
else if (WIFEXITED(status))
printf("Child %ld terminated with return status %d\n",
(long)pid, WEXITSTATUS(status));
else if (WIFSIGNALED(status))
printf("Child %ld terminated due to uncaught signal %d\n",
(long)pid, WTERMSIG(status));
else if (WIFSTOPPED(status))
printf("Child %ld stopped due to signal %d\n",
(long)pid, WSTOPSIG(status));
}
}
if (pid == 0) // child
{
if (typeflag == FTW_F)
{
sprintf(buf, "||| Child [%d] of parent [%d]: %s |||\n", getpid(), getppid(), filename);
write(1, buf, strlen(buf));
/* Both of them are about reading function */
totalLines = storeLinesInArray(filename, arr);
retval = for_each_file(filename, totalLines, key, arr);
sprintf(buf, "||| Child [%d] of parent [%d] is about to exit |||\n", getpid(), getppid());
write(1, buf, strlen(buf));
}
else if (typeflag == FTW_DP || typeflag == FTW_D)
{
sprintf(buf, "%*s%s\n\n", ftwinfo->level * 4, "", filepath);
write(1, buf, strlen(buf));
}
}
return 0;
}
FTW_DP and FTW_D indicates folders FTW_F indicates files. Basically, I tried in the code fork() every time. If it is parent, It waits for its children read the files. Because the function is recursive, it will fork every calling. But, there's something about it I cant't get it forks more than one for one file. For example there should be one child for 1a.txt but for this scheme it is 8. Forking subject is really difficult. I do everyday exercises and try to understand it. Your explanations and helps will improve my skill in that branch.
#edit: mcve code
#define _POSIX_C_SOURCE 200809L
#define _XOPEN_SOURCE 700
#include <unistd.h>
#include <dirent.h>
#include <stdlib.h>
#include <locale.h>
#include <string.h>
#include <ftw.h>
#include <stdio.h>
#define TOTALNUMBEROFLINES 1000
#define BUFSIZE 1000
void err_sys(const char *const str)
{
perror(str);
fflush(stdout);
exit(1);
}
int storeLinesInArray(const char *file, char lines[][BUFSIZE])
{
return 0;
}
static int for_each_file(const char *filepath, int totalLines, const char *key, const char arr[][BUFSIZE])
{
fprintf(stdout, "File name is = %s\n", filepath);
fflush(stdout);
return 0;
}
static int soner_each_time(const char *filepath, const struct stat *info,
int typeflag, struct FTW *ftwinfo)
{
pid_t pid = 0;
char buf[BUFSIZE];
int status;
/* The variables are related to functions of reading file */
int totalLines;
char arr[TOTALNUMBEROFLINES][BUFSIZE];
int retval;
const char *const filename = filepath + ftwinfo->base;
if (( pid = fork()) < 0) {
perror("failed fork");
exit(-1);
}
else if( pid > 0 ) // parent
{
if (typeflag == FTW_DP || typeflag == FTW_D)
{
sprintf(buf, "%*s%s\n\n", ftwinfo->level * 4, "", filepath);
write(1, buf, strlen(buf));
pid = wait(&status);
if (pid == -1)
perror("Failed to wait for child");
else if (WIFEXITED(status) && !WEXITSTATUS(status))
printf("parent [%d] reaped child [%d]\n", getpid(), pid);
else if (WIFEXITED(status))
printf("Child %ld terminated with return status %d\n",
(long)pid, WEXITSTATUS(status));
else if (WIFSIGNALED(status))
printf("Child %ld terminated due to uncaught signal %d\n",
(long)pid, WTERMSIG(status));
else if (WIFSTOPPED(status))
printf("Child %ld stopped due to signal %d\n",
(long)pid, WSTOPSIG(status));
}
}
if (pid == 0) // child
{
if (typeflag == FTW_F)
{
sprintf(buf, "||| Child [%d] of parent [%d]: %s |||\n", getpid(), getppid(), filename);
write(1, buf, strlen(buf));
/* Both of them are about reading function */
totalLines = storeLinesInArray(filename, arr);
retval = for_each_file(filename, totalLines, "not needed now", arr);
sprintf(buf, "||| Child [%d] of parent [%d] is about to exit |||\n", getpid(), getppid());
write(1, buf, strlen(buf));
}
else if (typeflag == FTW_DP || typeflag == FTW_D)
{
sprintf(buf, "%*s%s\n\n", ftwinfo->level * 4, "", filepath);
write(1, buf, strlen(buf));
}
}
return 0;
}
int main(int argc, char *argv[])
{
if (nftw("here is directory path", soner_each_time, 15, FTW_CHDIR)) {
fprintf(stderr, "Failed directory.\n");
exit(-1);
}
return 0;
}
You had a few bugs. The corrected code is below.
The child did not do an exit call, so it would continue with it's own nftw, so many files were being redundantly processed. I added the exit(0).
forks were being done so fast that the system would run out of free pids.
I've added three things to fix this:
A "reap" routine that loops on waitpid(0,&status,WNOHANG) to catch done children
Added a loop around the fork to catch the "out of slots" problem
Added a throttling mechanism to limit the number of active children to a sane/useful value
I've annotated the source to point out the places were the bugs were.
While not hard bugs, doing a fork for each file adds significant overhead. The disk bandwidth will saturate with about four active child threads, so using more just slows things down. Forking a child for the directory doesn't do much since the "meaty" processing is going to be for the file.
Anyway, here's the corrected code [please pardon the gratuitous style cleanup]:
#define _POSIX_C_SOURCE 200809L
#define _XOPEN_SOURCE 700
#include <unistd.h>
#include <dirent.h>
#include <stdlib.h>
#include <locale.h>
#include <string.h>
#include <errno.h>
#include <ftw.h>
#include <stdio.h>
#include <sys/wait.h>
#define TOTALNUMBEROFLINES 1000
#define BUFSIZE 1000
// output verbose/debug messages
int opt_v;
// limit of number of children that can be used at one time (if non-zero)
int opt_T;
int pendcnt; // number of active children
void
err_sys(const char *const str)
{
perror(str);
fflush(stdout);
exit(1);
}
int
storeLinesInArray(const char *file, char lines[][BUFSIZE])
{
return 0;
}
static int
for_each_file(const char *filepath, int totalLines, const char *key, const char arr[][BUFSIZE])
{
fprintf(stdout, "File name is = %s\n", filepath);
fflush(stdout);
return 0;
}
// reap_some -- reap a few processes
int
reap_some(int final)
{
pid_t pid;
int status;
int reapcnt;
reapcnt = 0;
// reap all completed children
while (1) {
pid = waitpid(0,&status,WNOHANG);
if (pid == 0)
break;
if (pid == -1) {
if (errno != ECHILD)
perror("Failed to wait for child");
break;
}
if (WIFSIGNALED(status)) {
printf("Child %ld terminated due to uncaught signal %d\n",
(long) pid, WTERMSIG(status));
++reapcnt;
continue;
}
if (WIFSTOPPED(status)) {
printf("Child %ld stopped due to signal %d\n",
(long) pid, WSTOPSIG(status));
continue;
}
if (WIFEXITED(status)) {
++reapcnt;
if (WEXITSTATUS(status) == 0) {
if (opt_v)
printf("parent [%d] reaped child [%d]\n", getpid(), pid);
}
else
printf("Child %ld terminated with return status %d\n",
(long) pid, WEXITSTATUS(status));
continue;
}
}
// bump down the number of children that are "in-flight"
pendcnt -= reapcnt;
return reapcnt;
}
static int
soner_each_time(const char *filepath, const struct stat *info, int typeflag, struct FTW *ftwinfo)
{
pid_t pid = 0;
char *bp;
int lvl;
char buf[BUFSIZE];
/* The variables are related to functions of reading file */
int totalLines;
char arr[TOTALNUMBEROFLINES][BUFSIZE];
int retval;
const char *const filename = filepath + ftwinfo->base;
switch (typeflag) {
case FTW_DP:
case FTW_D:
bp = buf;
for (lvl = 0; lvl < ftwinfo->level; ++lvl)
bp += sprintf(bp," ");
bp += sprintf(bp, "%s\n\n",filepath);
write(1, buf, strlen(buf));
//reap_some(0);
break;
case FTW_F:
// BUGFIX:
// limit the number of in-flight children
// too many children serves no purpose -- they saturate the system
// resources and performance actually goes _down_ because the system
// spends more time doing context switches between them than the actual
// work. more than a few children to process files produces little
// benefit after the disk I/O is running at maximum
if (opt_T) {
while (pendcnt > opt_T)
reap_some(0);
}
// BUGFIX:
// without a throttle, we spawn children so fast we're going to get
// [many] failures here (i.e. we use up _all_ available pids)
while (1) {
pid = fork();
if (pid >= 0)
break;
reap_some(0);
}
// parent
// keep track of the child count
if (pid > 0) {
++pendcnt;
break;
}
// child
sprintf(buf, "||| Child [%d] of parent [%d]: %s |||\n",
getpid(), getppid(), filename);
if (opt_v)
write(1, buf, strlen(buf));
/* Both of them are about reading function */
totalLines = storeLinesInArray(filename, arr);
retval = for_each_file(filename, totalLines, "not needed now", arr);
sprintf(buf, "||| Child [%d] of parent [%d] is about to exit (RETVAL: %d) |||\n", getpid(), getppid(), retval);
if (opt_v)
write(1, buf, strlen(buf));
// BUGFIX:
// child won't exit without this -- causing multiple children to redo
// the same files (i.e. they would continue the nftw -- only parent
// should do that)
exit(0);
break;
}
return 0;
}
int
main(int argc, char **argv)
{
char *cp;
--argc;
++argv;
opt_T = 10;
for (; argc > 0; --argc, ++argv) {
cp = *argv;
if (*cp != '-')
break;
switch (cp[1]) {
case 'T': // throttle
cp += 2;
opt_T = (*cp != 0) ? atoi(cp) : 0;
break;
case 'v': // verbose messages
opt_v = 1;
break;
}
}
cp = *argv;
printf("opt_T=%d opt_v=%d -- %s\n",opt_T,opt_v,cp);
sleep(3);
printf("away we go ...\n");
if (nftw(cp, soner_each_time, 15, FTW_CHDIR)) {
fprintf(stderr, "Failed directory.\n");
exit(1);
}
// wait for all children to complete
while (pendcnt > 0)
reap_some(1);
return 0;
}
UPDATE:
Changed the code to do directory processing in the parent only (i.e. child is forked only for files). Fixed a bug. So, now, the -T throttle parameter works with a much lower value and can be the equivalent of "number of workers". Changed program to use a default throttle value.
UPDATE #2:
I said parent because there is only one parent. I wonder whether I may trace wrong.
No, you are correct. There is only one parent. That was by design.
I would like to make parent for each directory like explained in the first scheme.
Actually, you wouldn't/won't with a proper understanding of what's truly involved. Obi Wan Kenobi: "These are not the droids you're looking for"
There are a number of technical, performance, and system saturation issues with doing a recursive fork on each directory. The example I coded avoids all these with the best compromise for design and performance. It also allowed the master to "run ahead" of the children and keep children as busy as possible, regardless of the number of files/subdirs in a given directory.
Side note: I've got 40+ years experience and I've written a number of nftw equivalent programs. So, the following comes from all that.
What's the desired end result?
You've only got skeleton code, but what you actually do [intend to do] influences the architecture. Your ultimate program may be:
CPU bound [constantly waiting for CPU operations like multiplies, etc]
Memory bound [constantly waiting for reads from or writes to DRAM to complete]
I/O bound [constantly waiting for I/O operations to complete]
Also, do you want pre-order or post-order [like FTW_DEPTH] traversal? I presume pre-order
You can no longer use nftw.
You will need to do your equivalent using opendir/readdir/closedir [which is what nftw does].
What you need is a process that does a single level in the hierarchy. It's torture to get nftw to abort and start a new one to achieve that.
Below is some pseudo code for this.
But ... The implementation becomes more complex and will not provide better performance and may actually degrade performance. It may also cause unrelated programs to crash, such as Firefox, vlc, window managers, etc.
You'll now need interprocess communication and shared memory
With my example above, there was only one control process. To maintain throttling, only a simple increment/decrement of pendcnt was required.
When you add recursive forks for directories, now any subprocess forked for a directory has to increment/decrement the global copy of pendcnt in shared memory. It must use an interprocess semaphore to control access to that variable. Or, perhaps, some atomic increment/decrement primitives [ala C11 atomics].
Now, contention for that semaphore becomes a [delay] factor.
Performance:
Having more than a few active processes actually degrades performance. In other words, forking for the directory will actually run slower than a single process.
Beyond a few "worker" processes that do something with a file, the disk I/O bandwidth will be used up. You'll get no further benefit by adding more processes.
With many processes, they may actually interfere with one another. Consider that process A requests a disk read. But, so does process B. A's read completes inside the kernel, but before it can be returned to A, the kernel buffer for A's read has to be repurposed to fulfill B's read. A's read will have to be repeated.
This is what's known as [virtual memory] page "thrashing".
Locking up and crashing the system
As more and more disk I/O is done, more and more kernel buffers have to be used to contain the data. The kernel may have to evict page buffers to make room. Some of them may be for the unrelated programs mentioned above.
In other words, your program's many processes may monopolize the CPU, disk, and memory usage. Some programs like Firefox will timeout [and crash] because they see long delays that they wouldn't see otherwise and assume that something internal to them caused the delay.
I've run such an nftw program and seen Firefox say: "Killing locked up javascript script".
Worse yet, I've had vlc fall behind in timing and start skipping frames. This caused the window manager to get confused because it thought this was due to some logic error instead of just a very slow response system. The end result was that the window manager aborted and had to be manually restarted.
This can also slow down more critical programs and kernel daemons.
In certain cases, this could only be cleaned up by a system reboot.
Also, running many processes on a system you share with others will turn you into a "bad citizen", so be careful about consuming too many resources.
Anyway, here's the pseudo code:
// pseudo -- loose pseudo-code for non-nftw method
//
// NOTES:
// (1) pendcnt must now be a _shared_ memory variable (e.g. shmget, etc)
// (2) access must be locked by a shared memory semaphore
// (3) we must now have a list of our outstanding children
// (4) we can no longer do a blind waitpid(0,&status,WNOHANG) as we need to
// keep track of when our direct children complete
struct entfile {
struct dirent ent;
struct stat st;
};
// dodir -- enter/exit directory and perform all actions
void
dodir(const char *subdir)
{
// NOTE: you can create a wrapper struct for this that also has stat
struct entfile dirlist[1000];
// add subdir to directory stack ...
dirstack_push(subdir);
// enter directory
chdir(subdir);
// do whatever you'd like ...
process_directory(subdir);
// open directory
dirctl = opendir(".");
// pre-save all entries [skipping "." and ".."]
// this prevents too many open directory descriptors
// NOTE: we should stat(2) the file if d_type not supported
while (1) {
dirent = readdir(dirctl);
stat(dirent->d_name,&st);
add_to_dirent_list(dirlist,dirent,&st);
}
// close directory _before_ we process any entries
closedir(dirctl);
// process all file entries -- pre-order
for (ALL_IN_DIRLIST(ent,dirlist)) {
if (ent->ent.d_type == ISFILE)
doentry(ent);
}
wait_for_all_on_pendlist();
// process all directory entries -- pre-order
for (ALL_IN_DIRLIST(dirent,dirlist)) {
if (ent->ent.d_type == ISDIR)
doentry(ent);
}
wait_for_all_on_pendlist();
// remove directory from stack
dirstack_pop();
// exit directory
chdir("..")
}
// doentry -- process a directory entry
void
doentry(struct entfile *ent)
{
char *tail;
tail = ent->ent.d_name;
do {
// does throttle, etc.
pid = forkme();
// parent
// see notes above
if (pid) {
// NOTE: these semaphore waits can be costly
sem_wait();
++pendcnt;
sem_post();
add_pid_to_pendlist(pid,tail,...);
break;
}
// child
switch (ent->st.st.st_mode & ...) {
case ISFILE:
process_file(tail);
break;
case ISDIR:
dodir(tail);
break;
}
exit(0);
} while (0);
}
// wait for immediate children
void
wait_for_all_on_pendlist(void)
{
while (MORE_IN_PENDLIST) {
for (FORALL_IN_PENDLIST(tsk)) {
pid = waitpid(tsk->pid,&tsk->status,WNOHANG);
// check status like reap_some
if (pid > 0)
remove_pid_from_pendlist(tsk);
}
}
}

Wait() runs twice?

In my code below, I'm running a parent process which forks off into two child processes. After child(getpid());, both children exit with a status.
However, when I run the parent process, it somehow always decides to run the parent section twice (sets two different pid values), and I just can't get it to run just once. Is there a way to make wait stop after getting one value?
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
void child(int n) { //n: child pid
printf("\nPID of child: %i \n", n);
//random number rand
int randFile = open("/dev/random", O_RDONLY);
int r;
if(rand < 0)
printf("ERROR: %s\n", strerror(errno));
else {
unsigned int seed;
read(randFile, &seed, 4); //&rand is a pointer, 4 bytes
int randClose = close(randFile);
srand(seed); //seeds rand() with random from /dev/random
r = rand();
if(randClose < 0)
printf("ERROR: %s\n", strerror(errno));
//range between 5 and 20 seconds
r = r % 20;
if( r < 5)
r = 5;
}
// printf("\n%i\n", r);
sleep(r);
// sleep(1);
printf("\n child with pid %i FINISHED\n", n);
exit( r );
}
int main() {
printf("\nPREFORK\n");
int parentPID = getpid();
int child0 = fork();
if(child0 < 0)
printf("ERROR: %s\n", strerror(errno));
int child1 = fork();
if(child1 < 0)
printf("\nERROR: %s\n", strerror(errno));
if(getpid() == parentPID)
printf("\nPOSTFORK\n");
//if children
if(child1 == 0) //using child1 as child-testing value b/c when child1 is set, both children are already forked
child(getpid());
int status;
int pid = wait(&status);
//parent
if(getpid() != 0) {
if( pid < 0)
printf("\nERROR: %s\n", strerror(errno));
if ( pid > 0 && pid != parentPID) {
printf("\nPID of FINISHED CHILD: %i\n Asleep for %i seconds\n", pid, WEXITSTATUS(status));
printf("PARENT ENDED. PROGRAM TERMINATING");
}
}
return 0;
}
The parent is doing:
int child0 = fork(); // + test if fork failed
int child1 = fork(); // + test if fork failed
First you only have the parent.
After 1st fork you have the parent and the 1st child, both at the same execution point, so just before the next fork.
So just after that the parent re-creates a child, and the 1st child creates its own child (and will act like the parent).
You have to use if/else so that you are sure that the child don't fork. i.e.:
child0 = fork(); // add check for errors
if (child0 == 0) {
// the 1st child just have to call that
child(getpid());
exit(0);
}
// here we are the parent
child1 = fork();
if (child1 == 0) {
// the 2nd child just have to call that
child(getpid());
exit(0);
}
You can do that differently, of course, this is just an example. The main point is to not call fork() within the child.

how to not block parent with waitpid

I need to create a program that creates n number of processes and displays information. When each process ends, I am to print it's PID and the exit status. The way I am doing it, the parent program waits to create the next process until the current one ends. I need it so that it keeps creating the child processes and just displays the exit information when ever one process ends without blocking the parent from continuing. I can;t figure out where to put my wait to ensure this. Below is my code:
int main (int argc, char *argv[])
{
if (argc < 2)
{
printf("\n\nUsage: %s <enter a number (12 or less)>\n\n", argv[0]);
exit (-1);
}
else
{
int *processNum = (int *)malloc(sizeof(12));
int processNumTemp;
processNumTemp = atoi(argv[1]);
processNum = &processNumTemp;
if(*processNum > 12 || *processNum < 1)
{
printf("\n\nUsage: %s <enter a number (12 or lrss)>\n\n", argv[0]);
}
else
{
parentInfo(processNum);
createChildProcess(processNum);
}
}
return 0;
}
//Name: parentInfo
//Description: Displays information about the parent process
//Parameters: processNum - stores the number of child processes to create
// (entered at the command line).
//Return: none
void parentInfo(int *processNum)
{
printf("Parent process ID: %d\n", getppid());
printf("Number of processes to create: %d\n", *processNum);
}
//Name: createChildProcess
//Description: Creates n number of child processes.
// For each child process, it says its a child process and it
// displays its PID.
// After each child process closes, the parent displays info.
//Parameters: processNum - stores the number of child processes to create
// (entered at the command line).
//Return: none
void createChildProcess(int *processNum)
{
int i;
int childStatus;
pid_t childpid;
/*The for loop will create n number of processes based on the value of processNum.*/
for(i = 1; i <= *processNum; i++)
childpid = fork();
//Executes if fork didn't work
if(childpid < 0)
{
perror("fork");
exit(1);
}
//Executes if the fork worked
else if( childpid == 0)
{
int pid = getpid();
//Prints a message and the child processe's PID
printf("\nHello I am a child process.\n");
printf("My PID is %d. \n", getpid());
for(int x = 1; x <= pid; x ++);
exit(15);
}
}
//Executes after the child process has ended
//Checks the child process's exit status
waitpid(childpid, &childStatus, WUNTRACED);
printf("\nPID of the child process that was just created: %d.\n", childpid);
if(WIFEXITED(childStatus))
{
printf("PID %d exited normally. Exit number: %d\n", childpid, WEXITSTATUS(childStatus));
}
else if(WIFSTOPPED(childStatus))
{
printf("PID %d was stopped by %d\n", childpid, WSTOPSIG(childStatus));
}
else if(WIFSIGNALED(childStatus))
{
printf("PID %d exited due to signal %d\n.", childpid, WTERMSIG(childStatus));
}
else
{
perror("waitpid");
}
}
Before fork code
signal(SIGCHLD, childHandler);
In childHandler put your waitpid code.
void childHandler(int signum)
{
pid_t childpid;
int childstatus;
while ((childpid = waitpid( -1, &childstatus, WNOHANG)) > 0)
{
if (WIFEXITED(childStatus))
{
printf("PID %d exited normally. Exit number: %d\n", childpid, WEXITSTATUS(childStatus));
}
else
if (WIFSTOPPED(childStatus))
{
printf("PID %d was stopped by %d\n", childpid, WSTOPSIG(childStatus));
}
else
if (WIFSIGNALED(childStatus))
{
printf("PID %d exited due to signal %d\n.", childpid, WTERMSIG(childStatus));
}
else
{
perror("waitpid");
}
}
}
}
You should not use async-unsafe calls like printf inside a signal handler so alter your code to save the status in a global or heap allocated array - you know the size to create from processNum - and print the status info outside the handler.
Also, as currently structured, your parent could end before reaping all the children. Add a counter for the children so that you wait on all of them before the parent exits.
Look into signal SIGCHLD. If you have it blocked, you must unblock it or might instead explicitly check for it.
The purpose of wait is to, well, wait, so the way to solve your problem is to first create all the children, then start waiting for them to terminate.
Here is a program which does that:
// fork
#include <unistd.h>
// wait
#include <sys/types.h>
#include <sys/wait.h>
// exit
#include <stdlib.h>
//printf
#include <stdio.h>
void child( int id)
{
int seed= id;
int x= rand_r( &seed) % 10;
sleep( x);
exit( x);
}
int main( int argc, char *argv[])
{
const int n= 5;
int i;
printf( "creating %d children.\n", n);
for ( i= 0; i < n; ++i) {
pid_t pid= fork();
if ( !pid)
child( i); // does not return
else
printf( "child [0x%x] created.\n", pid);
}
// all the children are created now
// now we wait for them to terminate
printf( "waiting for children to terminate.\n", n);
for ( i= 0; i < n; ++i) {
int result;
pid_t pid= wait( &result);
printf( "child [0x%x] terminated with result [%u].\n", pid, WEXITSTATUS( result));
}
puts( "all children terminated.");
}

Resources