I have a C program that forks a child process at some point in a loop. The child process waits for the parent process to finish its job (some numerical calculations). If things go wrong, the parent process aborts and the child process should continue from the state when it was forked and retry the calculation with some modifications. Otherwise, the parents keeps running, and the child process should be killed.
The communication between the parent and child process is through a memory mapped file, which only has 1 byte as a character that indicates the status of the parent process.
The memory map is done like this
char child_flag[]="W";
fp1 = fopen( "child_interface.dat","wb");
// the interface file has two bytes, but only one is meaningful to the program
fwrite(child_flag, 1, sizeof(child_flag), fp1);
fclose(fp1);
printf("child_interface.dat created\n");
if(mmap_child_flag() ==0) {
printf("memory map of parent-child interface successful.\n");
fflush(stdout);
}
The wait loop in the child process is like this
child_pid = fork();
if (child_pid ==0) { /* child process, wait for parent process to finish*/
mmap_child_flag();
while(child_file[0]=='W' ){ //Child waits
usleep(100000);
}
if(child_file[0]=='R'){ // run child process (as a new parent process)
child_file[0]='W';
goto label2;
}
if(child_file[0]=='K'){ //Kill child process
exit(0);
}
}
The problem is that the child process seems to get stuck in the sleep while loop, even when the parent process has set the status to 'K' (checked in the file that is memory mapped). This code has been run on several linux based super computers, and the behavior seems very inconsistent. On some platforms, it can run smoothly, but on some others, it constantly get stuck in the while loop. Sometimes, if I add some statements inside the while loop after the usleep call, it can then run just fine.
However, I'm not sure if the sleep while loop is the root cause of this problem. My guess is that because the process has almost nothing to do except to check a byte in the memory, the system let it sleep all the time and somehow "forget" to let it check the memory. Can such thing happen in the Linux system?
This the function that does the actual mapping
/* Memory map for parent-child processes interface */
int mmap_child_flag()
{
int fd_child;
struct stat st_child;
// open files
if ((fd_child = open("child_interface.dat", O_RDWR)) == -1){
perror("open child_interface.dat");
exit(1);
}
// stat
if (stat("child_interface.dat", &st_child) == -1){
perror("stat of child_interface.dat");
exit(1);
}
// map, child_file is global char array
child_file = mmap(0, st_child.st_size, PROT_WRITE, MAP_SHARED, fd_child, 0);
if (child_file == (char *)(-1)) {
perror("mmap child_interface.dat");
exit(1);
}
return 0;
}
The problem is that the child process seems to get stuck in the sleep while loop, even when the parent process has set the status to 'K' (checked in the file that is memory mapped).
There are several odd things about your program, with one of them being that you are using shared memory for this task at all. See below for a better approach.
Issues with the current approach
As to the question as it stands, however, you have a synchronization problem. The contents of the mapped memory are being changed outside the scope of the child process, but you've given it no reason to suspect that that might be the case. The compiler can therefore assume that if the wait loop condition is satisfied when it is first evaluated, then it will be satisfied on every subsequent evaluation, too.
For a more complicated interaction, you might need to set up a process-shared mutex or similar to guard access to the shared memory, but for this, it would probably be sufficient to declare child_file as a pointer to volatile char.
A better approach
You want the child to wait for a one- or maybe two-byte instruction from the parent. You presently do this by polling the contents of a shared memory segment, but that's complex to set up and use, as you discovered. It would be a lot easier to use a pipe to convey the needed information from parent to child:
setup: Declare an array. Call pipe().
child use: The child performs a blocking read() on the pipe.
parent use: write() the message to the pipe when ready, then close it. Or just close it.
Note that the pipe itself then provides adequate synchronization, and that there is no need for a wait loop. Note also that the child can detect the case that the parent dies without sending any message, which your shared memory approach does not support.
A shared memory region is good for sharing a lot of data, but it is a bad way to communicate between processes. The reason is that you can't get a notification that something has been changed, nor do you get a notification if the other user of the shared memory died.
To communicate between two processes, use pipe() if you need to create a one-way communication channel, or if you need bidirectional communication, use socketpair(). You can use poll() to wait for the other side to send some data. You will also get notified if the process on the other side terminated.
You were using a loop like this:
while(child_file[0]=='W' ){ //Child waits
usleep(100000);
}
This is bad, since you are wasting on average 50 ms of time that you could have spent doing something useful. Apart from that, there is also the problem that both the compiler and the CPU can sometimes change the order in which things are written to memory. If you have more data in child_file than just the flag at the start, then this might be an issue, unless you use atomics or explicit barriers.
In *nix systems, processes are created by using fork() system call. Consider for example, init process creates another process.. First it forks itself and creates the a process which has the context like init. Only on calling exec(), this child process turns out to be a new process. So why is the intermediate step ( of creating a child with same context as parent ) needed? Isn't that a waste of time and resource, because we are creating a context ( consumes time and wastes memory ) and then over writing it?
Why is this not implemented as allocating a vacant memory area and then calling exec()? This would save time and resources right?
The intermediate step enables you to set up shared resources in the child process without the external program being aware of it. The canonical example is constructing a pipe:
// read output of "ls"
// (error checking omitted for brevity)
int pipe_fd[2];
pipe(&pipe_fd);
if (fork() == 0) { // child:
close(pipe_fd[0]); // we don't want to read from the pipe
dup2(pipe_fd[1], 1); // redirect stdout to the write end of the pipe
execlp("ls", "ls", (char *) NULL);
_exit(127); // in case exec fails
}
// parent:
close(pipe_fd[1]);
fp = fdopen(pipe_fd[0], "r");
while (!feof(fp)) {
char line[256];
fgets(line, sizeof line, fp);
...
}
Note how the redirection of standard output to the pipe is done in the child, between fork and exec. Of course, for this simple case, there could be a spawning API that would simply do this automatically, given the proper parameters. But the fork() design enables arbitrary manipulation of per-process resources in the child — one can close unwanted file descriptors, modify per-process limits, drop privileges, manipulate signal masks, and so on. Without fork(), the API for spawning processes would end up either extremely fat or not very useful. And indeed, the process spawning calls of competing operating systems typically fall somewhere in between.
As for the waste of memory, it is avoided with the copy on write technique. fork() doesn't allocate new memory for the child process, but points the child to the parent's memory, with the instructions to make a copy of a page only if the page is ever written to. This makes fork() not only memory-efficient, but also fast, because it only needs to copy a "table of contents".
This is an old complaint. Many people have asked Why fork() first? and typically they suggest an operation that will both create a new process from scratch and run a program in it. This operation is called something like spawn().
And they always say, Won't that be faster?
And in fact, every system other than the Unix family does go the "spawn" way. Only Unix is based on fork() and exec().
But it's funny, Unix has always been much faster than other full-featured systems. It has always handled way more users and load.
And Unix has been made even faster over the years. Fork() no longer really duplicates the address space, it just shares it using a technique called copy-on-write. (A very old fork optimization called vfork() is also still around.)
Drink the Kool-Aid.
I don't know exactly how the init process works on a kernel in terms of forking but to answer you question of why you need to call fork then exec is simply because once you exec there is no turning back.
If you check out the documentation here, it essentially requires a new process to be spawned (the fork call) in order for the parent process to resume control and either wait for it to finish or sit as a daemon probably would.
Only on calling exec(), this child process turns out to be a new
process.
Not really. After a fork, you already have new process, even not that much different from its parent. There are some cases where no exec need to follow a fork.
So why is the intermediate step ( of creating a child with same
context as parent ) needed?
One reason would be because it is an efficient way to create the whole shebang. Cloning is usually less complex than creating from scratch.
Isn't that a waste of time and resource, because we are creating a
context ( consumes time and wastes memory ) and then over writing it?
It is not a waste of time and resource as most of this resource is virtual, due to the copy on write mechanism used. Moreover, it is incorrect to state the created context is overwritten. Nothing is rewritten given the fact nothing was actually written in the first place. That's the whole point of COW. "Only" the process address space (code, heap and stack) are substituted, not overwritten. A lot of the process context is partially or totally preserved, including environment, file descriptors, priority, ignored signals, current and root directory, limits, various masks, processor bindings, privileges and several other things foreign to the process address space.
Suppose I have a main process running and in its execution it has initialized some pointers and created some instances of a predefined structure.
Now if I fork this main process, is seperate memory allocated for the pointers?And are duplicate instances of the previously existing variables, data structures created for this new process?
As an example of my requirement consider -
struct CKT
{
...
}
main()
{
...Some computations with the structure and other pointers.....
pid_t pid = fork();
if(pid == 0) //child
{
..some more computations with the structure...but I need a
..separate instance of it with all the pointers in it as well..
}
else if(pid > 0) // parent
{
..working with the original instance of the structure..
}
// merging the child process with the parent...
// after reading the data of the child processes structure's data...
// and considering a few cases...
}
Can anyone explain how do I achieve this??
Yes, theorically, the fork system call will duplicate, among other, the stack of the parent. In pratical, otherwise, there is a common method, named copy-on-write, used in that case.
It consists on copy a given parent's memory page only when the child's process is trying to modify this memory space. It allows to reduce the cost of the fork system call.
The one thing which is not copy is the return value of fork: 0 in the child, and the PID of the child in the father.
Yes. It might not copy the memory space of the old process immediately, though. The OS will use copy-on-write where possible, copying each memory page the first time it is modified in either process.
COW is what makes one common use of fork (shortly followed by an exec in the child) efficient. The child process never actually uses most of the memory space inherited from the parent.
The copies in the new process will have exactly the same numeric addresses as they did in the old process, so all the pointers from the old process remain valid in the new process and point to the new process's objects. That's part of the point of virtual memory, it allows different process to refer to different physical memory using the same pointer value.
pointer and memory content both will be duplicated for the fork child.
all kind of data pointers, memory, variable will be duplicate in a separate memory for the child process created with fork. and you could not change pointers neither memory content from process child directly.
but you can change variable of parent process from child process using memory share
Refer to this link to see how to it: How to share memory between process fork()?
Yes, your forked process receives copies of all privately mapped memory (default memory mappings via malloc, calloc, stack frames, global variables)
Your child receives shared copies of all open file descriptors. Means those file descriptors will remain valid and open until both parent and child close them. Seeks on those file descriptors are also shared. If you wish to make a file descriptor child-private then you will have to fdreopen it. Otherwise it is very recommended to close all file descriptors you don't need in children immediately after forking.
Your child will receive the same shared MAP_SHARED mappings of memory. Those will continue to access the same physical memory shared between parent and child. This applies to all shared memory aquired through the shm* family of calls and mmapwith MAP_SHARED.
Your child will not receive any mappings marked with MADV_DONTFORK flag via madvise. Those will become invalid in the child. This is not default behavior and you do not have to worry about it unless explicitly used.
You might get the result you are looking for by using a shared memory segment. Use the mmap system call to create a shared memory segment, and put all your shared structures in that segment. Since you cannot use malloc on this segment (it's returned by the syscall as a pointer to the whole segment), you must copy manually the structures, and do the shared memory usage tracking by yourself.
Perhaps you can allocate your data first locally, then evaluate how much memory is used by them, and do the shared memory allocation with the correct size. It is also possible to reallocate the shared segment to a bigger size, in which case you will have to signal the realloc somehow from one end to the other (maybe by using the first integer pointed by the shared map to store that value?).
man pages:
mmap
munmap
In many programs and man pages of Linux, I have seen code using fork(). Why do we need to use fork() and what is its purpose?
fork() is how you create new processes in Unix. When you call fork, you're creating a copy of your own process that has its own address space. This allows multiple tasks to run independently of one another as though they each had the full memory of the machine to themselves.
Here are some example usages of fork:
Your shell uses fork to run the programs you invoke from the command line.
Web servers like apache use fork to create multiple server processes, each of which handles requests in its own address space. If one dies or leaks memory, others are unaffected, so it functions as a mechanism for fault tolerance.
Google Chrome uses fork to handle each page within a separate process. This will prevent client-side code on one page from bringing your whole browser down.
fork is used to spawn processes in some parallel programs (like those written using MPI). Note this is different from using threads, which don't have their own address space and exist within a process.
Scripting languages use fork indirectly to start child processes. For example, every time you use a command like subprocess.Popen in Python, you fork a child process and read its output. This enables programs to work together.
Typical usage of fork in a shell might look something like this:
int child_process_id = fork();
if (child_process_id) {
// Fork returns a valid pid in the parent process. Parent executes this.
// wait for the child process to complete
waitpid(child_process_id, ...); // omitted extra args for brevity
// child process finished!
} else {
// Fork returns 0 in the child process. Child executes this.
// new argv array for the child process
const char *argv[] = {"arg1", "arg2", "arg3", NULL};
// now start executing some other program
exec("/path/to/a/program", argv);
}
The shell spawns a child process using exec and waits for it to complete, then continues with its own execution. Note that you don't have to use fork this way. You can always spawn off lots of child processes, as a parallel program might do, and each might run a program concurrently. Basically, any time you're creating new processes in a Unix system, you're using fork(). For the Windows equivalent, take a look at CreateProcess.
If you want more examples and a longer explanation, Wikipedia has a decent summary. And here are some slides here on how processes, threads, and concurrency work in modern operating systems.
fork() is how Unix create new processes. At the point you called fork(), your process is cloned, and two different processes continue the execution from there. One of them, the child, will have fork() return 0. The other, the parent, will have fork() return the PID (process ID) of the child.
For example, if you type the following in a shell, the shell program will call fork(), and then execute the command you passed (telnetd, in this case) in the child, while the parent will display the prompt again, as well as a message indicating the PID of the background process.
$ telnetd &
As for the reason you create new processes, that's how your operating system can do many things at the same time. It's why you can run a program and, while it is running, switch to another window and do something else.
fork() is used to create child process. When a fork() function is called, a new process will be spawned and the fork() function call will return a different value for the child and the parent.
If the return value is 0, you know you're the child process and if the return value is a number (which happens to be the child process id), you know you're the parent. (and if it's a negative number, the fork was failed and no child process was created)
http://www.yolinux.com/TUTORIALS/ForkExecProcesses.html
fork() is basically used to create a child process for the process in which you are calling this function. Whenever you call a fork(), it returns a zero for the child id.
pid=fork()
if pid==0
//this is the child process
else if pid!=0
//this is the parent process
by this you can provide different actions for the parent and the child and make use of multithreading feature.
fork() will create a new child process identical to the parent. So everything you run in the code after that will be run by both processes — very useful if you have for instance a server, and you want to handle multiple requests.
System call fork() is used to create processes. It takes no arguments and returns a process ID. The purpose of fork() is to create a new process, which becomes the child process of the caller. After a new child process is created, both processes will execute the next instruction following the fork() system call. Therefore, we have to distinguish the parent from the child. This can be done by testing the returned value of fork():
If fork() returns a negative value, the creation of a child process was unsuccessful.
fork() returns a zero to the newly created child process.
fork() returns a positive value, the process ID of the child process, to the parent. The returned process ID is of type pid_t defined in sys/types.h. Normally, the process ID is an integer. Moreover, a process can use function getpid() to retrieve the process ID assigned to this process.
Therefore, after the system call to fork(), a simple test can tell which process is the child. Please note that Unix will make an exact copy of the parent's address space and give it to the child. Therefore, the parent and child processes have separate address spaces.
Let us understand it with an example to make the above points clear. This example does not distinguish parent and the child processes.
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#define MAX_COUNT 200
#define BUF_SIZE 100
void main(void)
{
pid_t pid;
int i;
char buf[BUF_SIZE];
fork();
pid = getpid();
for (i = 1; i <= MAX_COUNT; i++) {
sprintf(buf, "This line is from pid %d, value = %d\n", pid, i);
write(1, buf, strlen(buf));
}
}
Suppose the above program executes up to the point of the call to fork().
If the call to fork() is executed successfully, Unix will make two identical copies of address spaces, one for the parent and the other for the child.
Both processes will start their execution at the next statement following the fork() call. In this case, both processes will start their execution at the assignment
pid = .....;
Both processes start their execution right after the system call fork(). Since both processes have identical but separate address spaces, those variables initialized before the fork() call have the same values in both address spaces. Since every process has its own address space, any modifications will be independent of the others. In other words, if the parent changes the value of its variable, the modification will only affect the variable in the parent process's address space. Other address spaces created by fork() calls will not be affected even though they have identical variable names.
What is the reason of using write rather than printf? It is because printf() is "buffered," meaning printf() will group the output of a process together. While buffering the output for the parent process, the child may also use printf to print out some information, which will also be buffered. As a result, since the output will not be send to screen immediately, you may not get the right order of the expected result. Worse, the output from the two processes may be mixed in strange ways. To overcome this problem, you may consider to use the "unbuffered" write.
If you run this program, you might see the following on the screen:
................
This line is from pid 3456, value 13
This line is from pid 3456, value 14
................
This line is from pid 3456, value 20
This line is from pid 4617, value 100
This line is from pid 4617, value 101
................
This line is from pid 3456, value 21
This line is from pid 3456, value 22
................
Process ID 3456 may be the one assigned to the parent or the child. Due to the fact that these processes are run concurrently, their output lines are intermixed in a rather unpredictable way. Moreover, the order of these lines are determined by the CPU scheduler. Hence, if you run this program again, you may get a totally different result.
You probably don't need to use fork in day-to-day programming if you are writing applications.
Even if you do want your program to start another program to do some task, there are other simpler interfaces which use fork behind the scenes, such as "system" in C and perl.
For example, if you wanted your application to launch another program such as bc to do some calculation for you, you might use 'system' to run it. System does a 'fork' to create a new process, then an 'exec' to turn that process into bc. Once bc completes, system returns control to your program.
You can also run other programs asynchronously, but I can't remember how.
If you are writing servers, shells, viruses or operating systems, you are more likely to want to use fork.
Multiprocessing is central to computing. For example, your IE or Firefox can create a process to download a file for you while you are still browsing the internet. Or, while you are printing out a document in a word processor, you can still look at different pages and still do some editing with it.
Fork creates new processes. Without fork you would have a unix system that could only run init.
Fork() is used to create new processes as every body has written.
Here is my code that creates processes in the form of binary tree.......It will ask to scan the number of levels upto which you want to create processes in binary tree
#include<unistd.h>
#include<fcntl.h>
#include<stdlib.h>
int main()
{
int t1,t2,p,i,n,ab;
p=getpid();
printf("enter the number of levels\n");fflush(stdout);
scanf("%d",&n);
printf("root %d\n",p);fflush(stdout);
for(i=1;i<n;i++)
{
t1=fork();
if(t1!=0)
t2=fork();
if(t1!=0 && t2!=0)
break;
printf("child pid %d parent pid %d\n",getpid(),getppid());fflush(stdout);
}
waitpid(t1,&ab,0);
waitpid(t2,&ab,0);
return 0;
}
OUTPUT
enter the number of levels
3
root 20665
child pid 20670 parent pid 20665
child pid 20669 parent pid 20665
child pid 20672 parent pid 20670
child pid 20671 parent pid 20670
child pid 20674 parent pid 20669
child pid 20673 parent pid 20669
First one needs to understand what is fork () system call. Let me explain
fork() system call creates the exact duplicate of parent process, It makes the duplicate of parent stack, heap, initialized data, uninitialized data and share the code in read-only mode with parent process.
Fork system call copies the memory on the copy-on-write basis, means child makes in virtual memory page when there is requirement of copying.
Now Purpose of fork():
Fork() can be used at the place where there is division of work like a server has to handle multiple clients, So parent has to accept the connection on regular basis, So server does fork for each client to perform read-write.
fork() is used to spawn a child process. Typically it's used in similar sorts of situations as threading, but there are differences. Unlike threads, fork() creates whole seperate processes, which means that the child and the parent while they are direct copies of each other at the point that fork() is called, they are completely seperate, neither can access the other's memory space (without going to the normal troubles you go to access another program's memory).
fork() is still used by some server applications, mostly ones that run as root on a *NIX machine that drop permissions before processing user requests. There are some other usecases still, but mostly people have moved to multithreading now.
The rationale behind fork() versus just having an exec() function to initiate a new process is explained in an answer to a similar question on the unix stack exchange.
Essentially, since fork copies the current process, all of the various possible options for a process are established by default, so the programmer does not have supply them.
In the Windows operating system, by contrast, programmers have to use the CreateProcess function which is MUCH more complicated and requires populating a multifarious structure to define the parameters of the new process.
So, to sum up, the reason for forking (versus exec'ing) is simplicity in creating new processes.
Fork() system call use to create a child process. It is exact duplicate of parent process. Fork copies stack section, heap section, data section, environment variable, command line arguments from parent.
refer: http://man7.org/linux/man-pages/man2/fork.2.html
Fork() was created as a way to create another process with shared a copy of memory state to the parent. It works the way it does because it was the most minimal change possible to get good threading capabilities in time-slicing mainframe systems that previously lacked this capability. Additionally, programs needed remarkably little modification to become multi-process, fork() could simply be added in the appropriate locations, which is rather elegant. Basically, fork() was the path of least resistance.
Originally it actually had to copy the entire parent process' memory space. With the advent of virtual memory, it has been hacked and changed to be more efficient, with copy-on-write mechanisms avoiding the need to actual copy any memory.
However, modern systems now allow the creation of actual threads, which simply share the parent process' actual heap. With modern multi-threading programming paradigms and more advanced languages, it's questionable whether fork() provides any real benefit, since fork() actually prevents processes from communicating through memory directly, and forces them to use slower message passing mechanisms.