I have a C program that forks a child process at some point in a loop. The child process waits for the parent process to finish its job (some numerical calculations). If things go wrong, the parent process aborts and the child process should continue from the state when it was forked and retry the calculation with some modifications. Otherwise, the parents keeps running, and the child process should be killed.
The communication between the parent and child process is through a memory mapped file, which only has 1 byte as a character that indicates the status of the parent process.
The memory map is done like this
char child_flag[]="W";
fp1 = fopen( "child_interface.dat","wb");
// the interface file has two bytes, but only one is meaningful to the program
fwrite(child_flag, 1, sizeof(child_flag), fp1);
fclose(fp1);
printf("child_interface.dat created\n");
if(mmap_child_flag() ==0) {
printf("memory map of parent-child interface successful.\n");
fflush(stdout);
}
The wait loop in the child process is like this
child_pid = fork();
if (child_pid ==0) { /* child process, wait for parent process to finish*/
mmap_child_flag();
while(child_file[0]=='W' ){ //Child waits
usleep(100000);
}
if(child_file[0]=='R'){ // run child process (as a new parent process)
child_file[0]='W';
goto label2;
}
if(child_file[0]=='K'){ //Kill child process
exit(0);
}
}
The problem is that the child process seems to get stuck in the sleep while loop, even when the parent process has set the status to 'K' (checked in the file that is memory mapped). This code has been run on several linux based super computers, and the behavior seems very inconsistent. On some platforms, it can run smoothly, but on some others, it constantly get stuck in the while loop. Sometimes, if I add some statements inside the while loop after the usleep call, it can then run just fine.
However, I'm not sure if the sleep while loop is the root cause of this problem. My guess is that because the process has almost nothing to do except to check a byte in the memory, the system let it sleep all the time and somehow "forget" to let it check the memory. Can such thing happen in the Linux system?
This the function that does the actual mapping
/* Memory map for parent-child processes interface */
int mmap_child_flag()
{
int fd_child;
struct stat st_child;
// open files
if ((fd_child = open("child_interface.dat", O_RDWR)) == -1){
perror("open child_interface.dat");
exit(1);
}
// stat
if (stat("child_interface.dat", &st_child) == -1){
perror("stat of child_interface.dat");
exit(1);
}
// map, child_file is global char array
child_file = mmap(0, st_child.st_size, PROT_WRITE, MAP_SHARED, fd_child, 0);
if (child_file == (char *)(-1)) {
perror("mmap child_interface.dat");
exit(1);
}
return 0;
}
The problem is that the child process seems to get stuck in the sleep while loop, even when the parent process has set the status to 'K' (checked in the file that is memory mapped).
There are several odd things about your program, with one of them being that you are using shared memory for this task at all. See below for a better approach.
Issues with the current approach
As to the question as it stands, however, you have a synchronization problem. The contents of the mapped memory are being changed outside the scope of the child process, but you've given it no reason to suspect that that might be the case. The compiler can therefore assume that if the wait loop condition is satisfied when it is first evaluated, then it will be satisfied on every subsequent evaluation, too.
For a more complicated interaction, you might need to set up a process-shared mutex or similar to guard access to the shared memory, but for this, it would probably be sufficient to declare child_file as a pointer to volatile char.
A better approach
You want the child to wait for a one- or maybe two-byte instruction from the parent. You presently do this by polling the contents of a shared memory segment, but that's complex to set up and use, as you discovered. It would be a lot easier to use a pipe to convey the needed information from parent to child:
setup: Declare an array. Call pipe().
child use: The child performs a blocking read() on the pipe.
parent use: write() the message to the pipe when ready, then close it. Or just close it.
Note that the pipe itself then provides adequate synchronization, and that there is no need for a wait loop. Note also that the child can detect the case that the parent dies without sending any message, which your shared memory approach does not support.
A shared memory region is good for sharing a lot of data, but it is a bad way to communicate between processes. The reason is that you can't get a notification that something has been changed, nor do you get a notification if the other user of the shared memory died.
To communicate between two processes, use pipe() if you need to create a one-way communication channel, or if you need bidirectional communication, use socketpair(). You can use poll() to wait for the other side to send some data. You will also get notified if the process on the other side terminated.
You were using a loop like this:
while(child_file[0]=='W' ){ //Child waits
usleep(100000);
}
This is bad, since you are wasting on average 50 ms of time that you could have spent doing something useful. Apart from that, there is also the problem that both the compiler and the CPU can sometimes change the order in which things are written to memory. If you have more data in child_file than just the flag at the start, then this might be an issue, unless you use atomics or explicit barriers.
Related
Take the following example:
int main(void)
{
pid_t pid;
pid = fork();
if (pid == 0)
ChildProcess();
else
ParentProcess();
}
So correct me if I am wrong, once fork() executes a child process is created. Now going by this answer fork() returns twice. That is once for the parent process and once for the child process.
Which means that two separate processes come into existence DURING the fork call and not after it ending.
Now I don't get it how it understands how to return 0 for the child process and the correct PID for the parent process.
This where it gets really confusing. This answer states that fork() works by copying the context information of the process and manually setting the return value to 0.
First am I right in saying that the return to any function is placed in a single register? Since in a single processor environment a process can call only one subroutine that returns only one value (correct me if I am wrong here).
Let's say I call a function foo() inside a routine and that function returns a value, that value will be stored in a register say BAR. Each time a function wants to return a value it will use a particular processor register. So if I am able to manually change the return value in the process block I am able to change the value returned to the function right?
So am I correct in thinking that is how fork() works?
How it works is largely irrelevant - as a developer working at a certain level (ie, coding to the UNIX APIs), you really only need to know that it works.
Having said that however, and recognising that curiosity or a need to understand at some depth is generally a good trait to have, there are any number of ways that this could be done.
First off, your contention that a function can only return one value is correct as far as it goes but you need to remember that, after the process split, there are actually two instances of the function running, one in each process. They're mostly independent of each other and can follow different code paths. The following diagram may help in understanding this:
Process 314159 | Process 271828
-------------- | --------------
runs for a bit |
calls fork |
| comes into existence
returns 271828 | returns 0
You can hopefully see there that a single instance of fork can only return one value (as per any other C function) but there are actually multiple instances running, which is why it's said to return multiple values in the documentation.
Here's one possibility on how it could work.
When the fork() function starts running, it stores the current process ID (PID).
Then, when it comes time to return, if the PID is the same as that stored, it's the parent. Otherwise it's the child. Pseudo-code follows:
def fork():
saved_pid = getpid()
# Magic here, returns PID of other process or -1 on failure.
other_pid = split_proc_into_two();
if other_pid == -1: # fork failed -> return -1
return -1
if saved_pid == getpid(): # pid same, parent -> return child PID
return other_pid
return 0 # pid changed, child, return zero
Note that there's a lot of magic in the split_proc_into_two() call and it almost certainly won't work that way at all under the covers(a). It's just to illustrate the concepts around it, which is basically:
get the original PID before the split, which will remain identical for both processes after they split.
do the split.
get the current PID after the split, which will be different in the two processes.
You may also want to take a look at this answer, it explains the fork/exec philosophy.
(a) It's almost certainly more complex than I've explained. For example, in MINIX, the call to fork ends up running in the kernel, which has access to the entire process tree.
It simply copies the parent process structure into a free slot for the child, along the lines of:
sptr = (char *) proc_addr (k1); // parent pointer
chld = (char *) proc_addr (k2); // child pointer
dptr = chld;
bytes = sizeof (struct proc); // bytes to copy
while (bytes--) // copy the structure
*dptr++ = *sptr++;
Then it makes slight modifications to the child structure to ensure it will be suitable, including the line:
chld->p_reg[RET_REG] = 0; // make sure child receives zero
So, basically identical to the scheme I posited, but using data modifications rather than code path selection to decide what to return to the caller - in other words, you'd see something like:
return rpc->p_reg[RET_REG];
at the end of fork() so that the correct value gets returned depending on whether it's the parent or child process.
In Linux fork() happens in kernel; the actual place is the _do_fork here. Simplified, the fork() system call could be something like
pid_t sys_fork() {
pid_t child = create_child_copy();
wait_for_child_to_start();
return child;
}
So in the kernel, fork() really returns once, into the parent process. However the kernel also creates the child process as a copy of the parent process; but instead of returning from an ordinary function, it would synthetically create a new kernel stack for the newly created thread of the child process; and then context-switch to that thread (and process); as the newly created process returns from the context switching function, it would make the child process' thread end up returning to user mode with 0 as the return value from fork().
Basically fork() in userland is just a thin wrapper returns the value that the kernel put onto its stack/into return register. The kernel sets up the new child process so that it returns 0 via this mechanism from its only thread; and the child pid is returned in the parent system call as any other return value from any system call such as read(2) would be.
You first need to know how multitasking works. It is not useful to understand all the details, but every process runs in some kind of a virtual machine controlled by the kernel: a process has its own memory, processor and registers, etc. There is mapping of these virtual objects onto the real ones (the magic is in the kernel), and there is some machinery that swap virtual contexts (processes) to physical machine as time pass.
Then, when the kernel forks a process (fork() is an entry to the kernel), and creates a copy of almost everything in the parent process to the child process, it is able to modify everything needed. One of these is the modification of the corresponding structures to return 0 for the child and the pid of the child in the parent from current call to fork.
Note: nether say "fork returns twice", a function call returns only once.
Just think about a cloning machine: you enter alone, but two persons exit, one is you and the other is your clone (very slightly different); while cloning the machine is able to set a name different than yours to the clone.
The fork system call creates a new process and copies a lot of state from the parent process. Things like the file descriptor table gets copied, the memory mappings and their contents, etc. That state is inside the kernel.
One of the things the kernel keeps track for every process are the values of registers this process needs to have restored at the return from a system call, trap, interrupt or context switch (most context switches happen on system calls or interrupts). Those registers are saved on a syscall/trap/interrupt and then restored when returning to userland. System calls return values by writing into that state. Which is what fork does. Parent fork gets one value, child process a different one.
Since the forked process is different from the parent process, the kernel could do anything to it. Give it any values in registers, give it any memory mappings. To actually make sure that almost everything except the return value is the same as in the parent process requires more effort.
For each running process, the kernel has a table of registers, to load back when a context switch is made. fork() is a system call; a special call that, when made, the process gets a context switch and the kernel code executing the call runs in a different (kernel) thread.
The value returned by system calls is placed in a special register (EAX in x86) that your application reads after the call. When the fork() call is made, the kernel makes a copy of the process, and in each table of registers of each process descriptor writes the appropiate value: 0, and the pid.
I'm using fork() in C to split up the work of running through local arrays, having each process run through half and then multiply the numbers at each point in the arrays and then set the product in a third array.
pid_t pid;
pid = fork();
if (pid == 0){
for (i=1; i<((SIZE/2)+1); i++)
{
output[i] = (one[i] * two[i]);
}
exit(0);
}
else{
wait(NULL);
for (i=((SIZE/2)+1); i<(SIZE+1); i++)
{
output[i] = one[i]*two[i];
}
}
However, when I print the product array after this segment of code i'm only receiving the section set by the parent process, i'm assuming this is because the child process is storing it's values elsewhere in memory which the parent is unable to pick up when printing the product array, but i'm not entirely sure. Thanks in advance for any help.
it seems that you have fork confused with threading.
Forking copies the whole process. Forking isn't like firing off a thread (well it is similar, but threads share the process memory, forking copies the process memory). Changes made after the fork aren't shared between parent or children. If you want to share memory between a parent and child on UNIX while using fork() you need to setup a shared memory segment and put that array within that memory. Lookup shared memory (shmget, smctl) if you want to stick with the fork semantics.
forking has its uses, but is an older, traditional multi-processing API that has in most cases been superseded by multithreading. Forking a new process is much more expensive than creating a new thread, even though fork is optimized on modern OSes that support it. Probably the most common use of fork() is to create a daemon (fork + parent exit) or to execute a command (pipe + fork + exec) as in the implementation of the popen() call.
If using C, you should look into the pthreads API or some other thread library that supports a system thread. Of course, looking at your intended task, you can still use fork, but once you get the hang of threads, it isn't any more complex than using fork with shared memory, unless the algorithm you are implementing is complex.
When you fork, the new child process gets a copy of the parent's address space. It is completely separate. If you need to communicate between parent and child, you will need to use pipes, shared memory, or such.
Note: in any modern Linux, the child's page table is pointing to all of the parent's pages, and both pages table's entries are marked "copy on write". Thus both processes are actually looking at the same physical memory. However, as soon as either process tries to write to a page of memory, it traps and then get's a private copy of the page to modify. From the processes' point of view, it is the same, except that the fork is a lot faster.
So I've been working on code for my OS class, and in my project, I have to create a new child process for each file on the command line, and pipe information from the child to the parent. We're supposed to be re-creating Unix's wc utility (I've handled this part already).
What I've tried thus far is:
for(i=0; i<argcount; i++){
int pid;
pid = fork();
if(pid == 0){
/* Child Process */
/* Close read pipes */
close(l_pipe[0]);
close(w_pipe[0]);
close(c_pipe[0]);
wc(lflag, wflag, cflag, filenames[i]);
} else {
/* Parent Process for piping */
/* Close write pipes */
close(l_pipe[1]);
close(w_pipe[1]);
close(c_pipe[1]);
/* Read from pipes */
read(l_pipe[0], &buffer, sizeof(count_t));
lines+=buffer;
read(w_pipe[0], &buffer, sizeof(count_t));
words+=buffer;
read(c_pipe[0], &buffer, sizeof(count_t));
bytes+=buffer;
}
}
However, this creates as many parents as there are children, which is obviously wrong. I'm not exactly sure where I should be forking. I have to use a pipeline between the child and parent processes, and I'm certain that the parent needs to read() as many times as the child write()s.
Thank you for any suggestions you can provide.
As well as the issues raised by Greg Hewgill in his answer, I observe the following issues:
You could have the loop fork and run the child processes, while the parent part simply goes back to the next iteration of the loop.
You would then have a loop to read from the pipes.
Before that loop, the parent process would close the write ends of all three pipes (otherwise it will never see EOF on the pipes).
The loop to read from the pipes should read from each pipe in turn, rather than draining each pipe in turn.
I'm assuming that you have count_t buffer; — it helps to show variable declarations. If you have some sort of char buffer[sizeof(count_t)] instead, then you have all sorts of problems, large and small.
When the second child is created in your current scheme, the write ends of the pipes are all closed, so the second and subsequent children will be unable to send anything to the parent. You must move the three calls to close the write ends of the pipes so it is outside the loop.
Your pipes must be in global variables so that the wc function can use them. This isn't the end of the world, but it is often neater to avoid global variables. This is a second-order problem; you have other more major problems to fix first.
If you need to associate sizes with individual files, you have more bookkeeping to do. At the moment, you only aggregate grand totals. In that case, your current synchronous design is likely appropriate. Failing that, you'd have the children write a PID or another ID number plus the count on the pipe in one operation (to ensure the operation is atomic). Individual write operations are not interleaved unless they're too big for the pipe's internal buffer, which won't be a problem for a couple of integers.
There's a couple of things I notice straight away.
In your child process branch, you're not calling _exit() after wc(). This means that your child process will loop around and start forking itself for further children.
In your parent process branch, you are waiting for the response from the child you just spawned before proceeding to create the next child. So essentially you've serialised the process and won't take advantage of multiple processes.
In *nix systems, processes are created by using fork() system call. Consider for example, init process creates another process.. First it forks itself and creates the a process which has the context like init. Only on calling exec(), this child process turns out to be a new process. So why is the intermediate step ( of creating a child with same context as parent ) needed? Isn't that a waste of time and resource, because we are creating a context ( consumes time and wastes memory ) and then over writing it?
Why is this not implemented as allocating a vacant memory area and then calling exec()? This would save time and resources right?
The intermediate step enables you to set up shared resources in the child process without the external program being aware of it. The canonical example is constructing a pipe:
// read output of "ls"
// (error checking omitted for brevity)
int pipe_fd[2];
pipe(&pipe_fd);
if (fork() == 0) { // child:
close(pipe_fd[0]); // we don't want to read from the pipe
dup2(pipe_fd[1], 1); // redirect stdout to the write end of the pipe
execlp("ls", "ls", (char *) NULL);
_exit(127); // in case exec fails
}
// parent:
close(pipe_fd[1]);
fp = fdopen(pipe_fd[0], "r");
while (!feof(fp)) {
char line[256];
fgets(line, sizeof line, fp);
...
}
Note how the redirection of standard output to the pipe is done in the child, between fork and exec. Of course, for this simple case, there could be a spawning API that would simply do this automatically, given the proper parameters. But the fork() design enables arbitrary manipulation of per-process resources in the child — one can close unwanted file descriptors, modify per-process limits, drop privileges, manipulate signal masks, and so on. Without fork(), the API for spawning processes would end up either extremely fat or not very useful. And indeed, the process spawning calls of competing operating systems typically fall somewhere in between.
As for the waste of memory, it is avoided with the copy on write technique. fork() doesn't allocate new memory for the child process, but points the child to the parent's memory, with the instructions to make a copy of a page only if the page is ever written to. This makes fork() not only memory-efficient, but also fast, because it only needs to copy a "table of contents".
This is an old complaint. Many people have asked Why fork() first? and typically they suggest an operation that will both create a new process from scratch and run a program in it. This operation is called something like spawn().
And they always say, Won't that be faster?
And in fact, every system other than the Unix family does go the "spawn" way. Only Unix is based on fork() and exec().
But it's funny, Unix has always been much faster than other full-featured systems. It has always handled way more users and load.
And Unix has been made even faster over the years. Fork() no longer really duplicates the address space, it just shares it using a technique called copy-on-write. (A very old fork optimization called vfork() is also still around.)
Drink the Kool-Aid.
I don't know exactly how the init process works on a kernel in terms of forking but to answer you question of why you need to call fork then exec is simply because once you exec there is no turning back.
If you check out the documentation here, it essentially requires a new process to be spawned (the fork call) in order for the parent process to resume control and either wait for it to finish or sit as a daemon probably would.
Only on calling exec(), this child process turns out to be a new
process.
Not really. After a fork, you already have new process, even not that much different from its parent. There are some cases where no exec need to follow a fork.
So why is the intermediate step ( of creating a child with same
context as parent ) needed?
One reason would be because it is an efficient way to create the whole shebang. Cloning is usually less complex than creating from scratch.
Isn't that a waste of time and resource, because we are creating a
context ( consumes time and wastes memory ) and then over writing it?
It is not a waste of time and resource as most of this resource is virtual, due to the copy on write mechanism used. Moreover, it is incorrect to state the created context is overwritten. Nothing is rewritten given the fact nothing was actually written in the first place. That's the whole point of COW. "Only" the process address space (code, heap and stack) are substituted, not overwritten. A lot of the process context is partially or totally preserved, including environment, file descriptors, priority, ignored signals, current and root directory, limits, various masks, processor bindings, privileges and several other things foreign to the process address space.
As I am currently doing this project in only C, I've up untill this point only used my webserver as a single threaded application. However, I dont want that anymore! So I have the following code that handles my Work.
void BeginListen()
{
CreateSocket();
BindSocket();
ListenOnSocket();
while ( 1 )
{
ProcessConnections();
}
}
Now I've added fork(); before the start of ProcessConnection(); which helpes me allowing multiple connections! However, when I add code for daemoning the application found in this answer. I've encounted a little problem, using fork() will create a copy of my whole running app, which is the purpose of fork(). So, I'd like to solve this problem.
My ProcessConnection() looks like this
void ProcessConnections()
{
fork();
addr_size = sizeof(connector);
connecting_socket = accept(current_socket, (struct sockaddr *)&connector, &addr_size);
if ( connecting_socket < 0 )
{
perror("Accepting sockets");
exit(-1);
}
HandleCurrentConnection(connecting_socket);
DisposeCurrentConnection();
}
How would I do to simply just add a couple of lines above or after connecting=socket = accept... in order to make it accept more than one connection at the time? Can i use fork(); but when it comes down to DisposeCurrentConnection(); I want to kill that process and just have the parent-thread running.
I'm not a 100% sure what it is that you're trying to do, buy off the top of my head, I'd prefer to do the fork after the accept, and simply exit() when you're done. Keep in mind though, that you need to react to the SIGCHLD signal when the child process exits, otherwise you'll have a ton of zombie-processes hanging around, waiting to deliver their exit-status to the parent process. C-pseudo-code:
for (;;) {
connecting_socket = accept(server_socket);
if (connecting_socket < 0)
{
if (errno == EINTR)
continue;
else
{
// handle error
break;
}
}
if (! (child_pid = fork ()))
{
// child process, do work with connecting socket
exit (0);
}
else if (child_pid > 0)
{
// parent process, keep track of child_pid if necessary.
}
else
{
// fork failed, unable to service request, send 503 or equivalent.
}
}
The child_pid is needed to (as already mentioned) to kill the child-process, but also if you wish to use waitpid to collect the exit status.
Concerning the zombie-processes, if you're not interested in what happened to the process, you could install a signal hander for SIGCHLD and just loop on waitpid with -1 until it there are no more child-processes, like this
while (-1 != waitpid (-1, NULL, WNOHANG))
/* no loop body */ ;
The waitpid function will return the pid of the child that exited, so if you wish you can correlate this to some other information about the connection (if you did keep track of the pid). Keep in mind that accept will probably exit with errno set to EINTR, without a valid connection if a SIGCHLD is caught, so remember to check for this on accepts return.
EDIT:
Don't forget to check for error conditions, i.e. fork returns -1.
Talking about fork() and threads on unix is not strictly correct. Fork creates a whole new process, which has no shared address space with the parent.
I think you are trying to achieve a process-per-request model, much like a traditional unix web server such as NCSA httpd or Apache 1.x, or possibly build a multi-threaded server with shared global memory:
Process-per-request servers:
When you call fork(), the system creates a clone of the parent process, including file descriptiors. This means that you can accept the socket request and then fork. The child process has the socket request, which it can reply to and then terminate.
This is relatively efficient on unix, as the memory of the process is not physically copied - the pages are shared between the process. The system uses a mechanism called copy-on-write to make copies on a page-by-page basis when the child process writes to memory. Thus, the overhead of a process-per-request server on unix is not that great, and many systems use this architecture.
Better to use select() function which enables u to listen and connect from different
requests in one program.... It avoids blocking but forking creates a new address space
for the copy of the program which leads to memory inefficiency....
select(Max_descr, read_set, write_set, exception_set, time_out);
i.e u can
fd_set* time_out;
fd_set* read_set;
listen(1);
listen(2);
while(1)
{
if(select(20, read_set, NULL,NULL, timeout) >0)
{
accept(1);
accept(2); .....
pthread_create(func);....
}
else
}
Check the return value of fork(). If it is zero, you are the child process, and you can exit() after doing your work. If it is a positive number then it's the process ID of the newly created process. This can let you kill() the child processes if they are hanging around too long for some reason.
As per my comment, this server is not really multi-threaded, it is multi-process.
If you want a simple way to make it accept multiple connections (and you don't care too much about performance) then you can make it work with inetd. This leaves the work of spawning the processes and being a daemon to inetd, and you just need to write a program that handles and processes a single connection. edit: or if this is a programming exercise for you, you could grab the source of inetd and see how it does it
You can also do what you want to do without either threads or new processes, using select.
Here's an article that explains how to use select (pretty low overhead compared to fork or threads - here's an example of a lightweight web server written this way)
Also if you're not wedded to doing this in C, and C++ is OK, you might consider porting your code to use ACE. That is also a good place to look for design patterns of how to do this as I believe it supports pretty much any connection handling model and is very portable.