I get the idea behind copy-on-write. When I fork, the heap is marked as CoW, and when any process tries to change it, a copy is made. The question is: do I have to free it in a child's process nonetheless? Suppose a parent has a dynamic char *array, then it forks. A child process prints some const char, and exits. The child process did not alter the heap at all. Will there be a memory leak?
edit: My child process prints the array on heap, but doesn't modify it. Valgrind says there is a leak if I do not free that array. No leak/memory errors when I free it.
CoW is just a lazy optimization. You may freely think that fork() always makes the full copy of the process (in terms of memory at least) without any delay. But…
If you did prepare dynamic data chunk to "pass" to fork's child, then after fork you have two processes with two dynamic data chunks: parent and child (both are copies). When child exits, it's copy of memory is reclaimed, but parent should free that chunk right after fork by itself.
To be more clear, here is an example:
char *buf = malloc(123456);
// … fill buf for child …
int res = fork();
if (res == -1) {
fprintf(stderr, "fork failed\n");
exit(EXIT_FAILURE);
}
if (res == 0) {
// this is child process
// … do work with buf …
_Exit(EXIT_SUCCESS); // child reclaims buf by means of exit
}
// this is parent process
free(buf); // we don't need it in parent
// … other parent tasks here …
CoW is also very useful optimization in fork-exec technique, where child does nothing but exec with prepared arguments. exec replaces current process with specified executable image, retaining open descriptors and other things (more in man 2 execve). The only page that is copied after such fork is only current stack frame, making fork-exec very effective.
Some systems also provide vfork, that is very restrictive unfair version of fork, but on systems without CoW that is the only way to vfork-exec efficiently.
First the logical (process centered) view:
When you fork a process, the entire address space is copied into a new process as is. Your heap is essentially duplicated in both processes, and both processes can continue to use it just like one process could if fork() had never been called. Both processes can free an allocation that was done before the fork(), and they must do so if they want to reuse the address range connected to the allocation. CoW mappings are only an optimization that does not change these semantics.
Now the physical (system centered) view:
Your system kernel does not know about data ranges you have allocated using malloc(), it only knows about the memory pages it has allocated to the process at the request of malloc(). When you call fork() it marks all these pages as CoW, and references them from both processes. If any of the two processes writes to any of the CoW pages while the other process still exists, it will trap into the system which copies the entire page. And if one of the processes exits, it will at the very least lower the reference count of these pages, so that they do not have to be copied anymore.
So, what happens when you call free() in the child before exiting?
Well, the free() function will most likely write to the page containing the memory allocation to tell malloc() that the block is available again. This will trap into the system and copy the page, expect this operation to take a microsecond or two. If your parent process calls free() while the child is still alive, the same will happen. However, if your child does not free the page and exits, the kernel will know that it does not have to perform CoW anymore. If the parent frees and reuses the memory region afterwards, no copy needs to be done.
I assume, that what your child does is simply to check for some error condition, and exit immediately if it is met. In that case, the most prudent approach is to forget about calling free() in the child, and let the system do its work.
Related
This is sort of a technical question, maybe you can help me if you know about C and UNIX (or maybe it is a really newbie question!)
A question came up today while analizing some code in our Operative Systems course. We are learning what it means to "fork" a process in UNIX, we already know it creates a copy of the current process parallel to it and they have separate data sections.
But then I thought that maybe, if one creates a variable and a pointer pointing at it before doing fork(), because the pointer stores the memory address of the variable, one could try to modify the value of that variable from the child process by using that pointer.
We tried a code similar to this in class:
#include <stdio.h>
#include <sys/types.h>
#include <stdlib.h>
int main (){
int value = 0;
int * pointer = &value;
int status;
pid_t pid;
printf("Parent: Initial value is %d\n",value);
pid = fork();
switch(pid){
case -1: //Error (maybe?)
printf("Fork error, WTF?\n");
exit(-1);
case 0: //Child process
printf("\tChild: I'll try to change the value\n\tChild: The pointer value is %p\n",pointer);
(*pointer) = 1;
printf("\tChild: I've set the value to %d\n",(*pointer));
exit(EXIT_SUCCESS);
break;
}
while(pid != wait(&status)); //Wait for the child process
printf("Parent: the pointer value is %p\nParent: The value is %d\n",pointer,value);
return 0;
}
If you run it, you'll get something like this:
Parent: Initial value is 0
Child: I'll try to change the value
Child: The pointer value is 0x7fff733b0c6c
Child: I've set the value to 1
Parent: the pointer value is 0x7fff733b0c6c
Parent: The value is 0
It's obvious that the child process didn't affect at all the parent process. Frankly, I was expecting some "segmentation fault" error, because of accessing a not permitted memory address. But what really happened?
Remember, I'm not looking for a way to communicate processes, that's not the point. What I want to know is what did the code do. Inside the child process, the change is visible, so it DID something.
My main hypothesis is that pointers are not absolute to memory, they are relative to the process' stack. But I haven't been able to find an answer (no one in class knew, and googling I just found some questions about process communication) so I'd like to know from you, hopefully someone will know.
Thanks for taking your time reading!
The key here is the concept of a virtual address space.
Modern processors (Say anything newer then a 80386) have a memory management unit which maps from a per process virtual address space to physical memory pages under control of the kernel.
When the kernel sets up a process it creates a set of page table entries for that process that define the physical memory pages to virtual address space mapping, and it is in this virtual address space that the program executes.
Conceptually when you fork, the kernel copies the existing process pages to a new set of physical pages and sets up the new processes page tables so that as far as the new process is concerned it appears to be running in the same virtual memory layout as the original one had, while actually addressing entirely different physical memory.
The detail is more subtle as nobody wants to waste time copying hundreds of MB of data unless such is necessary.
When the process calls fork() the kernel sets up a second set of page table entries (for the new process), but points them at the same physical pages as the original process, it then sets the flag in both sets of pages to make the mmu consider them read only.....
As soon as either process writes to a page, the memory management unit generates a page fault (due to the PTE entry having the read only flag set), and the page fault handler then allocates a new page from physical memory, copies the data over, updates the page table entry and sets the pages back to read/write.
In this way, pages are only actually copied the first time either process tries to make a change to a copy on write page, and the slight of hand goes completely unnoticed by either process.
Regards, Dan.
Logically, the fork()ed process gets its own, independent copy of more or less the whole state of the parent process. That couldn't work if pointers in the child referred to memory belonging to the parent.
The details of how a particular UNIX-like kernel makes that work can vary. Linux implements the child process's memory via copy-on-write pages, which makes fork()ing comparatively cheap relative to other possible implementations. In that case, the child's pointers really do point to the parent process's memory, up until such time that either child or parent tries to modify that memory, at which time a copy is made for the child to use. That all relies on the underlying virtual memory system. Other UNIX and UNIX-like systems can and have done it differently.
The child modified a pointer that is perfectly legal in its address space because it is a copy of its parent. There was no effect on the parent because the memory is not logically shared. Each process gets to go its separate way after the fork.
UNIX has a number of ways of creating shared memory (where one process can modify memory and have that modification seen by another process), but fork is not one of them. And it's a good thing because otherwise, synchronization between the parent and child would be almost impossible.
I'm using fork() in C to split up the work of running through local arrays, having each process run through half and then multiply the numbers at each point in the arrays and then set the product in a third array.
pid_t pid;
pid = fork();
if (pid == 0){
for (i=1; i<((SIZE/2)+1); i++)
{
output[i] = (one[i] * two[i]);
}
exit(0);
}
else{
wait(NULL);
for (i=((SIZE/2)+1); i<(SIZE+1); i++)
{
output[i] = one[i]*two[i];
}
}
However, when I print the product array after this segment of code i'm only receiving the section set by the parent process, i'm assuming this is because the child process is storing it's values elsewhere in memory which the parent is unable to pick up when printing the product array, but i'm not entirely sure. Thanks in advance for any help.
it seems that you have fork confused with threading.
Forking copies the whole process. Forking isn't like firing off a thread (well it is similar, but threads share the process memory, forking copies the process memory). Changes made after the fork aren't shared between parent or children. If you want to share memory between a parent and child on UNIX while using fork() you need to setup a shared memory segment and put that array within that memory. Lookup shared memory (shmget, smctl) if you want to stick with the fork semantics.
forking has its uses, but is an older, traditional multi-processing API that has in most cases been superseded by multithreading. Forking a new process is much more expensive than creating a new thread, even though fork is optimized on modern OSes that support it. Probably the most common use of fork() is to create a daemon (fork + parent exit) or to execute a command (pipe + fork + exec) as in the implementation of the popen() call.
If using C, you should look into the pthreads API or some other thread library that supports a system thread. Of course, looking at your intended task, you can still use fork, but once you get the hang of threads, it isn't any more complex than using fork with shared memory, unless the algorithm you are implementing is complex.
When you fork, the new child process gets a copy of the parent's address space. It is completely separate. If you need to communicate between parent and child, you will need to use pipes, shared memory, or such.
Note: in any modern Linux, the child's page table is pointing to all of the parent's pages, and both pages table's entries are marked "copy on write". Thus both processes are actually looking at the same physical memory. However, as soon as either process tries to write to a page of memory, it traps and then get's a private copy of the page to modify. From the processes' point of view, it is the same, except that the fork is a lot faster.
int *g=NULL;
main()
{
g = malloc(40000000);//allocate 40MB in the parent
fork();
while(1);
}
If I run this program and watch the same in the top, I see the 40MB memory allocated against the parent and also the child.
But in the background I expected the copy on write to prevent the double allocation of memory, because I have not touched it yet..
Comments? Is COW specific to some platforms? How can I test if COW is indeed the way for fork?
Logically the child process has the 40MB of memory allocated to it, it just doesn't get a separate copy until it modifies it.
The COW operation doesn't change the amount of resources that are allocated to a process. Instead it just prevents physically duplicating them until an edit is made at which point the new process needs a separate copy.
Suppose I have a main process running and in its execution it has initialized some pointers and created some instances of a predefined structure.
Now if I fork this main process, is seperate memory allocated for the pointers?And are duplicate instances of the previously existing variables, data structures created for this new process?
As an example of my requirement consider -
struct CKT
{
...
}
main()
{
...Some computations with the structure and other pointers.....
pid_t pid = fork();
if(pid == 0) //child
{
..some more computations with the structure...but I need a
..separate instance of it with all the pointers in it as well..
}
else if(pid > 0) // parent
{
..working with the original instance of the structure..
}
// merging the child process with the parent...
// after reading the data of the child processes structure's data...
// and considering a few cases...
}
Can anyone explain how do I achieve this??
Yes, theorically, the fork system call will duplicate, among other, the stack of the parent. In pratical, otherwise, there is a common method, named copy-on-write, used in that case.
It consists on copy a given parent's memory page only when the child's process is trying to modify this memory space. It allows to reduce the cost of the fork system call.
The one thing which is not copy is the return value of fork: 0 in the child, and the PID of the child in the father.
Yes. It might not copy the memory space of the old process immediately, though. The OS will use copy-on-write where possible, copying each memory page the first time it is modified in either process.
COW is what makes one common use of fork (shortly followed by an exec in the child) efficient. The child process never actually uses most of the memory space inherited from the parent.
The copies in the new process will have exactly the same numeric addresses as they did in the old process, so all the pointers from the old process remain valid in the new process and point to the new process's objects. That's part of the point of virtual memory, it allows different process to refer to different physical memory using the same pointer value.
pointer and memory content both will be duplicated for the fork child.
all kind of data pointers, memory, variable will be duplicate in a separate memory for the child process created with fork. and you could not change pointers neither memory content from process child directly.
but you can change variable of parent process from child process using memory share
Refer to this link to see how to it: How to share memory between process fork()?
Yes, your forked process receives copies of all privately mapped memory (default memory mappings via malloc, calloc, stack frames, global variables)
Your child receives shared copies of all open file descriptors. Means those file descriptors will remain valid and open until both parent and child close them. Seeks on those file descriptors are also shared. If you wish to make a file descriptor child-private then you will have to fdreopen it. Otherwise it is very recommended to close all file descriptors you don't need in children immediately after forking.
Your child will receive the same shared MAP_SHARED mappings of memory. Those will continue to access the same physical memory shared between parent and child. This applies to all shared memory aquired through the shm* family of calls and mmapwith MAP_SHARED.
Your child will not receive any mappings marked with MADV_DONTFORK flag via madvise. Those will become invalid in the child. This is not default behavior and you do not have to worry about it unless explicitly used.
You might get the result you are looking for by using a shared memory segment. Use the mmap system call to create a shared memory segment, and put all your shared structures in that segment. Since you cannot use malloc on this segment (it's returned by the syscall as a pointer to the whole segment), you must copy manually the structures, and do the shared memory usage tracking by yourself.
Perhaps you can allocate your data first locally, then evaluate how much memory is used by them, and do the shared memory allocation with the correct size. It is also possible to reallocate the shared segment to a bigger size, in which case you will have to signal the realloc somehow from one end to the other (maybe by using the first integer pointed by the shared map to store that value?).
man pages:
mmap
munmap
I know about "mmap", but as far as i know if i want to share memory allocated by a parent process and accessed it by a client process i need to create a temporary file.
But this file will continue to exist if the processes die.
I was educated to never leave garbage behind. Both in real life and in programming.
The solution should work on Linux, FreeBSD and Solaris.
This article is a very good starting point for shared memory.
However, I'd recommend that you use a pipe instead generally, so as to avoid race conditions and all the mental overhead of concurrency. When you open a child process, its stdin and stdout are file descriptors that you can read and write to from the parent process.
Mmap usually supports "anonymous" mode which does not create any garbage. It works on LInux, according to man page works on Solaris, I am not sure about FreeBSD - look at the man page and search for MAP_ANON or MAP_ANONYMOUS.
Named Pipes should do for you as mentioned above. You can use two pipes (if possible) :
parentpid_childpid -- Parent writes and child reads
childpid_parentpid -- Child writes and parebt reads.
This works for me. Please mention if you have any special senarios to consider.
Allocate Memory in Parent.
Use this memory both in parent and Child.
Give the responsibility of freeing the memory to the parent.
Make the parent wait (wait system call) on the child.
Parent frees the memory before exiting.
Alternatively to be on safer side, before the child exits check if the parent is alive or not.
If not, free the memory in child itself.But this won't work if there can be multiple child.
You can use first few bits of memory to track the number of process using this memory. Whenever a process starts using memory it increment this count and before exiting it decrements the count.Plus if the count becomes 0 , Free the memory.
There is yet another method.Write a function ownMalloc which sits on top of system malloc (or whichever function you uses). This keeps track of all allocated memory and also the processes which uses it. It periodically goes through the the different chunks allocated and frees the chunk not in use.
Use POSIX shm_open(3) and related functions to map memory not backed by a file. The file descriptor returned by shm_open() should be automatically closed if parent and child cease to exist, but I'm not 100% sure if that's always the case. Maybe someone else can shed more light on that?
Allocate the memory in Parent process, keep on wait till the child process executes or let him do some other tasks. Once the child process is over then it will return to the parent process, Deallocate the memory there.
If the parent process needs to be stopped before child(Child is said to be orphaned in this case) then use the exec() which will start child process as a new process.