Can someone help me understand how the system handles variables that are set before a process makes a fork() call. Below is a small test program I wrote to try understanding what is going on behind the scenes.
I understand that the current state of a process is "cloned", variables included, at the time of the forking. My thought was, that if I malloc'd a 2D array before calling fork, I would need to free the array both in the parent and the child processes.
As you can see from the results below the sample code, the two values act as if they are totally separate from each other, yet they have the exact same address space. I expected that my final result for tmp would be -4 no matter which process completed first.
I am newer to C and Linux, so could someone explain how this is possible? Perhaps the variable tmp becomes a pointer to a pointer which is distinct in each process? Thanks so much.
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
int main()
{
int tmp = 1;
pid_t forkReturn= fork();
if(!forkReturn) {
/*child*/
tmp=tmp+5;
printf("Value for child %d\n",tmp);
printf("Address for child %p\n",&tmp);
}
else if(forkReturn > 0){
/*parent*/
tmp=tmp-10;
printf("Value for parent %d\n",tmp);
printf("Address for parent %p\n",&tmp);
}
else {
/*Error calling fork*/
printf("Error calling fork);
}
return 0;
}
RESULTS of standard out:
Value for child 6
Address for child 0xbfb478d8
Value for parent -9
Address for parent 0xbfb478d8
It did indeed copy the entire address space, and changing memory in the child process does not affect the parent. The key to understanding this is to remember that a pointer can only point to something in your own process, and the copy happens at a lower level.
However, you should not call malloc() or free() at all in the child of fork. This can deadlock (another thread was in malloc() when you called fork()). The only functions safe to call in the child are the ones also listed as safe for signal handlers. I used to be able to claim this was true only if you wrote multithreaded code; however Apple was kind enough to spawn a background thread in the standard library, so the deadlock is real all the time. The child of fork should never be allowed to drop out of the if block. Call _exit to make sure it doesn't.
Related
I do not know, if this is ok, but it compiles:
typedef struct
{
int fd;
char *str;
int c;
} ARG;
void *ww(void *arg){
ARG *a = (ARG *)arg;
write(a->fd,a->str,a->c);
return NULL;
}
int main (void) {
int fd = open("./smf", O_CREAT|O_WRONLY|O_TRUNC, S_IRWXU);
int ch = fork();
if (ch==0){
ARG *arg; pthread_t p1;
arg->fd = fd;
arg->str = malloc(6);
strcpy(arg->str, "child");
arg->c = 6;
pthread_create( &p1, NULL, ww, arg);
} else {
write(fd, "parent\0", 7);
wait(NULL);
}
return 0;
}
I am wait()int in parent, but I do not know if I should also pthread_join to merge threads or it is implicitly by wait(). However is it even safe to write to the same file in two threads? I run few times and sometimes output was 1) parentchild but sometimes only 2) parent, no other cases - I do not know why child did not write as well when parent wait()s for it. Can someone please explain why these outputs?
You need to call pthread_join() in the child process to avoid potential race conditions during the child process’s exit sequence (for example the child process can otherwise exit before its thread gets a chance to write to the file). Calling pthread_join() in the parent process won’t help,
As for the file, having both processes write to it is safe in the sense that it won’t cause a crash, but the order in which the data is written to the file will be indeterminate since the two processes are executing concurrently.
I do not know, if this is ok, but it compiles:
Without even any warnings? Really? I suppose the code you are compiling must include all the needed headers (else you should have loads of warnings), but if your compiler cannot be persuaded to spot
buggy.c:30:15: warning: ‘arg’ may be used uninitialized in this
function [-Wmaybe-uninitialized]
arg->fd = fd;
^
then it's not worth its salt. Indeed, variable arg is used uninitialized, and your program therefore exhibits undefined behavior.
But even if you fix that, after which the program can be made to compile without warnings, it still is not ok.
I am wait()int in parent, but I do not know if I should also
pthread_join to merge threads or it is implicitly by wait().
The parent process is calling wait(). This waits for a child process to terminate, if there are any. Period. It has no implications for the behavior of the child prior to its termination.
Moreover, in a pthreads program, the main thread is special: when it terminates, the whole program terminates, including all other threads. Your child process therefore suffers from a race condition: the main thread terminates immediately after creating a second thread, without ensuring that the other thread terminates first, so it is undefined what, if any, of the behavior of the second thread is actually performed. To avoid this issue, yes, in the child process, the main thread should join the other one before itself terminating.
However
is it even safe to write to the same file in two threads?
It depends -- both on the circumstances and on what you mean by "safe". POSIX requires the write() function to be thread-safe, but that does not mean that multiple threads or processes writing to the same file cannot still interfere with each other by overwriting each other's output.
Yours is a somewhat special case, however, in that parent and child are writing via the same open file description in the kernel, the child having inherited an association with that from its parent. According to POSIX, then, you should see both processes' output (if any; see above) in the file. POSIX provides no way to predict the order in which those outputs will appear, however.
I run few
times and sometimes output was 1) parentchild but sometimes only 2)
parent, no other cases - I do not know why child did not write as well
when parent wait()s for it. Can someone please explain why these
outputs?
The child process can terminate before its second thread performs its write. In this case you will see only the parent's output, not the child's.
I quote "when a process creates a new process using fork() call, Only the shared memory segments are shared between the parent process and the newly forked child process. Copies of the stack and the heap are made for the newly created process" from "operating system concepts" solutions by Silberschatz.
But when I tried this program out
#include <stdio.h>
#include <sys/types.h>
#define MAX_COUNT 200
void ChildProcess(void); /* child process prototype */
void ParentProcess(void); /* parent process prototype */
void main(void)
{
pid_t pid;
char * x=(char *)malloc(10);
pid = fork();
if (pid == 0)
ChildProcess();
else
ParentProcess();
printf("the address is %p\n",x);
}
void ChildProcess(void)
{
printf(" *** Child process ***\n");
}
void ParentProcess(void)
{
printf("*** Parent*****\n");
}
the result is like:
*** Parent*****
the address is 0x1370010
*** Child process ***
the address is 0x1370010
both parent and child printing the same address which is in heap.
can someone explain me the contradiction here. please clearly state what are all the things shared by the parent and child in memory space.
Quoting myself from another thread.
When a fork() system call is issued, a copy of all the pages
corresponding to the parent process is created, loaded into a separate
memory location by the OS for the child process. But this is not
needed in certain cases. Consider the case when a child executes an
"exec" system call or exits very soon after the fork(). When the
child is needed just to execute a command for the parent process,
there is no need for copying the parent process' pages, since exec
replaces the address space of the process which invoked it with the
command to be executed.
In such cases, a technique called copy-on-write (COW) is used. With
this technique, when a fork occurs, the parent process's pages are not
copied for the child process. Instead, the pages are shared between
the child and the parent process. Whenever a process (parent or child)
modifies a page, a separate copy of that particular page alone is made
for that process (parent or child) which performed the modification.
This process will then use the newly copied page rather than the
shared one in all future references. The other process (the one which
did not modify the shared page) continues to use the original copy of
the page (which is now no longer shared). This technique is called
copy-on-write since the page is copied when some process writes to it.
Also, to understand why these programs appear to be using the same space of memory (which is not the case), I would like to quote a part of the book "Operating Systems: Principles and Practice".
Most modern processors introduce a level of indirection, called
virtual addresses. With virtual addresses, every process's memory
starts at the "same" place, e.g., zero.
Each process thinks that it has the entire machine to itself, although
obviously that is not the case in reality.
So these virtual addresses are translations of physical addresses and doesn't represent the same physical memory space, to leave a more practical example we can do a test, if we compile and run multiple times a program that displays the direction of a static variable, such as this program.
#include <stdio.h>
int main() {
static int a = 0;
printf("%p\n", &a);
getchar();
return 0;
}
It would be impossible to obtain the same memory address in two
different programs if we deal with the physical memory directly.
And the results obtained from running the program several times are...
Yes, both processes are using the same address for this variable, but these addresses are used by different processes, and therefore aren't in the same virtual address space.
This means that the addresses are the same, but they aren't pointing to the same physical memory. You should read more about virtual memory to understand this.
The address is the same, but the address space is not. Each process has its own address space, so parent's 0x1370010 is not the same as child's 0x1370010.
You're probably running your program on an operating system with virtual memory. After the fork() call, the parent and child have separate address spaces, so the address 0x1370010 is not pointing to the same place. If one process wrote to *x, the other process would not see the change. (In fact those may be the same page of memory, or even the same block in a swap-file, until it's changed, but the OS makes sure that the page is copied as soon as either the parent or the child writes to it, so as far as the program can tell it's dealing with its own copy.)
When the kernel fork()s the process, the copied memory information inherits the same address information since the heap is effectively copied as-is. If addresses were different, how would you update pointers inside of custom structs? The kernel knows nothing about that information so those pointers would then be invalidated. Therefore, the physical address may change (and in fact often will change even during the lifetime of your executable even without fork()ing, but the logical address remains the same.
Yes address in both the case is same. But if you assign different value for x in child process and parent process and then also prints the value of x along with address of x, You will get your answer.
#include <stdio.h>
#include <sys/types.h>
#include <stdlib.h>
#include <unistd.h>
#define MAX_COUNT 200
void ChildProcess(void); /* child process prototype */
void ParentProcess(void); /* parent process prototype */
void main(void)
{
pid_t pid;
int * x = (int *)malloc(10);
pid = fork();
if (pid == 0) {
*x = 100;
ChildProcess();
}
else {
*x = 200;
ParentProcess();
}
printf("the address is %p and value is %d\n", x, *x);
}
void ChildProcess(void)
{
printf(" *** Child process ***\n");
}
void ParentProcess(void)
{
printf("*** Parent*****\n");
}
Output of this will be:
*** Parent*****
the address is 0xf70260 and value is 200
*** Child process ***
the address is 0xf70260 and value is 100
Now, You can see that value is different but address is same. So The address space for both the process is different. These addresses are not actual address but logical address so these could be same for different processes.
#include <stdio.h>
main()
{
int i, n=1;
for(i=0;i<n;i++) {
fork();
printf("Hello!");
}
}
I am confused if I put n=1, it prints Hello 2 times.
If n=2, it prints Hello 8 times
If n=3, it prints Hello 24 times..
and so on..
There's is no single "formula" how it's done as different operating systems do it in different ways. But what fork() does is it makes a copy of the process. Rough steps that are usually involved in that:
Stop current process.
Make a new process, and initialize or copy related internal structures.
Copy process memory.
Copy resources, like open file descriptors, etc.
Copy CPU/FPU registers.
Resume both processes.
you dont only fork the 'main' prozess, you also fork the children!
first itteration:
m -> c1
//then
m -> c2 c1-> c1.1
m -> c3 c1-> c1.1 c2->c2.1 c1.1 -> c1.1.1
for i = ....
write it in down this way:
main
fork
child(1)
fork child(2)
fork child(1.1)
fork child(3)
fork child(1.2)
fork child(2.1)
and so on ...
fork() always makes a perfect copy of the current process -- the only difference is the process ID).
Indeed in most modern operating systems, immediately after fork() the two processes share exactly the same chunk of memory. The OS makes a new copy only when the new process writes to that memory (in a mode of operation called "copy-on-write") -- if neither process ever changes the memory, then they can carry on sharing that chunk of memory, saving overall system memory.
But all of that is hidden from you as a programmer. Conceptually you can think of it as a perfect copy.
That copy includes all the values of variables at the moment the fork() happened. So if the fork() happens inside a for() loop, with i==2, then the new process will also be mid-way through a for() loop, with i==2.
However, the processes do not share i. When one process changes the value of i after the fork(), the other process's i is not affected.
To understand your program's results, modify the program so that you know which process is printing each line, and which iteration it is on.
#include <stdio.h>
# include <sys/types.h>
main() {
int i , n=4;
for(i=0;i<n;i++); {
int pid = getpid();
if(fork() == 0) {
printf("New process forked from %d with i=%d and pid=%d", pid, i, getpid());
}
printf("Hello number %d from pid %d\n", i, getpid());
}
}
Since the timing will vary, you'll get output in different orders, but you should be able to make sense of where all the "Hellos" are coming from, and why there are as many as there are.
When you fork() you create a new child process that has different virtual memory but initially shows in the same physical memory with the father. At that point both processes can only read data from the memory. If they want to write or change anything in that common memory, then they use different physical memory and have write properties to that memory. Therefore when you fork() your child has initially the same i value as its father, but that value changes separately for every child.
Yes, there is a formula:
fork() makes exactly one near-identical copy of the process. It returns twice, however. Once in the original process (the parent, returning the child pid) and once in the child (returning 0).
The following code never ends. Why is that?
#include <sys/types.h>
#include <stdio.h>
#include <unistd.h>
#define SIZE 5
int nums[SIZE] = {0, 1, 2, 3, 4};
int main()
{
int i;
pid_t pid;
pid = vfork();
if(pid == 0){ /* Child process */
for(i = 0; i < SIZE; i++){
nums[i] *= -i;
printf(”CHILD: %d “, nums[i]); /* LINE X */
}
}
else if (pid > 0){ /* Parent process */
wait(NULL);
for(i = 0; i < SIZE; i++)
printf(”PARENT: %d “, nums[i]); /* LINE Y */
}
return 0;
}
Update:
This code is just to illustrate some of the confusions I have regarding to vfork(). It seems like when I use vfork(), the child process doesn't copy the address space of the parent. Instead, it shares the address space. In that case, I would expect the nums array get updated by both of the processes, my question is in what order? How the OS synchronizes between the two?
As for why the code never ends, it is probably because I don't have any _exit() or exec() statement explicitly for exit. Am I right?
UPDATE2:
I just read: 56. Difference between the fork() and vfork() system call?
and I think this article helps me with my first confusion.
The child process from vfork() system call executes in the parent’s
address space (this can overwrite the parent’s data and stack ) which
suspends the parent process until the child process exits.
To quote from the vfork(2) man page:
The vfork() function has the same effect as fork(), except that the behaviour is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(), or returns from the function in which vfork() was called, or calls any other function before successfully calling _exit() or one of the exec family of functions.
You're doing a whole bunch of those things, so you shouldn't expect it to work. I think the real question here is: why you're using vfork() rather than fork()?
Don't use vfork. That's the simplest advice you can get. The only thing that vfork gives you is suspending the parent until the child either calls exec* or _exit. The part about sharing the address space is incorrect, some operating systems do it, other choose not to because it's very unsafe and has caused serious bugs.
Last time I looked at how applications use vfork in reality the absolute majority did it wrong. It was so bad that I threw away the 6 character change that enabled address space sharing on the operating system I was working on at that time. Almost everyone who uses vfork at least leaks memory if not worse.
If you really want to use vfork, don't do anything other than immediately call _exit or execve after it returns in the child process. Anything else and you're entering undefined territory. And I really mean "anything". You start parsing your strings to make arguments for your exec call and you're pretty much guaranteed that something will touch something it's not supposed to touch. And I also mean execve, not some other function from the exec family. Many libc out there do things in execvp, execl, execle, etc. that are unsafe in a vfork context.
What is specifically happening in your example:
If your operating system shares address space the child returning from main means that your environment cleans things up (flush stdout since you called printf, free memory that was allocated by printf and such things). This means that there are other functions called that will overwrite the stack frame the parent was stuck in. vfork returning in the parent returns to a stack frame that has been overwritten and anything can happen, it might not even have a return address on the stack to return to anymore. You first entered undefined behavior country by calling printf, then the return from main brought you into undefined behavior continent and the cleanup run after the return from main made you travel to undefined behavior planet.
From the official specification:
the behavior is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(),
In your program you modify data other than the pid variable, meaning the behavior is undefined.
You also have to call _exit to end the process, or call one of the exec family of functions.
The child must _exit rather than returning from main. If the child returns from main, then the stack frame does not exist for the parent when it returns from vfork.
just call the _exit instead of calling return or insert _exit(0) to the last line in "child process". return 0 calls exit(0) while close the stdout, so when another printf follows, the program crashes.
I have a parent with 5 child processes. I'm wanting to send a random variable to each child process. Each child will square the variable and send it back to the parent and the parent will sum them all together.
Is this even possible? I can't figure it out...
edit: this process would use shared memory.
There are a great number of ways to do this, all involving some form of inter-process communication. Which one you choose will depend on many factors, but some are:
shared memory.
pipes (popen and such).
sockets.
In general, I would probably popen a number of communications sessions in the parent before spawning the children; the parent will know about all five but each child can be configured to use only one.
Shared memory is also a possibility, although you'd probably have to have a couple of values in it per child to ensure communications went smoothly:
a value to store the variable and return value.
a value to store the state (0 = start, 1 = variable ready for child, 2 = variable ready for parent again).
In all cases, you need a way for the children to only pick up their values, not those destined for other children. That may be as simple as adding a value to the shared memory block to store the PID of the child. All children would scan every element in the block but would only process those where the state is 1 and the PID is their PID.
For example:
Main creates shared memory for five children. Each element has state, PID and value.
Main sets all states to "start".
Main starts five children who all attach to the shared memory.
Main stores all their PIDs.
All children start scanning the shared memory for state = "ready for child" and their PID.
Main puts in first element (state = "ready for child", PID = pid1, value = 7).
Main puts in second element (state = "ready for child", PID = pid5, value = 9).
Child pid1 picks up first element, changes value to 49, sets state to "ready for parent"), goes back to monitoring.
Child pid5 picks up second element, changes value to 81, sets state to "ready for parent"), goes back to monitoring.
Main picks up pid5's response, sets that state back to "start.
Main picks up pid1's response, sets that state back to "start.
This gives a measure of parallelism with each child continuously monitoring the shared memory for work it's meant to do, Main places the work there and periodically receives the results.
The nastiest method uses vfork() and lets the different children trample on different parts of memory before exiting; the parent then just adds up the modified bits of memory.
Highly unrecommended - but about the only case I've come across where vfork() might actually have a use.
Just for amusement (mine) I coded this up:
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>
#include <sys/wait.h>
int main(void)
{
int i;
int array[5];
int square[5];
long sum = 0;
srand(time(0));
for (i = 0; i < 5; i++)
{
array[i] = rand();
if (vfork() == 0)
{
square[i] = array[i] * array[i];
execl("/bin/true", "/bin/true", (char *)0);
}
else
wait(0);
}
for (i = 0; i < 5; i++)
{
printf("in: %d; square: %d\n", array[i], square[i]);
sum += square[i];
}
printf("Sum: %d\n", sum);
return(0);
}
This works. The previous trial version using 'exit(0)' in place of 'execl()' did not work; the square array was all zeroes. Example output (32-bit compilation on Solaris 10, SPARC):
in: 22209; square: 493239681
in: 27082; square: 733434724
in: 2558; square: 6543364
in: 17465; square: 305026225
in: 6610; square: 43692100
Sum: 1581936094
Sometimes, the sum overflows - there is much room for improvement in the handling of that.
The Solaris manual page for 'vfork()' says:
Unlike with the fork() function, the child process borrows
the parent's memory and thread of control until a call to
execve() or an exit (either abnormally or by a call to
_exit() (see exit(2)). Any modification made during this
time to any part of memory in the child process is reflected
in the parent process on return from vfork(). The parent
process is suspended while the child is using its resources.
That probably means the 'wait()' is unnecessary in my code. (However, trying to simplify the code seemed to make it behave indeterminately. It is rather crucial that i does not change prematurely; the wait() does ensure that synchronicity. Using _exit() instead of execl() also seemed to break things. Don't use vfork() if you value your sanity - or if you want any marks for your homework.)
Things like the anti thread might make this a little easier for you, see the examples (in particular the ns lookup program).