Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
How memory is shared in following scenarios?
Between Parent and child Processes
Between two irrelevant Processes
In which part of the physical memory does the shared memory (or) any other IPC used for communicating between processes exists?
Here it the program with explanation of Memory management between Parent and Child Process..
/*
SHARING MEMORY BETWEEN PROCESSES
In this example, we show how two processes can share a common
portion of the memory. Recall that when a process forks, the
new child process has an identical copy of the variables of
the parent process. After fork the parent and child can update
their own copies of the variables in their own way, since they
dont actually share the variable. Here we show how they can
share memory, so that when one updates it, the other can see
the change.
*/
#include <stdio.h>
#include <sys/ipc.h>
#include <sys/shm.h> /* This file is necessary for using shared
memory constructs
*/
main()
{
int shmid. status;
int *a, *b;
int i;
/*
The operating system keeps track of the set of shared memory
segments. In order to acquire shared memory, we must first
request the shared memory from the OS using the shmget()
system call. The second parameter specifies the number of
bytes of memory requested. shmget() returns a shared memory
identifier (SHMID) which is an integer. Refer to the online
man pages for details on the other two parameters of shmget()
*/
shmid = shmget(IPC_PRIVATE, 2*sizeof(int), 0777|IPC_CREAT);
/* We request an array of two integers */
/*
After forking, the parent and child must "attach" the shared
memory to its local data segment. This is done by the shmat()
system call. shmat() takes the SHMID of the shared memory
segment as input parameter and returns the address at which
the segment has been attached. Thus shmat() returns a char
pointer.
*/
if (fork() == 0) {
/* Child Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer b. */
b = (int *) shmat(shmid, 0, 0);
for( i=0; i< 10; i++) {
sleep(1);
printf("\t\t\t Child reads: %d,%d\n",b[0],b[1]);
}
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(b);
}
else {
/* Parent Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer a.
Thus the memory locations a[0] and a[1] of the parent
are the same as the memory locations b[0] and b[1] of
the parent, since the memory is shared.
*/
a = (int *) shmat(shmid, 0, 0);
a[0] = 0; a[1] = 1;
for( i=0; i< 10; i++) {
sleep(1);
a[0] = a[0] + a[1];
a[1] = a[0] + a[1];
printf("Parent writes: %d,%d\n",a[0],a[1]);
}
wait(&status);
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(a);
/* Child has exited, so parent process should delete
the cretaed shared memory. Unlike attach and detach,
which is to be done for each process separately,
deleting the shared memory has to be done by only
one process after making sure that noone else
will be using it
*/
shmctl(shmid, IPC_RMID, 0);
}
}
/*
POINTS TO NOTE:
In this case we find that the child reads all the values written
by the parent. Also the child does not print the same values
again.
1. Modify the sleep in the child process to sleep(2). What
happens now?
2. Restore the sleep in the child process to sleep(1) and modify
the sleep in the parent process to sleep(2). What happens now?
Thus we see that when the writer is faster than the reader, then
the reader may miss some of the values written into the shared
memory. Similarly, when the reader is faster than the writer, then
the reader may read the same values more than once. Perfect
i /*
SHARING MEMORY BETWEEN PROCESSES
In this example, we show how two processes can share a common
portion of the memory. Recall that when a process forks, the
new child process has an identical copy of the variables of
the parent process. After fork the parent and child can update
their own copies of the variables in their own way, since they
dont actually share the variable. Here we show how they can
share memory, so that when one updates it, the other can see
the change.
*/
#include <stdio.h>
#include <sys/ipc.h>
#include <sys/shm.h> /* This file is necessary for using shared
memory constructs
*/
main()
{
int shmid. status;
int *a, *b;
int i;
/*
The operating system keeps track of the set of shared memory
segments. In order to acquire shared memory, we must first
request the shared memory from the OS using the shmget()
system call. The second parameter specifies the number of
bytes of memory requested. shmget() returns a shared memory
identifier (SHMID) which is an integer. Refer to the online
man pages for details on the other two parameters of shmget()
*/
shmid = shmget(IPC_PRIVATE, 2*sizeof(int), 0777|IPC_CREAT);
/* We request an array of two integers */
/*
After forking, the parent and child must "attach" the shared
memory to its local data segment. This is done by the shmat()
system call. shmat() takes the SHMID of the shared memory
segment as input parameter and returns the address at which
the segment has been attached. Thus shmat() returns a char
pointer.
*/
if (fork() == 0) {
/* Child Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer b. */
b = (int *) shmat(shmid, 0, 0);
for( i=0; i< 10; i++) {
sleep(1);
printf("\t\t\t Child reads: %d,%d\n",b[0],b[1]);
}
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(b);
}
else {
/* Parent Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer a.
Thus the memory locations a[0] and a[1] of the parent
are the same as the memory locations b[0] and b[1] of
the parent, since the memory is shared.
*/
a = (int *) shmat(shmid, 0, 0);
a[0] = 0; a[1] = 1;
for( i=0; i< 10; i++) {
sleep(1);
a[0] = a[0] + a[1];
a[1] = a[0] + a[1];
printf("Parent writes: %d,%d\n",a[0],a[1]);
}
wait(&status);
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(a);
/* Child has exited, so parent process should delete
the cretaed shared memory. Unlike attach and detach,
which is to be done for each process separately,
deleting the shared memory has to be done by only
one process after making sure that noone else
will be using it
*/
shmctl(shmid, IPC_RMID, 0);
}
}
/*
POINTS TO NOTE:
In this case we find that the child reads all the values written
by the parent. Also the child does not print the same values
again.
1. Modify the sleep in the child process to sleep(2). What
happens now?
2. Restore the sleep in the child process to sleep(1) and modify
the sleep in the parent process to sleep(2). What happens now?
Thus we see that when the writer is faster than the reader, then
the reader may miss some of the values written into the shared
memory. Similarly, when the reader is faster than the writer, then
the reader may read the same values more than once. Perfect
inter-process communication requires synchronization between the
reader and the writer. You can use semaphores to do this.
Further note that "sleep" is not a synchronization construct.
We use "sleep" to model some amount of computation which may
exist in the process in a real world application.
Also, we have called the different shared memory related
functions such as shmget, shmat, shmdt, and shmctl, assuming
that they always succeed and never fail. This is done to
keep this proram simple. In practice, you should always check for
the return values from this function and exit if there is
an error.
*/nter-process communication requires synchronization between the
reader and the writer. You can use semaphores to do this.
Further note that "sleep" is not a synchronization construct.
We use "sleep" to model some amount of computation which may
exist in the process in a real world application.
Also, we have called the different shared memory related
functions such as shmget, shmat, shmdt, and shmctl, assuming
that they always succeed and never fail. This is done to
keep this proram simple. In practice, you should always check for
the return values from this function and exit if there is
an error.
*/
Related
I'm working on a program that stores a struct in a shared memory segment so I can use the data in the main process and multiple thread simultaneously. I'm using also mutex semaphores (pthread_mutex_lock() / pthread_mutex_unlock()) to guarantee mutual exclusion between processes that want to access that shared memory segment.
Firstly, a thread locks the struct and fills it with information:
void * client_thread(void *argc) {
thread_input *t_input = (thread_input *) argc; //struct that stores data usefull for the thread such as the shm segment
shm_data *shmdata; //<-- shared struct
shmdata = shmat(t_input->memid, 0, 0);
//...
pthread_mutex_lock(t_input->mtx);
shmdata->station = malloc((strlen(data.station)+1) * sizeof(char));
strcpy(shmdata->station, data.station);
shmdata->temperature = data.temperature;
shmdata->humidity = data.humidity;
shmdata->pressure = data.pressure;
shmdata->precipitation = data.precipitation;
pthread_mutex_unlock(t_input->mtx);
//...
}
This works fine. Then, using a synchronization semaphore, main process tries to access that shared memory segment of a struct, for example trying to print first variable. But it gets blocked in that line:
Main
//...
shm_data *shmdata; //<-- shared struct
shmdata = shmat(memid, 0, 0);
pthread_mutex_lock(&mtx);
write(1, "Station:\n", strlen("Station::")); //Prints fine
write(1, shmdata->station, strlen(shmdata->station)); //<-- Program blocks here
write(1, "Done!\n", strlen("Done!!")); //Program does not reach this point or any further
pthread_mutex_unlock(&mtx);
//...
I have tested that the memory id in both thread and main process it's the same and I guarantee that thread code will execute first and then the main process. Why does the program get blocked there?
In this C program, data is not being shared between process i.e. parent and child process. child has it's own data and parent has it's own data but pointer is showing the same address for both processes. How it is being done on background? Does fork generates copies of same data? if so then we have the same pointer's address for both processes. Or is it due to the statically allocated data that is being copied for each process and the data is independent for each process? I want to know how it is being done?
#include<stdio.h>
#include<sys/types.h>
#include<unistd.h>
int main()
{
//Static Array
int X[] = {1,2,3,4,5};
int i, status;
//The fork call
int pid = fork();
if(pid == 0) //Child process
{
//Child process modifies Array
for(i=0; i<5; i++)
X[i] = 5-i;
//Child prints Array
printf("Child Array:\t");
for(i=0; i<5; i++)
printf("%d\t", X[i]);
printf("\nArray ptr = %p\n", X);
}
else //Parent process
{
// Wait for the child to terminate and let
// it modify and print the array
waitpid(-1, &status, 0);
//Parent prints Array
printf("Parent Array:\t");
for(i=0; i<5; i++)
printf("%d\t", X[i]);
printf("\nArray ptr = %p\n", X);
}
return 0;
}
Here is the output of the program.
1 Child Array: 5 4 3 2 1
2 Array ptr = 0x7fff06c9f670
3 Parent Array: 1 2 3 4 5
4 Array ptr = 0x7fff06c9f670
When child process modifies array it should have also modified the data of parent process. What is going on in background?
When you fork a new process the new child process is a copy of the parent process. That's why pointers etc. are equal. Due to the wonders of virtual memory two processes can have the same memory map, but still be using different memory.
fork creates exact copy of the parent process memory image (exception see in man page). This is called Copy On Write(COW) fork. Upto time child only read data, both parent and child have same copy of data but when child write, a new copy is generated and then both child and parent have different copyies for their own data
fork() creates a copy of the calling process, including all the memory allocated to it.
Each process has its own address space and the values of pointers are within context of that address space. So printing the address of some variable in the original process will give the same output as printing that address in the spawned process.
However, as far as the operating system is concerned, the addresses are not equal. The operating system takes care of ensuring each process has the illusion of its own memory.
There are means of sharing memory between processes (i.e. what one process writes to the shared memory, the other one sees). However, that is not what happens by default, and still happens with the help of the host operating system.
So I have a very high data acquisition rate of 16MB/s. I am reading 4MB of data into a buffer from a device file and then processing it. However, this method of writing then reading was to slow for the project. I would like to implement a double buffer in C.
In order to simplify my idea of the double buffer I decided not to include reading from a device file for simplicity. What I have created is a C program that spawns two separate threads readThread and writeThread. I made readThread call my swap function that swaps the pointers of the buffers.
This implementation is terrible because I am using shared memory outside of the Mutex. I am actually slightly embarrassed to post it, but it will at least give you an idea of what I am trying to do. However, I can not seem to come up with a practical way of reading and writing to separate buffers at the same time then calling a swap once both threads finished writing and reading.
Can someone please tell me if its possible to implement double buffering and give me an idea of how to use signals to control when the threads read and write?
Note that readToBuff (dumb name I know) and writeToBuff aren't actually doing anything at present they have blank functions.
Here is my code:
#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>
pthread_t writeThread;
pthread_t readThread;
pthread_mutex_t buffer_mutex;
char buff1[4], buff2[4];
struct mutex_shared {
int stillReading, stillWriting, run_not_over;
char *writeBuff, *readBuff;
} SHARED;
void *writeToBuff(void *idk) {
while(!SHARED.run_not_over) {
SHARED.stillWriting = 1;
for(int i = 0; i < 4; i++) {
}
SHARED.stillWriting = 0;
while(SHARED.stillReading){};
}
printf("hello from write\n");
return NULL;
}
void *readToBuff(void *idk) {
while(!SHARED.run_not_over) {
SHARED.stillReading = 1;
for(int i = 0; i < 4; i++) {
}
while(SHARED.stillWriting){};
swap(writeThread,readThread);
}
printf("hello from read");
return NULL;
}
void swap(char **a, char **b){
pthread_mutex_lock(&buffer_mutex);
printf("in swap\n");
char *temp = *a;
*a = *b;
*b = temp;
SHARED.stillReading = 0;
//SHARED.stillWriting = 0;
pthread_mutex_unlock(&buffer_mutex);
}
int main() {
SHARED.writeBuff = buff1;
SHARED.readBuff = buff2;
printf("buff1 address %p\n", (void*) &buff1);
printf("buff2 address %p\n", (void*) &buff2);
printf("writeBuff address its pointing to %p\n", SHARED.writeBuff);
printf("readBuff address its pointing to %p\n", SHARED.readBuff);
swap(&SHARED.writeBuff,&SHARED.readBuff);
printf("writeBuff address its pointing to %p\n", SHARED.writeBuff);
printf("readBuff address its pointing to %p\n", SHARED.readBuff);
pthread_mutex_init(&buffer_mutex,NULL);
printf("Creating Write Thread\n");
if (pthread_create(&writeThread, NULL, writeToBuff, NULL)) {
printf("failed to create thread\n");
return 1;
}
printf("Thread created\n");
printf("Creating Read Thread\n");
if(pthread_create(&readThread, NULL, readToBuff, NULL)) {
printf("failed to create thread\n");
return 1;
}
printf("Thread created\n");
pthread_join(writeThread, NULL);
pthread_join(readThread, NULL);
exit(0);
}
Using a pair of semaphores seems like it would be easier. Each thread has it's own semaphore to indicate that a buffer is ready to be read into or written from, and each thread has it's own index into a circular array of structures, each containing a pointer to buffer and buffer size. For double buffering, the circular array only contains two structures.
The initial state sets the read thread's semaphore count to 2, the read index to the first buffer, the write threads semaphore count to 0, and the write index to the first buffer. The write thread is then created which will immediately wait on its semaphore.
The read thread waits for non-zero semaphore count (sem_wait) on its semaphore, reads into a buffer, sets the buffer size, increments the write threads semaphore count (sem_post) and "advances" it's index to the circular array of structures.
The write thread waits for non-zero semaphore count (sem_wait) on its semaphore, writes from a buffer (using the size set by read thread), increments the read threads semaphore count (sem_post) and "advances" it's index to the circular array of structures.
When reading is complete, the read thread sets a structure's buffer size to zero to indicate the end of the read chain, then waits for the write thread to "return" all buffers.
The circular array of structures could include more than just 2 structures, allowing for more nesting of data.
I've had to use something similar for high speed data capture, but in this case, the input stream was faster than a single hard drive, so two hard drives were used, and the output alternated between two write threads. One write thread operated on the "even" buffers, the other on the "odd" buffers.
In the case of Windows, with it's WaitForMultipleObjects() (something that just about every operating system other than Posix has), each thread can use a mutex and a semaphore, along with its own linked list based message queue. The mutex controls queue ownership for queue updates, the semaphore indicates number of items pending on a queue. For retrieving a message, a single atomic WaitForMultipleObjects() waits for a mutex and a non-zero semaphore count, and when both have occurred, decrements the semaphore count and unblocks the thread. A message sender, just needs a WaitForObject() on the mutex to update another threads message queue, then posts (releases) the threads semaphore and releases the mutex. This eliminates any priority issues between threads.
I quote "when a process creates a new process using fork() call, Only the shared memory segments are shared between the parent process and the newly forked child process. Copies of the stack and the heap are made for the newly created process" from "operating system concepts" solutions by Silberschatz.
But when I tried this program out
#include <stdio.h>
#include <sys/types.h>
#define MAX_COUNT 200
void ChildProcess(void); /* child process prototype */
void ParentProcess(void); /* parent process prototype */
void main(void)
{
pid_t pid;
char * x=(char *)malloc(10);
pid = fork();
if (pid == 0)
ChildProcess();
else
ParentProcess();
printf("the address is %p\n",x);
}
void ChildProcess(void)
{
printf(" *** Child process ***\n");
}
void ParentProcess(void)
{
printf("*** Parent*****\n");
}
the result is like:
*** Parent*****
the address is 0x1370010
*** Child process ***
the address is 0x1370010
both parent and child printing the same address which is in heap.
can someone explain me the contradiction here. please clearly state what are all the things shared by the parent and child in memory space.
Quoting myself from another thread.
When a fork() system call is issued, a copy of all the pages
corresponding to the parent process is created, loaded into a separate
memory location by the OS for the child process. But this is not
needed in certain cases. Consider the case when a child executes an
"exec" system call or exits very soon after the fork(). When the
child is needed just to execute a command for the parent process,
there is no need for copying the parent process' pages, since exec
replaces the address space of the process which invoked it with the
command to be executed.
In such cases, a technique called copy-on-write (COW) is used. With
this technique, when a fork occurs, the parent process's pages are not
copied for the child process. Instead, the pages are shared between
the child and the parent process. Whenever a process (parent or child)
modifies a page, a separate copy of that particular page alone is made
for that process (parent or child) which performed the modification.
This process will then use the newly copied page rather than the
shared one in all future references. The other process (the one which
did not modify the shared page) continues to use the original copy of
the page (which is now no longer shared). This technique is called
copy-on-write since the page is copied when some process writes to it.
Also, to understand why these programs appear to be using the same space of memory (which is not the case), I would like to quote a part of the book "Operating Systems: Principles and Practice".
Most modern processors introduce a level of indirection, called
virtual addresses. With virtual addresses, every process's memory
starts at the "same" place, e.g., zero.
Each process thinks that it has the entire machine to itself, although
obviously that is not the case in reality.
So these virtual addresses are translations of physical addresses and doesn't represent the same physical memory space, to leave a more practical example we can do a test, if we compile and run multiple times a program that displays the direction of a static variable, such as this program.
#include <stdio.h>
int main() {
static int a = 0;
printf("%p\n", &a);
getchar();
return 0;
}
It would be impossible to obtain the same memory address in two
different programs if we deal with the physical memory directly.
And the results obtained from running the program several times are...
Yes, both processes are using the same address for this variable, but these addresses are used by different processes, and therefore aren't in the same virtual address space.
This means that the addresses are the same, but they aren't pointing to the same physical memory. You should read more about virtual memory to understand this.
The address is the same, but the address space is not. Each process has its own address space, so parent's 0x1370010 is not the same as child's 0x1370010.
You're probably running your program on an operating system with virtual memory. After the fork() call, the parent and child have separate address spaces, so the address 0x1370010 is not pointing to the same place. If one process wrote to *x, the other process would not see the change. (In fact those may be the same page of memory, or even the same block in a swap-file, until it's changed, but the OS makes sure that the page is copied as soon as either the parent or the child writes to it, so as far as the program can tell it's dealing with its own copy.)
When the kernel fork()s the process, the copied memory information inherits the same address information since the heap is effectively copied as-is. If addresses were different, how would you update pointers inside of custom structs? The kernel knows nothing about that information so those pointers would then be invalidated. Therefore, the physical address may change (and in fact often will change even during the lifetime of your executable even without fork()ing, but the logical address remains the same.
Yes address in both the case is same. But if you assign different value for x in child process and parent process and then also prints the value of x along with address of x, You will get your answer.
#include <stdio.h>
#include <sys/types.h>
#include <stdlib.h>
#include <unistd.h>
#define MAX_COUNT 200
void ChildProcess(void); /* child process prototype */
void ParentProcess(void); /* parent process prototype */
void main(void)
{
pid_t pid;
int * x = (int *)malloc(10);
pid = fork();
if (pid == 0) {
*x = 100;
ChildProcess();
}
else {
*x = 200;
ParentProcess();
}
printf("the address is %p and value is %d\n", x, *x);
}
void ChildProcess(void)
{
printf(" *** Child process ***\n");
}
void ParentProcess(void)
{
printf("*** Parent*****\n");
}
Output of this will be:
*** Parent*****
the address is 0xf70260 and value is 200
*** Child process ***
the address is 0xf70260 and value is 100
Now, You can see that value is different but address is same. So The address space for both the process is different. These addresses are not actual address but logical address so these could be same for different processes.
I have a doubt regarding the program copy-pasted below. I am explaining my understanding of the program here: In this program, the parent creates a child and waits for it complete. Then, the child creates a thread and waits for it to complete. So, at the end, the variable 'value' will have the value '5' in child process and '0' in parent process since there are in fact two copies of variable 'value' one in parent and other other in child (since fork essentially copies the address space of parent to child). However, addresses of the variable 'value' in both parent and child are turning out to be the same. I don't understand how. I will be grateful if someone can explain this behaviour.
#include <stdio.h>
#include <pthread.h>
int value = 0;
void *runner( void *param );
int main ()
{
int pid ;
pthread_t tid;
pthread_attr_t attr;
pid = fork();
if( pid == 0 ) /* child */
{
pthread_attr_init( &attr );
pthread_create( &tid, &attr, runner, NULL );
pthread_join(tid, NULL);
printf( "CHILD: value = %d, address = %p\n", value, &value );
}
else if( pid > 0 ) /* Parent */
{
wait(NULL);
printf( "PARENT: value = %d, address = %p\n", value, &value );
}
}
void *runner( void *param )
{
value = 5;
pthread_exit(0);
}
Modern operating systems provide a virtual address space for each process, so a coinciding address doesn't mean that the two variables are stored at the same destination on physical memory.
Moreover, most operating systems use the copy-on-write technique when forking. This means that parts of the address space of the parent process are not copied to the child's address space until the child attempts to change them.
Desktop CPUs, and many embedded CPUs have something called a Memory Management Unit (MMU). An MMU translates from virtual addresses to physical addresses, so each process runs on its own virtual address space, separated from other processes.
An MMU allows the operating system to use some important techniques, like on-demand paging, in addition to the separation between processes mentioned above.
Efficient implementation of fork() requires the use of an MMU: as you have just discovered, the parent and child process use the same virtual addresses, but in different virtual address spaces, thus usually (ignoring memory-mapped files and shared memory) mapping to different physical addresses.