So I'm just wondering if I had a simple task to do in concurrency, how would I do this with multiple processes using fork() and exec() from a parent process, while locking some aspects of the parent process' memory (so that they don't overwrite each other), but making it available to those processes later?
I know I can do this with POSIX threads with their mutex locks, but what's the process equivalent to that? Is there a way to "lock" shared memory amongst threads? And then would I have to "wait()" for the other threads to finish those locked areas of memory before the other threads could access it?
If you're using the pthreads implementation of mutexes, you would still use them to synchronize between processes... you would place them in shared memory. Initializing a pthread mutex in shared memory addresses this.
You can also use a simple pipe to synchronize access -- pre-fill the pipe with a token and require a successful read of the token to permit resource access. Then write the token back into the pipe in order to release the resource.
First: if you call exec and it succeeds then your process image will be overwritten. You will loose any shared memory and you will need to set it up with your favourite shared memory paradigm (e.g. posix shared memory shm_open).
If you fork then any memory that was mapped shared will remain shared. Means you can place your favourite mutex (e.g. pthread_mutex_t, sem_t) into it and use it with the standard functions that go with it.
void * shared_memory = mmap(
NULL // anywhere
, sysconf(_SC_PAGESIZE) // mmap only works in chunks of pages
// typically 0x1000
, PROT_READ | PROT_WRITE // read-write
, MAP_SHARED // shared
| MAP_ANONYMOUS // anonymous, non-file backed
#ifdef MAP_HASSEMAPHORE
| MAP_HASSEMAPHORE // OS X requires this flag in case you
// intend to have semaphores in that segment
#endif
, -1 // no file backing
, 0
);
if (shared_memory == MAP_FAILED) {
perror("mmap");
abort();
}
// we use that memory to place a mutex tehre
pthread_mutex_t * mutex = shared_memory;
pthread_mutex_init(mutex, NULL);
pid_t pid = fork();
if (pid < 0) {
perror("fork");
abort();
}
if (!pid) {
// child goes here
// use the mutex here
} else {
// parent goes here
// use the mutex here
}
Related
Parent:
shm_id = shmget(IPC_PRIVATE, (1 << 16), IPC_CREAT | IPC_EXCL | 0777);
setenv("SOME_ENV_VAR",stringof(shm_id);
if(fork()=0){
execve(some_path,argv);
}
Child:
int shm_id = atoi(getenv("SOME_ENV_VAR"));
int *shared_mem = (int*)shmat(shm_id,0,NULL);
if(!shared_mem)
return;
shared_mem[0]++;
I want to edit the shared memory in the child. Any reasons why this should not work? I am allocating the shared mem block via shmget in the Parent.Im placing the shm_id as an env variable for the child to read it after the fork and exec.
In the child, I am reading the proper shm_id then trying to get a pointer to the shared memory via shmat. In my code I have verified the shm_id in Parent and Child are the same... Any ideas?
The key_t argument to shmget is not the same as the identifier that that function returns. It’s not sensible to substitute one for the other.
However, if you change that and communicate the shmid instead of the key, your basic approach will work.
The shmid is a system-wide global identifier, and shmat will succeed if you have the appropriate process permissions, even if you are an unrelated process. (And, even if you are related, an execve will detach any shared memory segments, requiring an explicit re-attach.)
Note that the spec is not terribly explicit about this, saying that "[e]ach individual shared memory segment ... shall be identified by a unique positive integer, called ... a shared memory identifier, shmid.".
On OS level segments are identified by the key, the ID is local to a process only. Each process needs to do a get (passing them same key) and an at to use the memory.
An example here: http://www.csl.mtu.edu/cs4411.ck/www/NOTES/process/shm/shmat.html
This question already has answers here:
How do I synchronize access to shared memory in LynxOS/POSIX?
(4 answers)
Closed 6 years ago.
I have two processes that have data in shared memory. This data is going to be updated by both of these process. I was looking for some locking mechanism between two processes. With threads it was easy to have a shared mutex lock. In my case, I tried to save the mutex variable in the shared memory, which then will be used by both processes for locking. This didn't work though. How do I share a mutex between two processes. Some say mutexes cannot be shared, use semaphores. Why mutex cannot be shared but semaphores can be?
It is possible, you have to use the flag PTHREAD_PROCESS_SHARED:
pthread_mutexattr_t mattr;
pthread_mutexattr_init(&mattr);
pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED);
// Init the shared mem barrier
if ( (rv = pthread_mutex_init(&nshared, &mattr)) != 0 ) {
fprintf(stderr, "Failed to initiliaze the shared mutex.\n");
return rv;
}
Where the variable nshared is mapped in shared memory.
Take a look at this documentation. Also, keep in mind that the default value for the mutex is to not share it among processes.
also, take a look at these posts post1 post2
Bonus code to chek the status of the mutex:
void showPshared(pthread_mutexattr_t *mta) {
int rc;
int pshared;
printf("Check pshared attribute\n");
rc = pthread_mutexattr_getpshared(mta, &pshared);
printf("The pshared attributed is: ");
switch (pshared) {
case PTHREAD_PROCESS_PRIVATE:
printf("PTHREAD_PROCESS_PRIVATE\n");
break;
case PTHREAD_PROCESS_SHARED:
printf("PTHREAD_PROCESS_SHARED\n");
break;
default :
printf("! pshared Error !\n");
exit(1);
}
return;
}
I don't remember were I took this piece of code ... found it! here is the source of hal knowledge.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
How memory is shared in following scenarios?
Between Parent and child Processes
Between two irrelevant Processes
In which part of the physical memory does the shared memory (or) any other IPC used for communicating between processes exists?
Here it the program with explanation of Memory management between Parent and Child Process..
/*
SHARING MEMORY BETWEEN PROCESSES
In this example, we show how two processes can share a common
portion of the memory. Recall that when a process forks, the
new child process has an identical copy of the variables of
the parent process. After fork the parent and child can update
their own copies of the variables in their own way, since they
dont actually share the variable. Here we show how they can
share memory, so that when one updates it, the other can see
the change.
*/
#include <stdio.h>
#include <sys/ipc.h>
#include <sys/shm.h> /* This file is necessary for using shared
memory constructs
*/
main()
{
int shmid. status;
int *a, *b;
int i;
/*
The operating system keeps track of the set of shared memory
segments. In order to acquire shared memory, we must first
request the shared memory from the OS using the shmget()
system call. The second parameter specifies the number of
bytes of memory requested. shmget() returns a shared memory
identifier (SHMID) which is an integer. Refer to the online
man pages for details on the other two parameters of shmget()
*/
shmid = shmget(IPC_PRIVATE, 2*sizeof(int), 0777|IPC_CREAT);
/* We request an array of two integers */
/*
After forking, the parent and child must "attach" the shared
memory to its local data segment. This is done by the shmat()
system call. shmat() takes the SHMID of the shared memory
segment as input parameter and returns the address at which
the segment has been attached. Thus shmat() returns a char
pointer.
*/
if (fork() == 0) {
/* Child Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer b. */
b = (int *) shmat(shmid, 0, 0);
for( i=0; i< 10; i++) {
sleep(1);
printf("\t\t\t Child reads: %d,%d\n",b[0],b[1]);
}
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(b);
}
else {
/* Parent Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer a.
Thus the memory locations a[0] and a[1] of the parent
are the same as the memory locations b[0] and b[1] of
the parent, since the memory is shared.
*/
a = (int *) shmat(shmid, 0, 0);
a[0] = 0; a[1] = 1;
for( i=0; i< 10; i++) {
sleep(1);
a[0] = a[0] + a[1];
a[1] = a[0] + a[1];
printf("Parent writes: %d,%d\n",a[0],a[1]);
}
wait(&status);
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(a);
/* Child has exited, so parent process should delete
the cretaed shared memory. Unlike attach and detach,
which is to be done for each process separately,
deleting the shared memory has to be done by only
one process after making sure that noone else
will be using it
*/
shmctl(shmid, IPC_RMID, 0);
}
}
/*
POINTS TO NOTE:
In this case we find that the child reads all the values written
by the parent. Also the child does not print the same values
again.
1. Modify the sleep in the child process to sleep(2). What
happens now?
2. Restore the sleep in the child process to sleep(1) and modify
the sleep in the parent process to sleep(2). What happens now?
Thus we see that when the writer is faster than the reader, then
the reader may miss some of the values written into the shared
memory. Similarly, when the reader is faster than the writer, then
the reader may read the same values more than once. Perfect
i /*
SHARING MEMORY BETWEEN PROCESSES
In this example, we show how two processes can share a common
portion of the memory. Recall that when a process forks, the
new child process has an identical copy of the variables of
the parent process. After fork the parent and child can update
their own copies of the variables in their own way, since they
dont actually share the variable. Here we show how they can
share memory, so that when one updates it, the other can see
the change.
*/
#include <stdio.h>
#include <sys/ipc.h>
#include <sys/shm.h> /* This file is necessary for using shared
memory constructs
*/
main()
{
int shmid. status;
int *a, *b;
int i;
/*
The operating system keeps track of the set of shared memory
segments. In order to acquire shared memory, we must first
request the shared memory from the OS using the shmget()
system call. The second parameter specifies the number of
bytes of memory requested. shmget() returns a shared memory
identifier (SHMID) which is an integer. Refer to the online
man pages for details on the other two parameters of shmget()
*/
shmid = shmget(IPC_PRIVATE, 2*sizeof(int), 0777|IPC_CREAT);
/* We request an array of two integers */
/*
After forking, the parent and child must "attach" the shared
memory to its local data segment. This is done by the shmat()
system call. shmat() takes the SHMID of the shared memory
segment as input parameter and returns the address at which
the segment has been attached. Thus shmat() returns a char
pointer.
*/
if (fork() == 0) {
/* Child Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer b. */
b = (int *) shmat(shmid, 0, 0);
for( i=0; i< 10; i++) {
sleep(1);
printf("\t\t\t Child reads: %d,%d\n",b[0],b[1]);
}
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(b);
}
else {
/* Parent Process */
/* shmat() returns a char pointer which is typecast here
to int and the address is stored in the int pointer a.
Thus the memory locations a[0] and a[1] of the parent
are the same as the memory locations b[0] and b[1] of
the parent, since the memory is shared.
*/
a = (int *) shmat(shmid, 0, 0);
a[0] = 0; a[1] = 1;
for( i=0; i< 10; i++) {
sleep(1);
a[0] = a[0] + a[1];
a[1] = a[0] + a[1];
printf("Parent writes: %d,%d\n",a[0],a[1]);
}
wait(&status);
/* each process should "detach" itself from the
shared memory after it is used */
shmdt(a);
/* Child has exited, so parent process should delete
the cretaed shared memory. Unlike attach and detach,
which is to be done for each process separately,
deleting the shared memory has to be done by only
one process after making sure that noone else
will be using it
*/
shmctl(shmid, IPC_RMID, 0);
}
}
/*
POINTS TO NOTE:
In this case we find that the child reads all the values written
by the parent. Also the child does not print the same values
again.
1. Modify the sleep in the child process to sleep(2). What
happens now?
2. Restore the sleep in the child process to sleep(1) and modify
the sleep in the parent process to sleep(2). What happens now?
Thus we see that when the writer is faster than the reader, then
the reader may miss some of the values written into the shared
memory. Similarly, when the reader is faster than the writer, then
the reader may read the same values more than once. Perfect
inter-process communication requires synchronization between the
reader and the writer. You can use semaphores to do this.
Further note that "sleep" is not a synchronization construct.
We use "sleep" to model some amount of computation which may
exist in the process in a real world application.
Also, we have called the different shared memory related
functions such as shmget, shmat, shmdt, and shmctl, assuming
that they always succeed and never fail. This is done to
keep this proram simple. In practice, you should always check for
the return values from this function and exit if there is
an error.
*/nter-process communication requires synchronization between the
reader and the writer. You can use semaphores to do this.
Further note that "sleep" is not a synchronization construct.
We use "sleep" to model some amount of computation which may
exist in the process in a real world application.
Also, we have called the different shared memory related
functions such as shmget, shmat, shmdt, and shmctl, assuming
that they always succeed and never fail. This is done to
keep this proram simple. In practice, you should always check for
the return values from this function and exit if there is
an error.
*/
I have a server which is multiprocess and multithreaded. The child processes while handling request updates some statistics. This statistics data is a struct updated by all the child processes. Each of these child processes is again multithreaded. The number of child processes is dynamic(increases or decreases) based on the number of requests.
To synchronize the writes to this stat struct, I use mmap. Here is how the map is initialized.
fd = open(mapfile, O_CREAT|O_RDWR, 0666);
write(fd, dummy, MMAP_FILE_SIZE); //dummy is all zeros
void *addr = mmap(0, sizeof(stat_t), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
close(fd);
if (addr == (void *)-1) {
mapped = 0;
}
else {
gStat = (stat_t*)addr;
}
// here gStat struct is initialized
Also in the code where I manipulate the stats, I use a lock to synchronize across multiple threads within the process.
Now, the issue I am having is that under heavy load, the writes don't seem to synchronize. The statistics are not updated correctly. Under normal load, the stats get incremented correctly.
According to my understanding, If MAP_SHARED is specified, write references change the underlying object and the mapping type is retained across fork(). What am I missing?
I'm attempting the simple task of creating a mutex in shared memory. I have the following code to declare a section of shared memory, and attach it to an int*.
int *mutex;
// allocate shared memory for mutex
if ((shmid2 = shmget(IPC_PRIVATE, 4, IPC_CREAT | 0666)) < 0) {
printf("Could not allocate shared memory for mutex: %d.\n", errno);
exit(errno);
}
if ((mutex = shmat(shmid2, NULL, 0)) == (int*)-1) {
printf("Could not attach shared memory for mutex: %d\n", errno);
exit(errno);
}
// set the mutex to one
mutex[0] = 1;
Now, I attempt to define a critical section, surrounded by locking and unlocking the mutex. (Inside of one of many child processes).
while (*mutex == 0) ;
mutex[0] = 0;
// critical section
...
// end critical section
mutex[0] = 1;
However, I'm finding that this technique does not work, and two child processes can enter the critical section simultaneously, without much issue (it happens very often). So I'm wondering what I can do to fix this, without the use of pthreads.
Your options are:
Use POSIX semaphores instead of trying to implement them yourself with shared-memory spinlocks. See the documentation for semop (2) and related functions for details.
If you must use shared-memory semaphores, you will need to use an atomic compare/exchange. Otherwise, two processes can both simultaneously see *mutex == 0 and set it to 1 at the same time, without "noticing" that the other process is doing the same thing.