Is it safe to use shm_unlink even there are other process currently open the shared memory?
For exmaple:
Process B shm_open and then Process A shm_unlink. Is it ok for Process B?
Yes, it is safe to call shm_unlink if another process has it opened. Per the man page for shm_unlink:
The operation of shm_unlink() is analogous to unlink(2): it removes a
shared memory object name, and, once all processes have unmapped the
object, de-allocates and destroys the contents of the associated
memory region. After a successful shm_unlink(), attempts to shm_open()
an object with the same name will fail (unless O_CREAT was specified,
in which case a new, distinct object is created).
Related
I have the following code snippet :
fd_mem = shm_open(MEM_NAME , O_RDWR | O_CREAT | O_EXCL , 0600);
//Why do we use unlink before having mmaped ?
shm_unlink ( MEM_NAME );
ftruncate (fd_mem , mem_size)
plateau = (char*) mmap(NULL , mem_size , PROT_READ | PROT_WRITE , MAP_SHARED , fd_mem , 0);
My question is: why do we use "unlink" before having mapped the file into the virtual memory of the process? I'm confused as to how shm_unlink() works in that regard. I would think it deletes the file making fd_mem unusable but it doesn't.
I'm confused as to how shm_unlink works in that regard, I would think it deletes the file rendering fd_mem unusable but it doesn't.
shm_unlink() removes the name of the shared memory segment. The segment itself persists as long as any process has it open, but after it is unlinked, processes can no longer open it. Even so, new processes forked from one that holds the shared memory segment inherit that, and one process can copy a file descriptor to another via a UNIX-domain socket, so being unlinked does not inherently limit which or how many processes can access the segment.
This is exactly analogous to the situation with regular files and unlink(). Successfully unlinking a file name removes that name from its directory, but the file itself is not removed as long as any process has it open.
Among the reasons to do this sort of thing are
to ensure that resources are cleaned up, whenever and however a process terminates. Named shared-memory segments persist as long as either they remain linked or a process holds them open. By unlinking a segment immediately after creating and opening it, a process helps ensure that it will not live longer than wanted, even if that process crashes.
to avoid unwanted access. Named shared memory segments can be opened by any process with sufficient privilege. Unlinking a shared memory segment helps control that. It especially helps avoid unwanted shared usage by multiple copies of the same program.
Note also that it is possible to create anonymous shared memory segments, which are never linked to a name in the first place. This is much as if a named segment were created, opened, and immediately unlinked, but it leaves no window in which another process can unwantedly open the segment by name.
Opening a file or a shared memory segment increments a reference counter on the underlying "kernel object". The deletion operation, deletes the name of the object but does not decrement the reference counter. As long the reference counter is bigger than 0, the object is not destroyed.
The deletion of the object after opening it, is for the automatic cleanup when the process terminates voluntarily (exit) or unvoluntarily (receipt of a signal) : the termination triggers a "close" operation which decrements the reference counter. When the latter drops to 0, the object disappears because the deletion operation completes as well.
Without this tricks, an process may terminate without doing any cleanup and consequently leaves "garbage" entries in the file system.
Ok, what happens is that a file is only deleted once there are no more references towards said file, this includes open file descriptors, and since we have fd_mem, shm_unlink will remove the link in /dev/shm/MEM_NAME but the file will not be deleted until the fd_mem has been closed.
I am currently working on something using POSIX named semaphores and shared memory and I've read from the man pages that all open named semaphores are automatically closed on process termination. Is this also the case for shared memory objects, are they also closed and unmapped or simply just closed? I cannot find any information about this on the man pages.
The question seems to be about how and when to clean up POSIX shared memory used by one or more processes, or possibly about how to avoid shared memory being cleaned up prematurely.
POSIX shared memory is designed on a model intentionally similar to regular file access. In particular,
shm_open() will create and open a new, persistent shared-memory object or simply open an existing one, depending on whether there already is one with the specified name.
that region can be opened by other processes (and therefore must persist) until it is unlinked via shm_unlink().
a shared memory region lives after its unlinking as long as any process has it open, but it can no longer be opened via shm_open().
mapping a shared memory region via mmap() has the effect of holding it open while that mapping is in place, independent of the file descriptor used to map it
Moreover, memory mappings for a shared-memory region are in most respects the same as mappings for regular files. Mappings are per-process properties; they do not survive termination of the process to which they pertain. Mappings are preserved (duplicated) across fork()s.
On some systems, shared memory regions may even be accessible via the file system. Aside from the different functions for managing them, their most significant difference from regular files is probably that they do not persist across reboots.
Thus, you do not need to worry about termination of a process unwantedly tearing down a shared memory region that is in use by other processes. On the other hand, you can arrange for shared-memory regions to be cleaned up automatically by unlinking them after every process that needs to obtain access by name has done so. If you intend to grant access only to child processes (and maybe their children, etc.) then you can unlink immediately after creation. Children will inherit the mapping when you fork them.
So, in response to the actual question:
Is this also the case for shared memory objects, are they also closed and unmapped or simply just closed?
Shared memory objects open and / or mapped by a process are both closed and unmapped when that process terminates, but they are not automatically unlinked. They will persist at least until manually unlinked or the system is rebooted.
Is this also the case for shared memory objects, are they also closed
and unmapped or simply just closed?
They are unmapped but they may continue to occupy space in the backing filesystem (usually tmpfs/shmfs for /dev/shm on memory) if they are not explicitly unlinked.
On FreeBSD it is possible to get this automatic unlinking with the non-portable SHM_ANON flag. If you want this kind of behaviour you can either:
Use mmap(2) with the MAP_ANONYMOUS flag and share file descriptors via fork(2) or send them to other processes with sendmsg(2) using Unix domain sockets.
Use System V shared memory with the IPC_RMID flag, which automatically destroys the memory segment after the last process detaches it. The dettachment happens when the process dies or calls shmdt(2).
Use the newer Linux-only memfd_create(2) system call.
If I fork and exec another executable, will the newly spawned process be able to access memory shared through mmap from the parent.
...
fd = open(filename)
str = mmap (MAP_SHARED, .. fd)
pid = fork();
if(pid == 0) {
exec("executable_2");
}
....
My question is, is it possible to access (read only) the shared memory mapped from file, from this spawned executable_2?
EDIT: the main purpose would be to save reading time (I/O) since this file is read-only. The newly spawned process is not a copy of the calling process.
The child process would have to remap the memory to access it, but can do so unless the 'shared' memory was mapped privately.
This would apply to all forms of shared memory across an exec*() — the new process has a new, independent address space and any shared memory mapping must be done anew in the executed process.
Simply forking, of course, leaves the shared memory as shared memory. But using exec*() gives the new process a clean address space unsullied by shared memory from the program it was running as previously.
Note that since the file descriptor was not opened with O_CLOEXEC (or was not later modified to set the FD_CLOEXEC flag on it), the file descriptor is open in the executed process. Whether the executed process knows what it is open for is another matter altogether — it probably doesn't unless it gets told by the code that executed it (command line argument, or perhaps environment variable).
My question is regarding initializing memory obtained from using shm_open() and mmap(). One common advice I have seen in several places is to call shm_open() with flags O_CREAT|O_EXCL: if that succeeds then we are the first user of the shared memory and can initialize it, otherwise we are not the first and the shared memory has already been initialized by another process.
However, from what I understand about shm_open and from the testing that I did on Linux, this wouldn't work: the shared memory objects get left over in the system, even after the last user of the shared memory object has unmapped and closed. A simple test program which calls shm_open with O_CREAT|O_EXCL, then closes the descriptor and exit, will succeed on the first run, but will still fail on the second run, even though nobody else is using the shared memory at that time.
It actually seems to me that (at least on the system that I tested) the behavior of shm_open is pretty much identical to open(): if I modify my simple test program to write something to the shared memory (through the pointer obtained by mmap) and exit, then the shared memory object will keep its contents persistently (I can run another simple program to read back the data I wrote previously).
So is the advice about using shm_open with O_CREAT|O_EXCL just wrong, or am I missing something?
I do know that the shared memory object can be removed with shm_unlink(), but it seems that will only cause more problems:
If a process dies before calling shm_unlink() then we are back to the problem described above.
If one process calls shm_unlink() while some other processes are still mapped into the same shared memory, these other processes will still continue using it as usual. Now, if another process comes and calls shm_open() with the same name and O_CREAT specified, it will actually succeed in creating new shared memory object with the same name, which is totally unrelated to the old shared memory object the other processes are still using. Now we have a process trying to communicate with other processes via the shared memory and totally unaware that it is using a wrong channel.
I m used to Windows semantics where shared memory object exists only as long as at least one handle is open to it, so this Posix stuff is very confusing.
Since you use the O_EXCL flag I will assume that you have a set of processes gathered around one master (the creator of the segment).
Then, your master process will create the shared memory segment using a call to shm_open :
shmid = shm_open("/insert/name/here", O_CREAT|O_EXCL, 0644);
if (-1 == shmid) {
printf("Oops ..\n");
}
Here, the slaves are ready to use the segment. Since the master HAS to create the segment, there is no need to use the O_CREAT flag in the slaves calls. You'll just have to handle possible errors if the slave call is performed when the segment is not created yet or already destroyed.
When any of your processes is done with the segment, it should call shm_unlink(). In this kind of architecture, the master is usually feeding the slaves. When it has nothing more to say, it just shuts up. The slaves have then the responsibility to handle corresponding errors gracefully.
As you stated, if a process dies before calling the shm_unlink procedure, then the segment will continue to live thereafter. To avoid this in some cases, you could define your own signal handlers in order to perform the operation when signals such as SIGINT are received. Anyway, you won't be able to cover the mess in case SIGKILL is sent to your process.
EDIT :
To be more specific, the use of O_CREAT | O_EXCL is wrong when unnecessary. With the little example above, you can see that it is required for the master to create the segment, thus those flags are needed. On the other hand, none of the slave processes would have to ever create it. Thus, you will absolutely forbid the use of O_CREAT in the related calls.
Now, if another process calls shm_open(..., O_CREAT, ...) when the segment is already created, it will just retrieve a file descriptor related to this very segment. It will thus be on the right channel (if it has the rights to do so, see the mode argument)
You can do the following :
int test = shmget(key_t key,size,0); Put this at the star of each process. Zero flag here tries to open an existing shared memory if its not created yet test will equal -1 so you can make a check after this statement if test -1 go and creat a shared memory else you just got an id to an existing shared memory ..... I hope this help
I am working on a program by C that some processes need to access a shared memory on an embed linux. This shared memory needs to be initialized when it was created. Any process attaching to this memory may crash. When it restarted (may be by linux INIT), it must not initialize the shared memory again since other processes are using it. How to tell if current starting of the process that is creating shared memory is the first time or restarted. I came up with an idea that allocates a integer in shared memory where will be written as a number like 5678956 (any number other than ffffffff or 00000000) to claim this memory has been initialized. But I am not sure if this is working well since the critical data is storing this memory. Any advice would be appreciated. Thanks.
You should use both a shared semaphore and shared memory segment. Attempt opening the semaphore with sem_open using O_EXCL|O_CREAT and an initial value of 0. If that succeeds, create and initialize the shared memory segment, then post the semaphore and close it. If opening the seamphore in exclusive mode failed, open it non-exclusive and wait on the semaphore, then close it.
Another solution, if you prefer: Use a named file in the filesystem with mmap and MAP_SHARED for your shared memory. First create the file with a temporary name and populate it with the initial data it should contain. Then attempt to link it to the real name. If link fails with EEXIST, you're not the first process, and you can just delete your temp file and open and map the existing one. If link succeeds, you are the first process.