Is there any way, linux specific or not, to have posix shared memory segments (obtained with shm_open()) removed when no process is using them. i.e. have them reference counted and have the system remove them when the reference becomes 0
A few notes:
Establishing an atexit handler to remove them doesn't work if the program crashes.
Currently, the linux specific way, I embed the pid in the segment name, and try to find unused segments by walking /dev/shm in an external program. Which has the drawback of having to periodically clean them up externally in a rather hackish way.
As the program can run multiple copies, using a well defined name for the segment that the program reuses when it starts up is not feasible.
If there is a point in your program's execution when it is well known, that all processes that need to open the shared memory segment have already done so, you can safely unlink it. Unlinking removes the object from the global namespace but it sill lingers around as long as there is at least one process that keep its file descriptor open. If a crash occurs after that point, the file descriptor is automatically closed and the reference count is decremented. Once no open descriptors to the unlinked shared memory block remain, it is deleted.
This is useful in the following scenario: a process creates a shared memory block, unlinks it and then forks. The child inherits the file descriptor and can use the shared memory block to communicate with the parent. Once both processes terminate, the block is automatically removed as both file descriptors get closed.
While unlinked, the shared memory block is unavailable for other processes to open it. Meanwhile, if one use shm_open() with the same name as the unlinked block, a new and completely different shared memory block would be created instead.
No - at lest on Linux, the kernel doesn't contain anything that can do this. It's up to some application to call shm_unlink() at some point to get rid of a shared memory segment.
I found a way using a system command and the Linux command "fuser" which allow to list the processes which opened a file. This way, you can check if the shared memory file (located in /dev/shm") is still in use and delete it if not. Note that the operations of check / delete / create must be enclosed in a inter-processes critical section using a named mutex or named semaphore or file lock.
std::string shm_file = "/dev/shm/" + service_name + "Shm";
std::string cmd_line = "if [ -f " + shm_file + " ] ; then if ! fuser -s " + shm_file + " ; then rm -f " + shm_file + " ; else exit 2 ; fi else exit 3 ; fi";
int res = system(cmd_line.c_str());
switch (WEXITSTATUS(res)) {
case 0: _logger.warning ("The shared memory file " + shm_file + " was found orphan and is deleted"); break;
case 1: _logger.critical("The shared memory file " + shm_file + " was found orphan and cannot be deleted"); break;
case 2: _logger.trace ("The shared memory file " + shm_file + " is linked to alive processes"); break;
case 3: _logger.trace ("The shared memory file " + shm_file + " is not found"); break;
}
For the shared memory, created using sysV API, it is possible to have such a behaviour. On Linux only. It is not POSIX shared memory, but may work for you.
In the book The Linux Programming Interface one of the possible parameters for shmctl() is described the following way.
IPC_RMID
Mark the shared memory segment and its associated shmid_ds
data structure for deletion. If no processes currently have the
segment attached, deletion is immediate; otherwise, the segment is
removed after all processes have detached from it (i.e., when the
value of the shm_nattch field in the shmid_ds data structure falls to
0). In some applications, we can make sure that a shared memory
segment is tidily cleared away on application termination by marking
it for deletion immediately after all processes have attached it to
their virtual address space with shmat(). This is analogous to
unlinking a file once we’ve opened it. On Linux, if a shared segment
has been marked for deletion using IPC_RMID, but has not yet been
removed because some process still has it attached, then it is
possible for another process to attach that segment. However, this
behavior is not portable: most UNIX implementations prevent new
attaches to a segment marked for deletion. (SUSv3 is silent on what
behavior should occur in this scenario.) A few Linux applications have
come to depend on this behavior, which is why Linux has not been
changed to match other UNIX implementations.
Let's assume the most complicated case:
You have several processes communicating via shared memory
They can start and finish at any time, even multiple times. That means there is no master process, nor is there a dedicated "first" process that can initialize the shared memory.
That means, e.g., there is no point where you can safely unlink the shared memory, so neither Sergey's nor Hristo's answers work.
I see two possible solutions and would welcome feedback on them because the internet is horribly silent on this question:
Store the pid (or a more specific process identifier if you have one) of the last process that wrote to the shared memory inside the shared memory as a lock. Then you could do sth. like the following pseudo code:
int* pshmem = open shared memory()
while(true)
nPid = atomic_read(pshmem)
if nPid = 0
// your shared memory is in a valid state
break
else
// process nPid holds a lock to your shared memory
// or it might have crashed while holding the lock
if process nPid still exists
// shared memory is valid
break
else
// shared memory is corrupt
// try acquire lock
if atomic_compare_exchange(pshmem, nPid, my pid)
// we have the lock
reinitialize shared memory
atomic_write(pshem, 0) // release lock
else
// somebody else got the lock in the meantime
// continue loop
This verifies that the last writer didn't die while writing. The shared memory is still persisting longer than any of your processes.
Use a reader/writer file lock to find out if any process is the first process opening the shared memory object. The first process may then reinitialize the shared memory:
// try to get exclusive lock on lockfile
int fd = open(lockfile, O_RDONLY | O_CREAT | O_EXLOCK | O_NONBLOCK, ...)
if fd == -1
// didn't work, somebody else has it and is initializing shared memory
// acquire shared lock and wait for it
fd = open(lockfile, O_RDONLY | O_SHLOCK)
// open shared memory
else
// we are the first
// delete shared memory object
// possibly delete named mutex/semaphore as well
// create shared memory object (& semaphore)
// degrade exclusive lock to a shared lock
flock(fd, LOCK_SH)
File locks seem to be the only (?) mechanism on POSIX systems that is cleared up automatically when the process dies. Unfortunately, the list of caveats to use them is very, very long. The algorithm assumes flock is supported on the underlying filesystem at least on the local machine. The algorithm doesn't care if the locks are actually visible to other processes on NFS filesystems or not. They only have to be visible for all processes accessing the shared memory object.
This solution has been implemented on top of boost.interprocess.
Could you not just use a global counting semaphore to reference count? Wrap the attach and detach calls so that the semaphore is incremented when you attach to the memory and decremented when you detach. Release the segment when a detach reduces the semaphore to zero.
Not sure, if the below works, or feasible. But my try.
Why do not you execute the helper program, which is executed each time your program crashed.
ie:
/proc/sys/kernel/core_pattern to /path/to/Myprogram %p
Myprogram is executed when process crashes, and probably you can explore further.
see
man 5 core. for more information.
Hope this helps to some extend.
Related
I have the following code snippet :
fd_mem = shm_open(MEM_NAME , O_RDWR | O_CREAT | O_EXCL , 0600);
//Why do we use unlink before having mmaped ?
shm_unlink ( MEM_NAME );
ftruncate (fd_mem , mem_size)
plateau = (char*) mmap(NULL , mem_size , PROT_READ | PROT_WRITE , MAP_SHARED , fd_mem , 0);
My question is: why do we use "unlink" before having mapped the file into the virtual memory of the process? I'm confused as to how shm_unlink() works in that regard. I would think it deletes the file making fd_mem unusable but it doesn't.
I'm confused as to how shm_unlink works in that regard, I would think it deletes the file rendering fd_mem unusable but it doesn't.
shm_unlink() removes the name of the shared memory segment. The segment itself persists as long as any process has it open, but after it is unlinked, processes can no longer open it. Even so, new processes forked from one that holds the shared memory segment inherit that, and one process can copy a file descriptor to another via a UNIX-domain socket, so being unlinked does not inherently limit which or how many processes can access the segment.
This is exactly analogous to the situation with regular files and unlink(). Successfully unlinking a file name removes that name from its directory, but the file itself is not removed as long as any process has it open.
Among the reasons to do this sort of thing are
to ensure that resources are cleaned up, whenever and however a process terminates. Named shared-memory segments persist as long as either they remain linked or a process holds them open. By unlinking a segment immediately after creating and opening it, a process helps ensure that it will not live longer than wanted, even if that process crashes.
to avoid unwanted access. Named shared memory segments can be opened by any process with sufficient privilege. Unlinking a shared memory segment helps control that. It especially helps avoid unwanted shared usage by multiple copies of the same program.
Note also that it is possible to create anonymous shared memory segments, which are never linked to a name in the first place. This is much as if a named segment were created, opened, and immediately unlinked, but it leaves no window in which another process can unwantedly open the segment by name.
Opening a file or a shared memory segment increments a reference counter on the underlying "kernel object". The deletion operation, deletes the name of the object but does not decrement the reference counter. As long the reference counter is bigger than 0, the object is not destroyed.
The deletion of the object after opening it, is for the automatic cleanup when the process terminates voluntarily (exit) or unvoluntarily (receipt of a signal) : the termination triggers a "close" operation which decrements the reference counter. When the latter drops to 0, the object disappears because the deletion operation completes as well.
Without this tricks, an process may terminate without doing any cleanup and consequently leaves "garbage" entries in the file system.
Ok, what happens is that a file is only deleted once there are no more references towards said file, this includes open file descriptors, and since we have fd_mem, shm_unlink will remove the link in /dev/shm/MEM_NAME but the file will not be deleted until the fd_mem has been closed.
I am currently working on something using POSIX named semaphores and shared memory and I've read from the man pages that all open named semaphores are automatically closed on process termination. Is this also the case for shared memory objects, are they also closed and unmapped or simply just closed? I cannot find any information about this on the man pages.
The question seems to be about how and when to clean up POSIX shared memory used by one or more processes, or possibly about how to avoid shared memory being cleaned up prematurely.
POSIX shared memory is designed on a model intentionally similar to regular file access. In particular,
shm_open() will create and open a new, persistent shared-memory object or simply open an existing one, depending on whether there already is one with the specified name.
that region can be opened by other processes (and therefore must persist) until it is unlinked via shm_unlink().
a shared memory region lives after its unlinking as long as any process has it open, but it can no longer be opened via shm_open().
mapping a shared memory region via mmap() has the effect of holding it open while that mapping is in place, independent of the file descriptor used to map it
Moreover, memory mappings for a shared-memory region are in most respects the same as mappings for regular files. Mappings are per-process properties; they do not survive termination of the process to which they pertain. Mappings are preserved (duplicated) across fork()s.
On some systems, shared memory regions may even be accessible via the file system. Aside from the different functions for managing them, their most significant difference from regular files is probably that they do not persist across reboots.
Thus, you do not need to worry about termination of a process unwantedly tearing down a shared memory region that is in use by other processes. On the other hand, you can arrange for shared-memory regions to be cleaned up automatically by unlinking them after every process that needs to obtain access by name has done so. If you intend to grant access only to child processes (and maybe their children, etc.) then you can unlink immediately after creation. Children will inherit the mapping when you fork them.
So, in response to the actual question:
Is this also the case for shared memory objects, are they also closed and unmapped or simply just closed?
Shared memory objects open and / or mapped by a process are both closed and unmapped when that process terminates, but they are not automatically unlinked. They will persist at least until manually unlinked or the system is rebooted.
Is this also the case for shared memory objects, are they also closed
and unmapped or simply just closed?
They are unmapped but they may continue to occupy space in the backing filesystem (usually tmpfs/shmfs for /dev/shm on memory) if they are not explicitly unlinked.
On FreeBSD it is possible to get this automatic unlinking with the non-portable SHM_ANON flag. If you want this kind of behaviour you can either:
Use mmap(2) with the MAP_ANONYMOUS flag and share file descriptors via fork(2) or send them to other processes with sendmsg(2) using Unix domain sockets.
Use System V shared memory with the IPC_RMID flag, which automatically destroys the memory segment after the last process detaches it. The dettachment happens when the process dies or calls shmdt(2).
Use the newer Linux-only memfd_create(2) system call.
If I fork and exec another executable, will the newly spawned process be able to access memory shared through mmap from the parent.
...
fd = open(filename)
str = mmap (MAP_SHARED, .. fd)
pid = fork();
if(pid == 0) {
exec("executable_2");
}
....
My question is, is it possible to access (read only) the shared memory mapped from file, from this spawned executable_2?
EDIT: the main purpose would be to save reading time (I/O) since this file is read-only. The newly spawned process is not a copy of the calling process.
The child process would have to remap the memory to access it, but can do so unless the 'shared' memory was mapped privately.
This would apply to all forms of shared memory across an exec*() — the new process has a new, independent address space and any shared memory mapping must be done anew in the executed process.
Simply forking, of course, leaves the shared memory as shared memory. But using exec*() gives the new process a clean address space unsullied by shared memory from the program it was running as previously.
Note that since the file descriptor was not opened with O_CLOEXEC (or was not later modified to set the FD_CLOEXEC flag on it), the file descriptor is open in the executed process. Whether the executed process knows what it is open for is another matter altogether — it probably doesn't unless it gets told by the code that executed it (command line argument, or perhaps environment variable).
My question is regarding initializing memory obtained from using shm_open() and mmap(). One common advice I have seen in several places is to call shm_open() with flags O_CREAT|O_EXCL: if that succeeds then we are the first user of the shared memory and can initialize it, otherwise we are not the first and the shared memory has already been initialized by another process.
However, from what I understand about shm_open and from the testing that I did on Linux, this wouldn't work: the shared memory objects get left over in the system, even after the last user of the shared memory object has unmapped and closed. A simple test program which calls shm_open with O_CREAT|O_EXCL, then closes the descriptor and exit, will succeed on the first run, but will still fail on the second run, even though nobody else is using the shared memory at that time.
It actually seems to me that (at least on the system that I tested) the behavior of shm_open is pretty much identical to open(): if I modify my simple test program to write something to the shared memory (through the pointer obtained by mmap) and exit, then the shared memory object will keep its contents persistently (I can run another simple program to read back the data I wrote previously).
So is the advice about using shm_open with O_CREAT|O_EXCL just wrong, or am I missing something?
I do know that the shared memory object can be removed with shm_unlink(), but it seems that will only cause more problems:
If a process dies before calling shm_unlink() then we are back to the problem described above.
If one process calls shm_unlink() while some other processes are still mapped into the same shared memory, these other processes will still continue using it as usual. Now, if another process comes and calls shm_open() with the same name and O_CREAT specified, it will actually succeed in creating new shared memory object with the same name, which is totally unrelated to the old shared memory object the other processes are still using. Now we have a process trying to communicate with other processes via the shared memory and totally unaware that it is using a wrong channel.
I m used to Windows semantics where shared memory object exists only as long as at least one handle is open to it, so this Posix stuff is very confusing.
Since you use the O_EXCL flag I will assume that you have a set of processes gathered around one master (the creator of the segment).
Then, your master process will create the shared memory segment using a call to shm_open :
shmid = shm_open("/insert/name/here", O_CREAT|O_EXCL, 0644);
if (-1 == shmid) {
printf("Oops ..\n");
}
Here, the slaves are ready to use the segment. Since the master HAS to create the segment, there is no need to use the O_CREAT flag in the slaves calls. You'll just have to handle possible errors if the slave call is performed when the segment is not created yet or already destroyed.
When any of your processes is done with the segment, it should call shm_unlink(). In this kind of architecture, the master is usually feeding the slaves. When it has nothing more to say, it just shuts up. The slaves have then the responsibility to handle corresponding errors gracefully.
As you stated, if a process dies before calling the shm_unlink procedure, then the segment will continue to live thereafter. To avoid this in some cases, you could define your own signal handlers in order to perform the operation when signals such as SIGINT are received. Anyway, you won't be able to cover the mess in case SIGKILL is sent to your process.
EDIT :
To be more specific, the use of O_CREAT | O_EXCL is wrong when unnecessary. With the little example above, you can see that it is required for the master to create the segment, thus those flags are needed. On the other hand, none of the slave processes would have to ever create it. Thus, you will absolutely forbid the use of O_CREAT in the related calls.
Now, if another process calls shm_open(..., O_CREAT, ...) when the segment is already created, it will just retrieve a file descriptor related to this very segment. It will thus be on the right channel (if it has the rights to do so, see the mode argument)
You can do the following :
int test = shmget(key_t key,size,0); Put this at the star of each process. Zero flag here tries to open an existing shared memory if its not created yet test will equal -1 so you can make a check after this statement if test -1 go and creat a shared memory else you just got an id to an existing shared memory ..... I hope this help
I am working on a program by C that some processes need to access a shared memory on an embed linux. This shared memory needs to be initialized when it was created. Any process attaching to this memory may crash. When it restarted (may be by linux INIT), it must not initialize the shared memory again since other processes are using it. How to tell if current starting of the process that is creating shared memory is the first time or restarted. I came up with an idea that allocates a integer in shared memory where will be written as a number like 5678956 (any number other than ffffffff or 00000000) to claim this memory has been initialized. But I am not sure if this is working well since the critical data is storing this memory. Any advice would be appreciated. Thanks.
You should use both a shared semaphore and shared memory segment. Attempt opening the semaphore with sem_open using O_EXCL|O_CREAT and an initial value of 0. If that succeeds, create and initialize the shared memory segment, then post the semaphore and close it. If opening the seamphore in exclusive mode failed, open it non-exclusive and wait on the semaphore, then close it.
Another solution, if you prefer: Use a named file in the filesystem with mmap and MAP_SHARED for your shared memory. First create the file with a temporary name and populate it with the initial data it should contain. Then attempt to link it to the real name. If link fails with EEXIST, you're not the first process, and you can just delete your temp file and open and map the existing one. If link succeeds, you are the first process.