I am working with some older real-time control system code written using the RTAI extensions to Linux. I see four different mechanisms in use to create and share memory across process boundaries.
1) RTAI shared memory (rt_shm_alloc & rt_shm_free):
This uses an unsigned long globally unique value as a key for accessing the shared memory. Behind the scenes (from user space at least) it uses an ioctl on a character device to generate the memory then mmap to make it available.
2) System V (ftok, shmget, shmat, shmctl, etc):
This uses ftok to generate a key that is used, along with an index value, to find and map a block of memory. I've not tried to see how this is actually implemented, but I'm assuming that somewhere behind the curtain it is using mmap.
3) Posix shared memory (shm_open, mmap, shm_unlink, etc):
This takes a string (with some restrictions on content) and provides a file handle which can be used to mmap the linked block of memory. This seems to be supported using a virtual filesystem.
4) direct use of mmap and character driver ioctl calls
Certain kernel modules provide an interface which directly supports using mmap to create and shared a block of memory.
All of these mechanisms seem to use mmap either explicitly or implicitly to alter the virtual memory subsystem to setup and manage the shared memory.
The question is: If a block of memory is shared using one of these systems, is there any way to setup an alias that will access the same memory in the other systems.
A use case:
I have two I/O subsystems. The first is implemented using linux kernel driver and exports it's current I/O state in one chunk of shared memory created using the RTAI shared memory mechanism. The second is based on the etherlab ethercat master which uses a custom kernel module and directly uses ioctl and mmap to create a shared memory block.
I have 40 or so other systems which need access to certain I/O fields, but don't really need to know which I/O subsystem the data came from.
What I want is a way to open and access the different types of shared memory in a single coherent way, isolating the underlying implementation details from the users. Does such a mechanism exist?
I have altered the ethercat stack to use the RTAI shared memory mechanism to solve this instance, but that's just a temporary solution (Read: hack).
Related
when a program started the OS will create a virtual memory, which divided into stack, heap, data, text to run a process on it.I know that each segment is used for specification purpose such as text saves the binary code of program, data saves static and global variable. My question is why the OS need to create the virtual memory and divide it into the segments ? How about if OS just use the physical memory and the process run directly on the physical memory. I think maybe the answer is related with running many process at the same time, sharing memory between process but i am not sure. It is kind if you give me an example about the benefit of creating virtual memory and dividing it into the segments.
In an environment with memory protection via a memory mapping unit, all memory is virtual (mapped via the MMU). It's possible to simply map each virtual address linearly to physical addresses, while still using the protection capabilities of the MMU, but doing that makes no sense. There are many reasons to prefer that memory not be directly mappped, such as being able to share program instructions and shared library code between instances of the same program or different programs, being able to fork, etc.
I am curious to know the details of how the kernel deals with concurrent writes / access to a shared mapping created through mmap. Looking in the man pages, all that I see for MAP_SHARED is the following:
Share this mapping. Updates to the mapping are visible to
other processes mapping the same region, and (in the case of
file-backed mappings) are carried through to the underlying
file. (To precisely control when updates are carried through
to the underlying file requires the use of msync(2).)
However, from my understanding, msync is dealing with synchronization from the page cache to the underlying storage device. However, it doesn't seem to deal with the case that two processes have mmaped a file. What happens when two processes write to the mapped region at the same time? If I create a shared mapping, how can I be sure that an unrelated process that happens to map the same file isn't doing concurrent writes?
The kernel on its own isn't going to do anything to prevent multiple processes (or even threads within a single process) from trashing the shared memory contents. You have to use synchronization constructs (such as a mutex) and ensure that all code that accesses the shared memory agrees on how to go about it.
The way to protect against another process simply writing to the file is to use flock before mmapping the region. You then need some way for the process that performs the mapping to communicate the file descriptor to the second process.
The answer to this question suggests using unix file sockets for that purpose.
I have a shared memory used by multiple processes, these processes are created using MPI.
Now I need a mechanism to control the access of this shared memory.
I know that named semaphore and flock mechanisms can be used to do this but just wanted to know if MPI provides any special locking mechanism for shared memory usage ?
I am working on C under Linux.
MPI actually does provide support for shared memory now (as of version 3.0). You might try looking at the One-sided communication chapter (http://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf) starting with MPI_WIN_ALLOCATE_SHARED (11.2.3). To use this, you'll have to make sure you have an implementation that supports it. I know that the most recent versions of both MPICH and Open MPI work.
No, MPI doesn't provide any support for shared memory. In fact, MPI would not want to support shared memory. The reason is that a program written with MPI is supposed to scale to a large number of processors, and a large number of processors never have shared memory.
However, it may happen, and often does, that groups of small number of processors (in that set of large number of processors) do have shared memory. To utilize that shared memory however, OpenMP is used.
OpenMP is very simple. I strongly suggest you learn it.
I'm writing a real time library which exports a standardized interface (VST) and is hosted by external applications.
The library must publish a table that is viewable by any thread in the same process (if it knows where to look) - to be clear, this table must be viewable by ALL dlls in the process space - if they know where to look.
Accessing the table must be fast. Virtual memory seems like overkill, and I've considered using a window handle (and I still may) to message pump, but I'd prefer an even faster method, if one is available.
Also, a shared data segment in the PE is something I'd like to avoid if possible. I think I'd almost rather use a window handle.
I'm not concerned with synchronization at the moment, I can handle that after the fact. I'd just like some suggestions for the fastest technique to publish the table within a process space.
You seem to be confused. All threads in the same process share the same address space, so you don't need any form of IPC: if a thread knows the address of the table, it can access it.
Use CreateFileMapping and pass in INVALID_FILE_HANDLE as the file handle.
This will create a named shared memory page(s) that is accessible by anyone who knows the name.
Don't be alarmed by the fact that MSDN docs say it's backed by the paging file - it will only go to disk in case your physical memory is exhausted, just like regular system memory.
In all regards, since it's supported by hardware MMU - it's identical to regular memory.
I'm working on a Linux kernel project and i need to find a way to allocate Virtual Memory without allocating Physical Memory. For example if I use this :
char* buffer = my_virtual_mem_malloc(sizeof(char) * 512);
my_virtual_mem_malloc is a new SYSCALL implemented by my kernel module. All data written on this buffer is stocked on file or on other server by using socket (not on Physical Memory). So to complete this job, i need to request Virtual Memory and get access to the vm_area_struct structure to redefine vm_ops struct.
Do you have any ideas about this ?
thx
This is not architecturally possible. You can create vm areas that have a writeback routine that copies data somewhere, but at some level, you must allocate physical pages to be written to.
If you're okay with that, you can simply write a FUSE driver, mount it somewhere, and mmap a file from it. If you're not, then you'll have to just write(), because redirecting writes without allocating a physical page at all is not supported by the x86, at the very least.
There are a few approaches to this problem, but most of them require you to first write to an intermediate memory.
Network File System (NFS)
The easiest approach is simply to have the server open some sort of a shared file system such as NFS and using mmap() to map a remote file to a memory address. Then, writing to that address will actually write the OS's page cache, wich will eventually be written to the remote file when the page cache is full or after predefined system timeout.
Distributed Shared Memory (DSM)
An alternative approach is using DSM with a very small cache size.
In computer science, distributed shared memory (DSM) is a form of memory architecture where physically separated memories can be addressed as one logically shared address space.
[...] Software DSM systems can be implemented in an operating system, or as a programming library and can be thought of as extensions of the underlying virtual memory architecture. When implemented in the operating system, such systems are transparent to the developer; which means that the underlying distributed memory is completely hidden from the users.
It means that each virtual address is logically mapped to a virtual address on a remote machine and writing to it will do the following: (a) receive the page from the remote machine and gain exclusive access. (b) update the page data. (c) release the page and send it back to the remote machine when it reads it again.
On typical DSM implementation, (c) will only happen when the remote machine will read the data again, but you might start from existing DSM implementation and change the behavior so that the data is sent once the local machine page cache is full.
I/O MMU
[...] the IOMMU maps device-visible virtual addresses (also called device addresses or I/O addresses in this context) to physical addresses.
This basically means to write directly to the network device buffer, which is actually implementing an alternative driver for that device.
Such approach seems the most complicated and I don't see any benefit from that approach.
This approach is actually not using any intermediate memory but is definitely not recommended unless the system has a heavy realtime requirement.