I've been googling this a little bit and haven't been able to find a clear answer so I'm hoping someone has some insight into shared memory segments in windows VS linux.
In Linux there are 2 ways of creating shared memory for IPC: shared memory segments (shmget, et al) and memory mapped files (mmap). From my brief understanding mmap requires you to have an actual file somewhere in the OS to map whereas shared memory segments are just name based segments in memory that can be attached to by processes.
In Windows there only seems to be the equivalent of memory mapped files where you have to have an actual file floating around somewhere.
My question: Is this actually the only kind of shared memory in Windows or does it have an api for creating non file based shared memory segments.
The Unix mmap() API is practically equivalent to the CreateFileMapping/MapViewOfFile Windows API. Both can map files and/or can create shared (anonymous) maps that are backed by the swap device (if any). As a matter of fact, glibc uses anonymous mmap() to implement malloc() when the requested memory size is sufficiently large.
Windows supports one additional mechanism - shared data sections in the executable file, something that is not implemented in Linux. First you create a named data section using the #pragma data_seg(".somename") and put the shared variables inside. Then you tell the linker to mark the section as read/write/shared with the following option: /SECTION:.somename,RWS. The whole process is described in MSDN. This only works for copies of the same module, either EXE or DLL file. Same module means same file in the same file system location: processes created from different copies of the same executable but located in different places won't see each others named sections as shared.
Yes, you can use non file-based shared memory segments in Windows.
#pragma comment(linker, "/SECTION:.shared,RWS")
#pragma data_seg(".shared")
int g_iShared = 0;
#pragma data_seg()
This might be a bit late.
Windows shared memory is not the same as Linux only some things are similar.
The biggest difference is the memory allocation granularity size. Linux is 4K and Windows is 64K. If it's important to have say arbitrary 8K pages mapped into specific 8K destinations well you are stuck on Windows and it just can't be done. (If someone figures this out then please let me know).
Another difference is you can mmap a new page over the top of an existing page effectively replacing the first page mapping. In windows you can't do this but instead must destroy the entire view and rebuild the entire view in what ever new layout that is required. So if the "view" contains 1024 pages and 1 page changes then in Linux you can just change that one page. In Windows you must drop all 1024 pages and re-view the same 1023 pages + the one new page!
On Linux you can share memory without a backing file by using ANONYMOUS memory but then it becomes hard to share or you can use shm_opn to create a shared memory file descriptor and pass that to mmap. Here is a good link.
http://nullprogram.com/blog/2016/04/10/
I've used this and it works.
Related
I am re-implementing mmap in a device driver for DMA.
I saw this question: Linux Driver: mmap() kernel buffer to userspace without using nopage that has an answer using vm_insert_page() to map one page at a time; hence, for multiple pages, needed to execute in a loop. Is there another API that handles this?
Previously I used dma_alloc_coherent to allocate a chunk of memory for DMA and used remap_pfn_range to build a page table that associates process's virtual memory to physical memory.
Now I would like to allocate a much larger chunk of memory using __get_free_pages with order greater than 1. I am not sure how to build page table in that case. The reason is as follows:
I checked the book Linux Device Drivers and noticed the following:
Background:
When a user-space process calls mmap to map device memory into its address space, the system responds by creating a new VMA to represent that mapping. A driver that supports mmap (and, thus, that implements the mmap method) needs to help that process by completing the initialization of that VMA.
Problem with remap_pfn_range:
remap_pfn_range won’t allow you to remap conventional addresses, which include the ones you obtain by calling get_free_page. Instead, it maps in the zero page. Everything appears to work, with the exception that the process sees private, zero-filled pages rather than the remapped RAM that it was hoping for.
The corresponding implementation using get_free_pages with order 0, i.e. only 1 page in scullp device driver:
The mmap method is disabled for a scullp device if the allocation order is greater than zero, because nopage deals with single pages rather than clusters of pages. scullp simply does not know how to properly manage reference counts for pages that are part of higher-order allocations.
May I know if there is a way to create VMA for pages obtained using __get_free_pages with order greater than 1?
I checked Linux source code and noticed there are some drivers re-implementing struct dma_map_ops->alloc() and struct dma_map_ops->map_page(). May I know if this is the correct way to do it?
I think I got the answer to my question. Feel free to correct me if I am wrong.
I happened to see this patch: mm: Introduce new vm_map_pages() and vm_map_pages_zero() API while I was googling for vm_insert_page.
Previouly drivers have their own way of mapping range of kernel pages/memory into user vma and this was done by invoking vm_insert_page() within a loop.
As this pattern is common across different drivers, it can be generalized by creating new functions and use it across the drivers.
vm_map_pages() is the API which could be used to mapped kernel memory/pages in drivers which has considered vm_pgoff.
After reading it, I knew I found what I want.
That function also could be found in Linux Kernel Core API Documentation.
As for the difference between remap_pfn_range() and vm_insert_page() which requires a loop for a list of contiguous pages, I found this answer to this question extremely helpful, in which it includes a link to explanation by Linus.
As a side note, this patch mm: Introduce new vm_insert_range and vm_insert_range_buggy API indicates that the earlier version of vm_map_pages() was vm_insert_range(), but we should stick to vm_map_pages(), since under the hood vm_map_pages() calls vm_insert_range().
I have a massive file ie 1TiB owned as 'filehandler', permitted rwx------. I mmap(2)-ed it into the 64bit address space, and all works successfully. This file handled by a process running as user 'filehandler'.
Other processes request services from this handler process running as other user than the filehandler. They login into handler through unix socket. They communicate by IPC rules, all is ok.
The entire file must not be shared to requesters due to security reasons. In the file only some parts are allowed to access for requester processes.
The best performance will be given if share of the memory, just the allowed parts of the file with the requesting processes.
For example the shm gives the key to access the segment for other processes, it is a practical targeting to requester.
Is there any way to share only the allowed parts of a mmap(2)-ed space to any processes identified like shm technology?
Is there any way to share only the allowed parts of a mmap(2)-ed space to any processes identified like shm technology?
TL;DR: No.
In more detail,
How to remap a file mmap(2)-ed in memory like shmget
mmap() and shmget()are not really comparable. A better comparison would be between the combination of shm_open() / ftruncate() / mmap() on one hand and the combination of shmget() / shmat() on the other. These are the main alternatives in POSIX for creating labeled shared-memory segments and mapping them into the process's address space. You should recognize there that the analog of shmget() is shm_open(), and the analog of mmap() in this context is shmat().
Now, returning to
Is there any way to share only the allowed parts of a mmap(2)-ed space to any processes identified like shm technology?
Note well that in both cases above, it is the object being mapped (a shared memory segment) that provides for sharing between unrelated processes, not anything to do with mmap() itself. The same applies when mmap() maps any other kind of object, such as a regular file. It is always the mapped object through which any shared access is mediated. It has to be this way, because a memory mapping is a property of one process -- it is not itself share-able.
Your design calls for a filehandler process to serve as gatekeeper to the data, rather than allowing clients to access it directly. That's fine, but it precludes the clients mapping the file into memory. You could probably arrange for client to access the data through a shared memory segment of either flavor, but that would require the server copying the right data out of the big file into the client's shared memory segment. That might indeed be something to consider, but you can forget about the server providing clients direct memory-mapped access to the file.
There's no connection between implementations of shmget system call (a System V AT&T derived implementation) and mmap (a berkeley's BSD system derived implementation) It's true that in BSD systems, AT&T shared memory is implemented by using mmaped private segments with no file attached, but that's of no use also, because you need the shared segment to be associated with a file.
As you need, the only possibility to share memory segments related to a file's contents are by using mmap system call, because System V shared memory segments have no means to associate a file with them.
All of these resources (either SysV or BSD) have a set of permissions bits associated with them that allow them to be used with some security, but as happens with files, only in a global (the entire resource) way, making you able to access the whole thing or nothing at all.
BTW, you can implement what you want by means of copying segment contents to a different, private, segment (only the size you want the client to be allowed to see) only the segments it is allowed access, and this way you can have finer control over whom and what the clients are allowed to do.
And last, remember that this approach requires copying of segments of shared memory, so you need to remember to copy back the exported segment for a customer if you don't want the modifications made by that client to be lost when the client finishes using them.
From my point of view, you are complicating things a little, but you better know how your application is designed than me.
How to have an isolated part of memory, that is NOT at all backed to any file or extra management layers such as piping and that can be shared between two dedicated processes on the same Windows machine?
Majority of articles point me into the direction of CreateFileMapping. Let's start from there:
How does CreateFileMapping with hFile=INVALID_HANDLE_VALUE actually work?
According to
https://msdn.microsoft.com/en-us/library/windows/desktop/aa366537(v=vs.85).aspx
it
"...creates a file mapping object of a specified size that is backed by the system paging file instead of by a file in the file system..."
Assume I write something into the memory which is mapped by CreateFileMapping with hFile=INVALID_HANDLE_VALUE. Under which conditions will this content be written to the page file on disk?
Also my understanding of what motivates the use of shared memory is to keep performance up and optimized. Why is the article "Creating Named Shared Memory"
(https://msdn.microsoft.com/de-de/library/windows/desktop/aa366551(v=vs.85).aspx) refering to
CreateFileMapping, if there is not a single attribute combination, that would prevent writing to files, e.g. the page file?
Going back to the original question: I am afraid, that CreateFileMapping is not good enough... So what would work?
You misunderstand what it means for memory to be "backed" by the system paging file. (Don't feel bad; Raymond Chen has described the text you quoted from MSDN as "one of the most misunderstood sentences in the Win32 documentation.") Almost all of the computer's memory is "backed" by something on disk; only the "non-paged pool", used exclusively by the kernel and as little as possible, isn't. If a page isn't backed by an ordinary named file, then it's backed by the system paging file. The operating system won't write pages out to the system paging file unless it needs to, but it can if it does need to.
This architecture is intended to ensure that processes can be completely "paged out" of RAM when they have nothing to do. This used to be much more important than it is nowadays, but it's still valuable; a typical Windows desktop will have dozens of processes "idle" waiting for events (e.g. needing to spool a print job) that may never happen. Those processes can get paged out and the memory can be put to more constructive use.
CreateFileMapping with hfile=INVALID_HANDLE_VALUE is, in fact, what you want. As long as the processes sharing the memory are actively doing stuff with it, it will remain resident in RAM and there will be no performance problem. If they go idle, yeah, it may get paged out, but that's fine because they're not doing anything with it.
You can direct the system not to page out a chunk of memory; that's what VirtualLock is for. But it's meant to be used for small chunks of memory containing secret information, where writing it to the page file could conceivably leak the secret. The MSDN page warns you that "Each version of Windows has a limit on the maximum number of pages a process can lock. This limit is intentionally small to avoid severe performance degradation."
In Understanding the Linux Kernel, 3rd edition, it says:
Shared libraries are especially convenient on systems that provide file memory mapping, because they reduce the amount of main memory requested for executing a
program. When the dynamic linker must link a shared library to a process, it does not copy the object code, but performs only a memory mapping of the relevant portion of the library file into the process’s address space. This allows the page frames containing the machine code of the library to be shared among all processes that are using the same code. Clearly, sharing is not possible if the program has been linked statically. (page 817)
I am interested in this, want to write a small program in C to verify, given two pids as input such as two gedit processes, and then get the address information from page frames to be shared. Does anyone know how to do it? From that book, I think the bss segment and text segment address from two or more gedit processes are same, is that correct?
It is not the text and bss sections of your gedit (or whatever) that have the same address, but the content of the libc.so shared library - and all other shared libraries used by the two gedit processes.
This, as the quoted text says, allows the shared library to be ONE copy, and this is the main benefit of the shared library in general.
bss is generally not shared - since that is per process data. text sections of two processes running the same executable, in Linux, will share the same code.
Unfortunately, the proof of this would be to look at the physical mapping of pages (page at address X in process A is at physical address Y, and page for address X in process B is also at physical address Y) within the processes, and that's, as far as I know, not easily available without groking about inside the OS kernel.
Look at the contents of /proc/*/maps.
The memory usage of a process can be displayed by running:
$ ps -C processname -o size
SIZE
3808
Is there any way to retrieve this information without executing ps (or any external program), or reading /proc?
On a Linux system, a process' memory usage can be queried by reading /proc/[pid]/statm. Where [pid] is the PID of the process. If a process wants to query its own data, it can do so by reading /proc/self/statm instead. man 5 proc says:
/proc/[pid]/statm
Provides information about memory usage, measured in pages. The
columns are:
size total program size
(same as VmSize in /proc/[pid]/status)
resident resident set size
(same as VmRSS in /proc/[pid]/status)
share shared pages (from shared mappings)
text text (code)
lib library (unused in Linux 2.6)
data data + stack
dt dirty pages (unused in Linux 2.6)
You could just open the file with: fopen("/proc/self/statm", "r") and read the contents.
Since the file returns results in 'pages', you will want to find the page size also. getpagesize () returns the size of a page, in bytes.
You have a few options to do find the memory usage of a program:
Run it within a profiler like Valgrind or memprof.
exec/proc_open/fork a new process to use ps, top, or pmap as you would from the command line
bundle the ps into your app and use it directly (it's open source, of course!)
Use the /proc system (which is all that ps does, anyways...)
Create a report the kernel, which watches over process memory operations. The /proc filesystem is just a view into the kernel's internal data structures, so this is really already done for you.
Develop your own mechanism to compute memory usage without kernel assistance.
The former are all educational from a system administration perspective, and would be the best options in a real-life situation, but the last bullet point is probably the most interesting. You'd probably want to read the source of Valgrind or memprof to see how it works, but essentially what you'd need to do is insert your mechanism between the app and the kernel, and intercept any requests for memory allocation. Additionally, when the process started, you would want to initialize its memory space with a preset value like 0xDEADBEEF. Then, after the process finished, you could read through the memory space and count the occurrences of words other than your preset value, giving you an estimate of memory usage.
Of course, things are always more complicated than they seem. What about memory used by shared libraries? Pipes? Shared memory between your processes and another? System calls? Virtual memory allocated but not used? Data buffered to the disk? There's a lot of calls to be made beyond your question 'memory of process', see this post for some additional concerns.