I want to write a semaphore to shared memory. My first idea was to pass the pointer returned by mmap to sem_init():
#include <stdio.h>
#include <semaphore.h>
#include <string.h>
#include <errno.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
int main(void)
{
sem_t *sem_ptr;
int shm_fd = shm_open("Shm", O_CREAT | O_RDWR, DEFFILEMODE);
fprintf(stderr, "%s\n", strerror(errno));
sem_ptr = mmap(NULL, sizeof(sem_t), PROT_WRITE, MAP_SHARED, shm_fd, 0);
fprintf(stderr, "%p\n", strerror(errno));
sem_init(sem_ptr, 1, 1);
fprintf(stderr, "%s\n", strerror(errno));
sem_destroy(sem_ptr);
return 0;
}
But it leads to this error(when sem_init() is called): Process finished with exit code 135 (interrupted by signal 7: SIGEMT)
Then I tried to initialize the semaphore with a sem_t variable and write it to the shared memory:
int main(void)
{
sem_t *sem_ptr;
sem_t s;
int shm_fd = shm_open("Shm", O_CREAT | O_RDWR, DEFFILEMODE);
fprintf(stderr, "%s\n", strerror(errno));
sem_ptr = mmap(NULL, sizeof(sem_t), PROT_WRITE, MAP_SHARED, shm_fd, 0);
fprintf(stderr, "%p\n", strerror(errno));
sem_init(&s, 1, 1);
fprintf(stderr, "%s\n", strerror(errno));
*sem_ptr = s;
sem_destroy(&s);
return 0;
}
Now the line *sem_ptr = s; leads to the same error as in the first programm
Can anyone help me please?
Your first strategy for creating the semaphore is correct. You can't necessarily copy a sem_t object to a different memory address and have it still work.
I'm not sure why you're getting SIGEMT, which I thought was never generated by modern Unixes. But when I run either of your programs on my computer, they crash with SIGBUS instead, and that pointed me at a bug that I know how to fix. When you mmap a file (a shared memory object is considered to be a file), and the size you ask for in the mmap call is bigger than the file, and then you access the memory area beyond the end of the file (by far enough that the CPU can trap this), you get a SIGBUS. And let me quote you a key piece of the shm_open manpage:
O_CREAT: Create the shared memory object if it does not exist. [...]
A new shared memory object initially has zero length—the size of
the object can be set using ftruncate(2).
What you need to do is call ftruncate on shm_fd to make the shared memory object big enough to hold the semaphore.
Some less-important bugs you should fix at the same time:
All of the system calls that work with memory maps may malfunction if you give them offsets or sizes that aren't a multiple of the system page size. (They're supposed to round up for you, but historically there have been a lot of bugs in this area.) You get the system page size by calling sysconf(_SC_PAGESIZE), and you round up with a little helper function shown below.
Most C library functions are allowed to set errno to a nonzero value even if they succeed. You should check whether each function actually failed before printing strerror(errno). (In the code below I used perror instead for brevity.)
The name of a shared memory object is required to start with a slash, followed by up to NAME_MAX characters that are not slashes.
sem_init may read from as well as writing to the memory pointed to by sem_ptr, and subsequent use of sem_wait and sem_post definitely will, so you should use PROT_READ|PROT_WRITE in the mmap call.
Putting it all together, this is a revised version of your first program which works on my computer. Because of the SIGEMT thing I can't promise it will work for you.
#include <fcntl.h>
#include <semaphore.h>
#include <stdio.h>
#include <sys/mman.h>
#include <unistd.h>
#ifndef DEFFILEMODE
# define DEFFILEMODE 0666
#endif
static long round_up(long n, long mult)
{
return ((n + mult - 1) / mult) * mult;
}
int main(void)
{
long pagesize;
long semsize;
sem_t *sem_ptr;
int shm_fd;
pagesize = sysconf(_SC_PAGESIZE);
if (pagesize == -1) {
perror("sysconf(_SC_PAGESIZE)");
return 1;
}
shm_fd = shm_open("/Shm", O_CREAT|O_RDWR, DEFFILEMODE);
if (shm_fd == -1) {
perror("shm_open");
return 1;
}
semsize = round_up(sizeof(sem_t), pagesize);
if (ftruncate(shm_fd, semsize) == -1) {
perror("ftruncate");
return 1;
}
sem_ptr = mmap(0, semsize, PROT_READ|PROT_WRITE, MAP_SHARED, shm_fd, 0);
if (sem_ptr == MAP_FAILED) {
perror("mmap");
return 1;
}
if (sem_init(sem_ptr, 1, 1)) {
perror("sem_init");
return 1;
}
sem_destroy(sem_ptr);
shm_unlink("/Shm");
return 0;
}
An additional complication you should be aware of is that calling sem_init on a semaphore that has already been initialized causes undefined behavior. This means you have to use some other kind of locking around the creation of the shared memory segment and the semaphore within. Off the top of my head I don't know how to do this in a bulletproof way.
Related
I'm not sure if this question makes sense, but let's say I have a pointer to some memory:
char *mem;
size_t len;
Is it possible to somehow map the contents of mem to another address as a read-only mapping? i.e. I want to obtain a pointer mem2 such that mem2 != mem and accessing mem2[i] actually reads mem[i] (without doing a copy).
My ultimate goal would be to take non-contiguous chunks of memory and make them appear to be contiguous by mapping them next to each other.
One approach I considered is to use fmemopen and then mmap, but there's no file descriptor associated with the result of fmemopen.
General case - no control over first mapping
/proc/[PID]/pagemap + /dev/mem
The only way I can think of making this work without any copying is by manually opening and checking /proc/[PID]/pagemap to get the Page Frame Number of the physical page corresponding to the page you want to "alias", and then opening and mapping /dev/mem at the corresponding offset. While this would work in theory, it would require root privileges, and is most likely not possible on any reasonable Linux distribution since the kernel is usually configured with CONFIG_STRICT_DEVMEM=y which puts strict restrictions over the usage of /dev/mem. For example on x86 it disallows reading RAM from /dev/mem (only allows reading memory-mapped PCI regions). Note that in order for this to work the page you want to "alias" needs to be locked to keep it in RAM.
In any case, here's an example of how this would work if you were able/willing to do this (I am assuming x86 64bit here):
#include <stdio.h>
#include <errno.h>
#include <limits.h>
#include <sys/mman.h>
#include <unistd.h>
#include <fcntl.h>
/* Get the physical address of an existing virtual memory page and map it. */
int main(void) {
FILE *fp;
char *endp;
unsigned long addr, info, physaddr, val;
long off;
int fd;
void *mem;
void *orig_mem;
// Suppose that this is the existing page you want to "alias"
orig_mem = mmap(NULL, 0x1000, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
if (orig_mem == MAP_FAILED) {
perror("mmap orig_mem failed");
return 1;
}
// Write a dummy value just for testing
*(unsigned long *)orig_mem = 0x1122334455667788UL;
// Lock the page to prevent it from being swapped out
if (mlock(orig_mem, 0x1000)) {
perror("mlock orig_mem failed");
return 1;
}
fp = fopen("/proc/self/pagemap", "rb");
if (!fp) {
perror("Failed to open \"/proc/self/pagemap\"");
return 1;
}
addr = (unsigned long)orig_mem;
off = addr / 0x1000 * 8;
if (fseek(fp, off, SEEK_SET)) {
perror("fseek failed");
return 1;
}
// Get its information from /proc/self/pagemap
if (fread(&info, sizeof(info), 1, fp) != 1) {
perror("fread failed");
return 1;
}
physaddr = (info & ((1UL << 55) - 1)) << 12;
printf("Value: %016lx\n", info);
printf("Physical address: 0x%016lx\n", physaddr);
// Ensure page is in RAM, should be true since it was mlock'd
if (!(info & (1UL << 63))) {
fputs("Page is not in RAM? Strange! Aborting.\n", stderr);
return 1;
}
fd = open("/dev/mem", O_RDONLY);
if (fd == -1) {
perror("open(\"/dev/mem\") failed");
return 1;
}
mem = mmap(NULL, 0x1000, PROT_READ, MAP_PRIVATE|MAP_ANONYMOUS, fd, physaddr);
if (mem == MAP_FAILED) {
perror("Failed to mmap \"/dev/mem\"");
return 1;
}
// Now `mem` is effecively referring to the same physical page that
// `orig_mem` refers to.
// Try reading 8 bytes (note: this will just return 0 if
// CONFIG_STRICT_DEVMEM=y).
val = *(unsigned long *)mem;
printf("Read 8 bytes at physaddr 0x%016lx: %016lx\n", physaddr, val);
return 0;
}
userfaultfd(2)
Other than what I described above, AFAIK there isn't a way to do what you want from userspace without copying. I.E. there is not a way to simply tell the kernel "map this second virtual addresses to the same memory of an existing one". You can however register an userspace handler for page faults through the userfaultfd(2) syscall and ioctl_userfaultfd(2), and I think this is overall your best shot.
The whole mechanism is similar to what the kernel would do with a real memory page, only that the faults are handled by a user-defined userspace handler thread. This is still pretty much an actual copy, but is atomic to the faulting thread and gives you more control. It could potentially also perform better in general since the copying is controlled by you and can therefore be done only if/when needed (i.e. at the first read fault), while in the case of a normal mmap + copy you always do the copying regardless if the page will ever be accessed later or not.
There is a pretty good example program in the manual page for userfaultfd(2) which I linked above, so I'm not going to copy-paste it here. It deals with one or more pages and should give you an idea about the whole API.
Simpler case - control over the first mapping
In the case you do have control over the first mapping which you want to "alias", then you can simply create a shared mapping. What you are looking for is memfd_create(2). You can use it to create an anonymous file which can then be mmaped multiple times with different permissions.
Here's a simple example:
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/types.h>
int main(void) {
int memfd;
void *mem_ro, *mem_rw;
// Create a memfd
memfd = memfd_create("something", 0);
if (memfd == -1) {
perror("memfd_create failed");
return 1;
}
// Give the file a size, otherwise reading/writing will fail
if (ftruncate(memfd, 0x1000) == -1) {
perror("ftruncate failed");
return 1;
}
// Map the fd as read only and private
mem_ro = mmap(NULL, 0x1000, PROT_READ, MAP_PRIVATE, memfd, 0);
if (mem_ro == MAP_FAILED) {
perror("mmap failed");
return 1;
}
// Map the fd as read/write and shared (shared is needed if we want
// write operations to be propagated to the other mappings)
mem_rw = mmap(NULL, 0x1000, PROT_READ|PROT_WRITE, MAP_SHARED, memfd, 0);
if (mem_rw == MAP_FAILED) {
perror("mmap failed");
return 1;
}
printf("ro mapping # %p\n", mem_ro);
printf("rw mapping # %p\n", mem_rw);
// This write can now be read from both mem_ro and mem_rw
*(char *)mem_rw = 123;
// Test reading
printf("read from ro mapping: %d\n", *(char *)mem_ro);
printf("read from rw mapping: %d\n", *(char *)mem_rw);
return 0;
}
I'm trying to share some data between different processes but I would like to use the same address in all of them.
From what I've read, it seems that this should be possible using mmap() with the MAP_FIXED flag (or MAP_FIXED_NOREPLACE).
Some people suggest to create a shared block and then broadcast the address of that memory block to all the other processes. This comment is a good summary of the whole idea.
So I tried to implement that into this example. There are a sender and a receiver. The sender will create a shared block of memory. The receiver tries to allocate that block of memory at the same address. The address is passed around using a shared memory block too.
The issue is that the receiver can't map the memory object to said address.
I get EINVAL error.
Is it possible to do what I'm trying? If so, how or what am I doing wrong in this example?
// header.h
#pragma once
#define SHM_DATA_ADDR "/bitarray-addr"
#define SHM_DATA "/bitarray-data"
#define NUM 3
#define SIZE (NUM * sizeof(int))
#define SIZE_ADDR (sizeof(void *))
// sender.c
#include "header.h"
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stddef.h>
int main() {
// Open the shared memory object for read-only access
// Set permissions
int fd = shm_open(SHM_DATA, O_CREAT | O_EXCL | O_RDWR, 0600);
int fd_addr = shm_open(SHM_DATA_ADDR, O_CREAT | O_EXCL | O_RDWR, 0600);
ftruncate(fd, SIZE);
ftruncate(fd_addr, SIZE_ADDR);
// Establish mapping between address space and memory object
int RDWR = PROT_READ | PROT_WRITE;
int *data = (int *)mmap(0, SIZE, RDWR, MAP_SHARED, fd, 0);
void **data_addr = (void **)mmap(0, SIZE_ADDR, RDWR, MAP_SHARED, fd_addr, 0);
*data_addr = data;
printf("%p\n", data);
printf("%p\n", data_addr);
printf("%p\n", *data_addr);
// Unmap address space
munmap(data_addr, SIZE);
munmap(data, SIZE);
// Close file descriptors
close(fd_addr);
close(fd);
return EXIT_SUCCESS;
}
// receiver.c
#include "header.h"
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stddef.h>
#include <errno.h>
int main() {
// Open the shared memory object for read-only access
// Set permissions
int fd = shm_open(SHM_DATA, O_RDONLY, 0600);
int fd_addr = shm_open(SHM_DATA_ADDR, O_RDONLY, 0600);
ftruncate(fd, SIZE);
ftruncate(fd_addr, SIZE_ADDR);
// Establish mapping between address space and memory object
void **data_addr = (void **)mmap(0, SIZE_ADDR, PROT_READ, MAP_SHARED, fd_addr, 0);
int *data = (int *)mmap(*data_addr, SIZE, PROT_READ, MAP_FIXED, fd, 0);
printf("%p\n", data_addr);
printf("%p\n", *data_addr);
printf("%p\n", data);
// Unmap address space
munmap(data_addr, SIZE_ADDR);
munmap(data, SIZE);
// Close file descriptors
close(fd_addr);
close(fd);
// Destroy shared memory object
shm_unlink(SHM_DATA_ADDR);
shm_unlink(SHM_DATA);
return EXIT_SUCCESS;
}
From the mmap(2) man page, emphasis mine:
The flags argument determines whether updates to the mapping are
visible to other processes mapping the same region, and whether
updates are carried through to the underlying file. This behavior is
determined by including exactly one of the following values in flags: [MAP_SHARED, MAP_SHARED_VALIDATE, MAP_PRIVATE]
In addition, zero or more of the following values can be ORed in
flags: [MAP_ANONYMOUS, MAP_FIXED, etc]
In your receiver code, you have:
int *data = (int *)mmap(*data_addr, SIZE, PROT_READ, MAP_FIXED, fd, 0);
which does not include any of MAP_SHARED, MAP_SHARED_VALIDATE, MAP_PRIVATE, hence the error. It's even stated explicitly in the Errors section:
EINVAL: flags contained none of MAP_PRIVATE, MAP_SHARED or MAP_SHARED_VALIDATE.
Change it to MAP_SHARED | MAP_FIXED and it runs successfully for me.
Trying to search a pattern in a big file using mmap. The file is huge (way more than the physical memory). My worry is that if I used the file size as the second parameter for mmap(), there won't be enough physical memory to satisfy the system call. So I used 0x1000 as the length in the hope that OS will automatically map the right part of file as my pointer moves. But the following code snippet gave segmentation fault.
Any ideas?
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
long fileSize(char *fname) {
struct stat stat_buf;
int rc = stat(fname, &stat_buf);
return rc == 0 ? stat_buf.st_size : -1;
}
int main(int argc, char *argv[]) {
long size = fileSize(argv[1]);
printf("size=%ld\n", size);
int fd = open(argv[1], O_RDONLY);
printf("fd=%d\n", fd);
char *p = mmap(0, 0x1000, PROT_READ, MAP_SHARED, fd, 0);
if (p == MAP_FAILED) {
perror ("mmap");
return 1;
}
long i;
int pktLen;
int *pInt;
for (i=0; i < size; i+=4) {
pInt = (int*)(p+i);
if (pInt[i] == 0x12345678) {
printf("found it at %ld\n", i); break;
}
}
if (i == size) {
printf("didn't find it\n");
}
close(fd);
return 0;
}
Update
Turned out I had a silly bug
The line
if (pInt[i] == 0x12345678) should have been if (pInt[0] == 0x12345678)
Use
struct stat info;
long page;
const char *map;
size_t size, mapping;
int fd, result;
page = sysconf(_SC_PAGESIZE);
if (page < 1L) {
fprintf(stderr, "Invalid page size.\n");
exit(EXIT_FAILURE);
}
fd = open(filename, O_RDONLY);
if (fd == -1) {
fprintf(stderr, "%s: Cannot open file: %s.\n", filename, strerror(errno));
exit(EXIT_FAILURE);
}
result = fstat(fd, &info);
if (result == -1) {
fprintf(stderr, "%s: Cannot get file information: %s.\n", filename, strerror(errno));
close(fd);
exit(EXIT_FAILURE);
}
if (info.st_size <= 0) {
fprintf(stderr, "%s: No data.\n", filename);
close(fd);
exit(EXIT_FAILURE);
}
size = info.st_size;
if ((off_t)size != info.st_size) {
fprintf(stderr, "%s: File is too large to map.\n", filename);
close(fd);
exit(EXIT_FAILURE);
}
/* mapping is size rounded up to a multiple of page. */
if (size % (size_t)page)
mapping = size + page - (size % (size_t)page);
else
mapping = size;
map = mmap(NULL, mapping, PROT_READ, MAP_SHARED | MAP_NORESERVE, fd, 0);
if (map == MAP_FAILED) {
fprintf(stderr, "%s: Cannot map file: %s.\n", filename, strerror(errno));
close(fd);
exit(EXIT_FAILURE);
}
if (close(fd)) {
fprintf(stderr, "%s: Unexpected error closing file descriptor.\n", filename);
exit(EXIT_FAILURE);
}
/*
* Use map[0] to map[size-1], but remember that it is not a string,
* and that there is no trailing '\0' at map[size].
*
* Accessing map[size] to map[mapping-1] is not allowed, and may
* generate a SIGBUS signal (and kill the process).
*/
/* The mapping is automatically torn down when the process exits,
* but you can also unmap it with */
munmap(map, mapping);
The important points in the code above:
You'll need to start your code with e.g.
#define _POSIX_C_SOURCE 200809L
#define _BSD_SOURCE
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <string.h>
#include <errno.h>
The _BSD_SOURCE is required for MAP_NORESERVE to be defined, even though it is a GNU/Linux-specific feature.
mapping (length in man 2 mmap) must be a multiple of page size (sysconf(_SC_PAGESIZE)).
MAP_NORESERVE flag tells the kernel that the mapping is backed by the file only, and as such, is allowed to be larger than available RAM + SWAP.
You can (but do not need to) close the file descriptor referring to the mapped file with no issues, because the mapping itself contains a reference in-kernel.
Years ago, on a different forum, I showed a simple program to manipulate a terabyte of data (1 TiB = 1,099,511,627,776 bytes) using this very approach (although it uses a sparse backing file; i.e. mostly implicit zeroes, with less than 250 MB of actual data written to the backing file -- mostly to reduce the amount of disk space needed). Of course, it requires a 64-bit machine running Linux, as the virtual memory on 32-bit machines is limited to 232 = 4 GiB (Linux does not support segmented memory models).
The Linux kernel is surprisingly efficient in choosing which pages to keep in RAM, and which pages to evict. Of course, you can make that even more efficient, by telling the kernel which parts of the mapping you are unlikely to access (and therefore can be evicted), by using posix_madvise(address, length, advice) with advice being POSIX_MADV_DONTNEED or POSIX_MADV_WILLNEED. This has the benefit that unlike unmapping the "dontneed" parts, you can, if you need to, re-access that part of the mapping. (If the pages are already evicted, the access to the mapping will just block until the pages are re-loaded to memory. In other words, you can use posix_madvise() to "optimize" eviction logic, without limiting what part of the mapping can be accessed.)
In your case, if you do a linear or semi-linear search over the data using e.g. memmem(), you can use posix_madvise(map, mapping, POSIX_MADV_SEQUENTIAL).
Personally, I'd run the search first without using any posix_madvise() calls, and then see if it makes a significant enough positive difference, using the same data set (and several runs, of course). (You can safely -- with no risk of losing any data -- clear the page cache between test runs using sudo sh -c 'sync ; echo 3 > /proc/sys/vm/drop_caches ; sync', if you wish to exclude the effects of having the large file (mostly) already cached, between timing runs.)
The SIGSEGV is because you're accessing beyond 0x1000 bytes (in the for loop). You have to mmap() the complete size bytes of the fd.
The concept of demand paging in virtual memory subsystem helps exact same scenarios like yours - applications/application data bigger than the physical memory size. After the mmap(), as and when you access the (virtual) address, if there is no physical page mapped to it (page fault), kernel will find out a physical page that can be used (page replacement).
fd = open(argv[1], O_RDONLY);
ptr = mmap(NULL, file_size, PROT_READ, MAP_PRIVATE, fd, 0);
/* Consume the entire file's data as needed */
munmap(ptr, file_size);
Alternately you can put a loop around the mmap()/munmap() to scan the file in PAGE_SIZE or in multiples of PAGE_SIZE. The last arg of mmap() - offset will come handy for that.
From man-page :
void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);
int munmap(void *addr, size_t length);
Pseudo-code :
fd = open(argv[1], O_RDONLY);
last_block_size = file_size % PAGE_SIZE;
num_pages = file_size / PAGE_SIZE + (last_block_size ? 1 : 0)
for (int i = 0; i < num_pages; i++) {
block_size = last_block_size && (i == num_pages - 1) ? last_block_size : PAGE_SIZE;
ptr = mmap(NULL, block_size, PROT_READ, MAP_PRIVATE, fd, i * PAGE_SIZE);
/* Consume the file's data range (ptr, ptr+block_size-1) as needed */
munmap(ptr, block_size);
}
Please use MAP_PRIVATE as the mapping might be just needed for your process alone. It just avoids few extra steps by the kernel for the MAP_SHARED.
Edit : It should have been MAP_PRIVATE in place of MAP_ANON. Changed.
When two processes share a segment of memory opened with shm_open and then it gets mmap-ed, does doing an mprotect on a portion of the shared memory in one process affects the permissions seen by the other process on this same portion? In other words, if one process makes part of the shared memory segment read-only, does it become read-only for the other process too?
I always like to address those questions in two parts.
Part 1 - Let's test it
Let's consider an example that is relatively similar to the one at shm_open(3).
Shared header - shared.h
#include <sys/mman.h>
#include <fcntl.h>
#include <semaphore.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0)
struct shmbuf {
char buf[4096];
sem_t sem;
};
Creator process - creator.c (Compile using gcc creator.c -o creator -lrt -lpthread)
#include <ctype.h>
#include <string.h>
#include "shared.h"
int
main(int argc, char *argv[])
{
if (argc != 2) {
fprintf(stderr, "Usage: %s /shm-path\n", argv[0]);
exit(EXIT_FAILURE);
}
char *shmpath = argv[1];
int fd = shm_open(shmpath, O_CREAT | O_EXCL | O_RDWR,
S_IRUSR | S_IWUSR);
if (fd == -1)
errExit("shm_open");
struct shmbuf *shm;
if (ftruncate(fd, sizeof(*shm)) == -1)
errExit("ftruncate");
shm = mmap(NULL, sizeof(*shm), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (shm == MAP_FAILED)
errExit("mmap");
if (sem_init(&shm->sem, 1, 0) == -1)
errExit("sem_init");
if (mprotect(&shm->buf, sizeof(shm->buf), PROT_READ) == -1)
errExit("mprotect");
if (sem_wait(&shm->sem) == -1)
errExit("sem_wait");
printf("got: %s\n", shm->buf);
shm_unlink(shmpath);
exit(EXIT_SUCCESS);
}
Writer process - writer.c (Compile using gcc writer.c -o writer -lrt -lpthread)
#include <string.h>
#include "shared.h"
int
main(int argc, char *argv[])
{
if (argc != 3) {
fprintf(stderr, "Usage: %s /shm-path string\n", argv[0]);
exit(EXIT_FAILURE);
}
char *shmpath = argv[1];
char *string = argv[2];
int fd = shm_open(shmpath, O_RDWR, 0);
if (fd == -1)
errExit("shm_open");
struct shmbuf *shm = mmap(NULL, sizeof(*shm), PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
if (shm == MAP_FAILED)
errExit("mmap");
strcpy(&shm->buf[0], string);
if (sem_post(&shm->sem) == -1)
errExit("sem_post");
exit(EXIT_SUCCESS);
}
What's supposed to happen?
The creator process, creates a new shared memory object, "sets" its size, maps it into memory (as shm), uses mprotect to allow only writes to the buffer (shm->buf) and waits for a semaphore to know when the writer (which we will discuss in a moment finishes its thing).
The writer process starts, opens the same shared memory object, writes into it whatever we tell it to and signals the semaphore.
Question is, will the writer be able to write to the shared memory object even though the creator changed the protection to READ-ONLY?
Let's find out. We can run it using:
# ./creator.c /shmulik &
# ./writer.c /shmulik hi!
got: hi!
#
[1]+ Done ./creator /shmulik
As you can see, the writer was able to write to the shared memory, even though the creator set it's protection to READ-ONLY.
Maybe the creator does something wrong? Let's try to add the following line to creator.c:
if (mprotect(&shm->buf, sizeof(shm->buf), PROT_READ) == -1)
errExit("mprotect");
memset(&shm->buf, 0, sizeof(shm->buf)); // <-- This is the new line
if (sem_wait(&shm->sem) == -1)
errExit("sem_wait");
Let's recompile & run the creator again:
# gcc creator.c -o creator -lrt -lpthread
# ./creator /shmulik
Segmentation fault
As you can see, the mprotect worked as expected.
How about we let the writer map the shared memory, then we change the protection? Well, it ain't going to change anything. mprotect ONLY affects the memory protection of the process calling it (and it's descendants).
Part 2 - Let's understand it
First, you have to understand that shm_open is a glibc method, it's not a systemcall.
You can get the glibc source code from their website and just look for shm_open to see that yourself.
The underlying implementation of shm_open is a regular call for open, just like the man page suggests.
As we already saw, most of the magic happens in mmap. When calling mmap, we have to use MAP_SHARED (rather than MAP_PRIVATE), otherwise every process is going to get a private memory segment to begin with, and obviously one ain't gonna affect the other.
When we call mmap, the hops are roughly:
ksys_mmap_pgoff.
vm_mmap_pgoff.
do_mmap
mmap_region
At that last point, you could see that we take the process' memory management context mm and allocate a new virtual memory area vma:
struct mm_struct *mm = current->mm;
...
vma = vm_area_alloc(mm);
...
vma->vm_page_prot = vm_get_page_prot(vm_flags);
This memory area is not shared with other processes.
Since mprotect changes only the vm_page_prot on the per-process vma, it doesn't affect other processes that map the same memory space.
The POSIX specification for mprotect() suggests that changes in the protection of shared memory should affect all processes using that shared memory.
Two of the error conditions detailed are:
[EAGAIN]
The prot argument specifies PROT_WRITE over a MAP_PRIVATE mapping and there are insufficient memory resources to reserve for locking the private page.
[ENOMEM]
The prot argument specifies PROT_WRITE on a MAP_PRIVATE mapping, and it would require more space than the system is able to supply for locking the private pages, if required.
These strongly suggest that memory that is mapped with MAP_SHARED should not fail because of a lack of memory for making copies.
See also the POSIX specification for mmap().
I'm working some code including communication between processes, using semaphores. I made structure like this:
typedef struct container {
sem_t resource, mutex;
int counter;
} container;
and use in that way (in main app and the same in subordinate processes)
container *memory;
shm_unlink("MYSHM"); //just in case
fd = shm_open("MYSHM", O_RDWR|O_CREAT|O_EXCL, 0);
if(fd == -1) {
printf("Error");
exit(EXIT_FAILURE);
}
memory = mmap(NULL, sizeof(container), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
ftruncate(fd, sizeof(container));
Everything is fine when I use one of the sem_ functions, but when I try to do something like
memory->counter = 5;
It doesn't work. Probably I got something wrong with pointers, but I tried almost everything and nothing seems to work. Maybe there's a better way to share variables, structures etc between processes ?
Unfortunately I'm not allowed to use boost or something similiar, the code is for educational purposes and I'm intentend to keep as simple as it's possible.
Because you're using shm_unlink() immediately before shm_open(), your two processes never open the same shared memory object - each one is creating a new, distinct object (even though they have the same name).
It's not clear what you mean by "doesn't work". The following minimal example, based on your code, works fine for me. What does it do for you?
#include <stdio.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <semaphore.h>
typedef struct container {
sem_t resource, mutex;
int counter;
} container;
int main()
{
container *memory;
int fd = shm_open("MYSHM", O_RDWR|O_CREAT|O_EXCL, 0);
if(fd == -1) {
perror("shm_open");
return 1;
}
ftruncate(fd, sizeof(container));
memory = mmap(NULL, sizeof(container), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
memory->counter = 5;
printf("%d\n", memory->counter);
return 0;
}
On reviewing your code, it's failing because you are trying to access the file descriptor in a child process after exec(), and shm_open sets the FD_CLOEXEC flag by default so the file descriptor is no longer open in the child. So you just need to unset that flag in the main process (eg. right after checking for the error after shm_open):
fdflags = fcntl(fd, F_GETFD);
fdflags &= ~FD_CLOEXEC;
fcntl(fd, F_SETFD, fdflags);
Why are you calling shm_open at all? A pretty good way to do this is to just call 'open' on some pathname and use that.