I'm working some code including communication between processes, using semaphores. I made structure like this:
typedef struct container {
sem_t resource, mutex;
int counter;
} container;
and use in that way (in main app and the same in subordinate processes)
container *memory;
shm_unlink("MYSHM"); //just in case
fd = shm_open("MYSHM", O_RDWR|O_CREAT|O_EXCL, 0);
if(fd == -1) {
printf("Error");
exit(EXIT_FAILURE);
}
memory = mmap(NULL, sizeof(container), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
ftruncate(fd, sizeof(container));
Everything is fine when I use one of the sem_ functions, but when I try to do something like
memory->counter = 5;
It doesn't work. Probably I got something wrong with pointers, but I tried almost everything and nothing seems to work. Maybe there's a better way to share variables, structures etc between processes ?
Unfortunately I'm not allowed to use boost or something similiar, the code is for educational purposes and I'm intentend to keep as simple as it's possible.
Because you're using shm_unlink() immediately before shm_open(), your two processes never open the same shared memory object - each one is creating a new, distinct object (even though they have the same name).
It's not clear what you mean by "doesn't work". The following minimal example, based on your code, works fine for me. What does it do for you?
#include <stdio.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <semaphore.h>
typedef struct container {
sem_t resource, mutex;
int counter;
} container;
int main()
{
container *memory;
int fd = shm_open("MYSHM", O_RDWR|O_CREAT|O_EXCL, 0);
if(fd == -1) {
perror("shm_open");
return 1;
}
ftruncate(fd, sizeof(container));
memory = mmap(NULL, sizeof(container), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
memory->counter = 5;
printf("%d\n", memory->counter);
return 0;
}
On reviewing your code, it's failing because you are trying to access the file descriptor in a child process after exec(), and shm_open sets the FD_CLOEXEC flag by default so the file descriptor is no longer open in the child. So you just need to unset that flag in the main process (eg. right after checking for the error after shm_open):
fdflags = fcntl(fd, F_GETFD);
fdflags &= ~FD_CLOEXEC;
fcntl(fd, F_SETFD, fdflags);
Why are you calling shm_open at all? A pretty good way to do this is to just call 'open' on some pathname and use that.
Related
I'm trying to share some data between different processes but I would like to use the same address in all of them.
From what I've read, it seems that this should be possible using mmap() with the MAP_FIXED flag (or MAP_FIXED_NOREPLACE).
Some people suggest to create a shared block and then broadcast the address of that memory block to all the other processes. This comment is a good summary of the whole idea.
So I tried to implement that into this example. There are a sender and a receiver. The sender will create a shared block of memory. The receiver tries to allocate that block of memory at the same address. The address is passed around using a shared memory block too.
The issue is that the receiver can't map the memory object to said address.
I get EINVAL error.
Is it possible to do what I'm trying? If so, how or what am I doing wrong in this example?
// header.h
#pragma once
#define SHM_DATA_ADDR "/bitarray-addr"
#define SHM_DATA "/bitarray-data"
#define NUM 3
#define SIZE (NUM * sizeof(int))
#define SIZE_ADDR (sizeof(void *))
// sender.c
#include "header.h"
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stddef.h>
int main() {
// Open the shared memory object for read-only access
// Set permissions
int fd = shm_open(SHM_DATA, O_CREAT | O_EXCL | O_RDWR, 0600);
int fd_addr = shm_open(SHM_DATA_ADDR, O_CREAT | O_EXCL | O_RDWR, 0600);
ftruncate(fd, SIZE);
ftruncate(fd_addr, SIZE_ADDR);
// Establish mapping between address space and memory object
int RDWR = PROT_READ | PROT_WRITE;
int *data = (int *)mmap(0, SIZE, RDWR, MAP_SHARED, fd, 0);
void **data_addr = (void **)mmap(0, SIZE_ADDR, RDWR, MAP_SHARED, fd_addr, 0);
*data_addr = data;
printf("%p\n", data);
printf("%p\n", data_addr);
printf("%p\n", *data_addr);
// Unmap address space
munmap(data_addr, SIZE);
munmap(data, SIZE);
// Close file descriptors
close(fd_addr);
close(fd);
return EXIT_SUCCESS;
}
// receiver.c
#include "header.h"
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <stddef.h>
#include <errno.h>
int main() {
// Open the shared memory object for read-only access
// Set permissions
int fd = shm_open(SHM_DATA, O_RDONLY, 0600);
int fd_addr = shm_open(SHM_DATA_ADDR, O_RDONLY, 0600);
ftruncate(fd, SIZE);
ftruncate(fd_addr, SIZE_ADDR);
// Establish mapping between address space and memory object
void **data_addr = (void **)mmap(0, SIZE_ADDR, PROT_READ, MAP_SHARED, fd_addr, 0);
int *data = (int *)mmap(*data_addr, SIZE, PROT_READ, MAP_FIXED, fd, 0);
printf("%p\n", data_addr);
printf("%p\n", *data_addr);
printf("%p\n", data);
// Unmap address space
munmap(data_addr, SIZE_ADDR);
munmap(data, SIZE);
// Close file descriptors
close(fd_addr);
close(fd);
// Destroy shared memory object
shm_unlink(SHM_DATA_ADDR);
shm_unlink(SHM_DATA);
return EXIT_SUCCESS;
}
From the mmap(2) man page, emphasis mine:
The flags argument determines whether updates to the mapping are
visible to other processes mapping the same region, and whether
updates are carried through to the underlying file. This behavior is
determined by including exactly one of the following values in flags: [MAP_SHARED, MAP_SHARED_VALIDATE, MAP_PRIVATE]
In addition, zero or more of the following values can be ORed in
flags: [MAP_ANONYMOUS, MAP_FIXED, etc]
In your receiver code, you have:
int *data = (int *)mmap(*data_addr, SIZE, PROT_READ, MAP_FIXED, fd, 0);
which does not include any of MAP_SHARED, MAP_SHARED_VALIDATE, MAP_PRIVATE, hence the error. It's even stated explicitly in the Errors section:
EINVAL: flags contained none of MAP_PRIVATE, MAP_SHARED or MAP_SHARED_VALIDATE.
Change it to MAP_SHARED | MAP_FIXED and it runs successfully for me.
I want to write a semaphore to shared memory. My first idea was to pass the pointer returned by mmap to sem_init():
#include <stdio.h>
#include <semaphore.h>
#include <string.h>
#include <errno.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
int main(void)
{
sem_t *sem_ptr;
int shm_fd = shm_open("Shm", O_CREAT | O_RDWR, DEFFILEMODE);
fprintf(stderr, "%s\n", strerror(errno));
sem_ptr = mmap(NULL, sizeof(sem_t), PROT_WRITE, MAP_SHARED, shm_fd, 0);
fprintf(stderr, "%p\n", strerror(errno));
sem_init(sem_ptr, 1, 1);
fprintf(stderr, "%s\n", strerror(errno));
sem_destroy(sem_ptr);
return 0;
}
But it leads to this error(when sem_init() is called): Process finished with exit code 135 (interrupted by signal 7: SIGEMT)
Then I tried to initialize the semaphore with a sem_t variable and write it to the shared memory:
int main(void)
{
sem_t *sem_ptr;
sem_t s;
int shm_fd = shm_open("Shm", O_CREAT | O_RDWR, DEFFILEMODE);
fprintf(stderr, "%s\n", strerror(errno));
sem_ptr = mmap(NULL, sizeof(sem_t), PROT_WRITE, MAP_SHARED, shm_fd, 0);
fprintf(stderr, "%p\n", strerror(errno));
sem_init(&s, 1, 1);
fprintf(stderr, "%s\n", strerror(errno));
*sem_ptr = s;
sem_destroy(&s);
return 0;
}
Now the line *sem_ptr = s; leads to the same error as in the first programm
Can anyone help me please?
Your first strategy for creating the semaphore is correct. You can't necessarily copy a sem_t object to a different memory address and have it still work.
I'm not sure why you're getting SIGEMT, which I thought was never generated by modern Unixes. But when I run either of your programs on my computer, they crash with SIGBUS instead, and that pointed me at a bug that I know how to fix. When you mmap a file (a shared memory object is considered to be a file), and the size you ask for in the mmap call is bigger than the file, and then you access the memory area beyond the end of the file (by far enough that the CPU can trap this), you get a SIGBUS. And let me quote you a key piece of the shm_open manpage:
O_CREAT: Create the shared memory object if it does not exist. [...]
A new shared memory object initially has zero length—the size of
the object can be set using ftruncate(2).
What you need to do is call ftruncate on shm_fd to make the shared memory object big enough to hold the semaphore.
Some less-important bugs you should fix at the same time:
All of the system calls that work with memory maps may malfunction if you give them offsets or sizes that aren't a multiple of the system page size. (They're supposed to round up for you, but historically there have been a lot of bugs in this area.) You get the system page size by calling sysconf(_SC_PAGESIZE), and you round up with a little helper function shown below.
Most C library functions are allowed to set errno to a nonzero value even if they succeed. You should check whether each function actually failed before printing strerror(errno). (In the code below I used perror instead for brevity.)
The name of a shared memory object is required to start with a slash, followed by up to NAME_MAX characters that are not slashes.
sem_init may read from as well as writing to the memory pointed to by sem_ptr, and subsequent use of sem_wait and sem_post definitely will, so you should use PROT_READ|PROT_WRITE in the mmap call.
Putting it all together, this is a revised version of your first program which works on my computer. Because of the SIGEMT thing I can't promise it will work for you.
#include <fcntl.h>
#include <semaphore.h>
#include <stdio.h>
#include <sys/mman.h>
#include <unistd.h>
#ifndef DEFFILEMODE
# define DEFFILEMODE 0666
#endif
static long round_up(long n, long mult)
{
return ((n + mult - 1) / mult) * mult;
}
int main(void)
{
long pagesize;
long semsize;
sem_t *sem_ptr;
int shm_fd;
pagesize = sysconf(_SC_PAGESIZE);
if (pagesize == -1) {
perror("sysconf(_SC_PAGESIZE)");
return 1;
}
shm_fd = shm_open("/Shm", O_CREAT|O_RDWR, DEFFILEMODE);
if (shm_fd == -1) {
perror("shm_open");
return 1;
}
semsize = round_up(sizeof(sem_t), pagesize);
if (ftruncate(shm_fd, semsize) == -1) {
perror("ftruncate");
return 1;
}
sem_ptr = mmap(0, semsize, PROT_READ|PROT_WRITE, MAP_SHARED, shm_fd, 0);
if (sem_ptr == MAP_FAILED) {
perror("mmap");
return 1;
}
if (sem_init(sem_ptr, 1, 1)) {
perror("sem_init");
return 1;
}
sem_destroy(sem_ptr);
shm_unlink("/Shm");
return 0;
}
An additional complication you should be aware of is that calling sem_init on a semaphore that has already been initialized causes undefined behavior. This means you have to use some other kind of locking around the creation of the shared memory segment and the semaphore within. Off the top of my head I don't know how to do this in a bulletproof way.
For a UNIX/C project, I'm supposed to allocate two shared memory segments (which child processes will eventually access with read-only and write permissions, respectively) of two integers each. But any time I try to call shmat(3), it ends up returning -1, setting errno to EACCES, apparently indicating insufficient permissions. I've produced what seems to be the minimum required code fro the error below, with possibly a couple extra includes:
#define _SVID_SOURCE
#include <errno.h>
#include <stdio.h>
#include <sys/shm.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
int main(int argc, char *argv[]){
int i, j, shmid, tshmid;
int * clock;
int * shmMsg;
tshmid = shmget(IPC_PRIVATE,sizeof(int)*2, IPC_CREAT | IPC_EXCL | 0777); //0777 permissions are more liberal than I need, but I've tried other various literals and numbers as well as just CREAT/EXCL.
if (tshmid < 1){
printf("Error: In parent process (%d), shmid came up %d \n",getpid(),shmid);
exit(-1);
}
clock = (int *) shmat(shmid,NULL,0); //I've also tried this with the second argument as (void *) 0, and with the third argument as "(SHM_R | SHM_W)" and "0777"
if (clock == (void *) -1){
printf("Error: First shmat couldn't shmattach to shmid #%d. ERRNO %d\n",shmid,errno);
shmdt(clock);
exit(-1);
} //it never even gets this far
shmdt(clock);
}
Each time, this produces an error message like:
Error: First shmat couldn't shmattach to shmid #1033469981. ERRNO 13
The longer version of my program initially returned an identical error, but errno was set to 43 (EIDRM: Segment identified by shm_id was removed). I've recursively chmodded the whole directory to full access, so that's not the issue, and every time my program crashes I have to manually deallocate the shared memory using ipcrm, so the shmids apply to actual segments. Why won't it attach?
Let's say I have the standard "Hello, World! \n" saved to a text file called hello.txt. If I want to change the 'H' to a 'R' or something, can I achieve this with mmap()?
mmap does not exist in the standard C99 (or C11) specification. It is defined in POSIX.
So assuming you have a POSIX system (e.g. Linux), you could first open(2) the file for read & write:
int myfd = open("hello.txt", O_RDWR);
if (myfd<0) { perror("hello.txt open"); exit(EXIT_FAILURE); };
Then you get the size (and other meta-data) of the file with fstat(2):
struct stat mystat = {};
if (fstat(myfd,&mystat)) { perror("fstat"); exit(EXIT_FAILURE); };
Now the size of the file is in mystat.st_size.
off_t myfsz = mystat.st_size;
Now we can call mmap(2) and we need to share the mapping (to be able to write inside the file thru the virtual address space)
void*ad = mmap(NULL, myfsz, PROT_READ|PROT_WRITE, MAP_SHARED,
myfd, 0);
if (ad == MMAP_FAILED) { perror("mmap"); exit(EXIT_FAILURE); };
Then we can overwrite the first byte (and we check that indeed the first byte in that file is H since you promised so):
assert (*(char*ad) == 'H');
((char*)ad) = 'R';
We might call msync(2) to ensure the file is updated right now on the disk. If we don't, it could be updated later.
Notably for very large mappings (notably those much larger than available RAM), we can assist the kernel (and its page cache) with hints given thru madvise(2) or posix_madvise(3)...
Notice that a mapping remains in effect even after a close(2). Use munmap & mprotect or mmap with MAP_FIXED on the same address range to change them.
On Linux, you could use proc(5) to query the address space. So your program could read (e.g. after fopen, using fgets in a loop) the pseudo /proc/self/maps file (or /proc/1234/maps for process of pid 1234).
BTW, mmap is used by dlopen(3); it can be called a lot of times, my manydl.c program demonstrates that on Linux you could have many hundreds of thousands of dlopen-ed shared files (so many hundreds of thousands of memory mappings).
Here's a working example.
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>
int main(){
int myFile = open("hello.txt", O_RDWR);
if(myFile < 0){
printf("open error\n");
}
struct stat myStat = {};
if (fstat(myFile, &myStat)){
printf("fstat error\n");
}
off_t size = myStat.st_size;
char *addr;
addr = mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, myFile, 0);
if (addr == MAP_FAILED){
printf("mmap error\n");
}
if (addr[0] != 'H'){
printf("Error: first char in file not H");
}
addr[0] = 'J';
return 0;
}
When two processes share a segment of memory opened with shm_open and then it gets mmap-ed, does doing an mprotect on a portion of the shared memory in one process affects the permissions seen by the other process on this same portion? In other words, if one process makes part of the shared memory segment read-only, does it become read-only for the other process too?
I always like to address those questions in two parts.
Part 1 - Let's test it
Let's consider an example that is relatively similar to the one at shm_open(3).
Shared header - shared.h
#include <sys/mman.h>
#include <fcntl.h>
#include <semaphore.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0)
struct shmbuf {
char buf[4096];
sem_t sem;
};
Creator process - creator.c (Compile using gcc creator.c -o creator -lrt -lpthread)
#include <ctype.h>
#include <string.h>
#include "shared.h"
int
main(int argc, char *argv[])
{
if (argc != 2) {
fprintf(stderr, "Usage: %s /shm-path\n", argv[0]);
exit(EXIT_FAILURE);
}
char *shmpath = argv[1];
int fd = shm_open(shmpath, O_CREAT | O_EXCL | O_RDWR,
S_IRUSR | S_IWUSR);
if (fd == -1)
errExit("shm_open");
struct shmbuf *shm;
if (ftruncate(fd, sizeof(*shm)) == -1)
errExit("ftruncate");
shm = mmap(NULL, sizeof(*shm), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (shm == MAP_FAILED)
errExit("mmap");
if (sem_init(&shm->sem, 1, 0) == -1)
errExit("sem_init");
if (mprotect(&shm->buf, sizeof(shm->buf), PROT_READ) == -1)
errExit("mprotect");
if (sem_wait(&shm->sem) == -1)
errExit("sem_wait");
printf("got: %s\n", shm->buf);
shm_unlink(shmpath);
exit(EXIT_SUCCESS);
}
Writer process - writer.c (Compile using gcc writer.c -o writer -lrt -lpthread)
#include <string.h>
#include "shared.h"
int
main(int argc, char *argv[])
{
if (argc != 3) {
fprintf(stderr, "Usage: %s /shm-path string\n", argv[0]);
exit(EXIT_FAILURE);
}
char *shmpath = argv[1];
char *string = argv[2];
int fd = shm_open(shmpath, O_RDWR, 0);
if (fd == -1)
errExit("shm_open");
struct shmbuf *shm = mmap(NULL, sizeof(*shm), PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
if (shm == MAP_FAILED)
errExit("mmap");
strcpy(&shm->buf[0], string);
if (sem_post(&shm->sem) == -1)
errExit("sem_post");
exit(EXIT_SUCCESS);
}
What's supposed to happen?
The creator process, creates a new shared memory object, "sets" its size, maps it into memory (as shm), uses mprotect to allow only writes to the buffer (shm->buf) and waits for a semaphore to know when the writer (which we will discuss in a moment finishes its thing).
The writer process starts, opens the same shared memory object, writes into it whatever we tell it to and signals the semaphore.
Question is, will the writer be able to write to the shared memory object even though the creator changed the protection to READ-ONLY?
Let's find out. We can run it using:
# ./creator.c /shmulik &
# ./writer.c /shmulik hi!
got: hi!
#
[1]+ Done ./creator /shmulik
As you can see, the writer was able to write to the shared memory, even though the creator set it's protection to READ-ONLY.
Maybe the creator does something wrong? Let's try to add the following line to creator.c:
if (mprotect(&shm->buf, sizeof(shm->buf), PROT_READ) == -1)
errExit("mprotect");
memset(&shm->buf, 0, sizeof(shm->buf)); // <-- This is the new line
if (sem_wait(&shm->sem) == -1)
errExit("sem_wait");
Let's recompile & run the creator again:
# gcc creator.c -o creator -lrt -lpthread
# ./creator /shmulik
Segmentation fault
As you can see, the mprotect worked as expected.
How about we let the writer map the shared memory, then we change the protection? Well, it ain't going to change anything. mprotect ONLY affects the memory protection of the process calling it (and it's descendants).
Part 2 - Let's understand it
First, you have to understand that shm_open is a glibc method, it's not a systemcall.
You can get the glibc source code from their website and just look for shm_open to see that yourself.
The underlying implementation of shm_open is a regular call for open, just like the man page suggests.
As we already saw, most of the magic happens in mmap. When calling mmap, we have to use MAP_SHARED (rather than MAP_PRIVATE), otherwise every process is going to get a private memory segment to begin with, and obviously one ain't gonna affect the other.
When we call mmap, the hops are roughly:
ksys_mmap_pgoff.
vm_mmap_pgoff.
do_mmap
mmap_region
At that last point, you could see that we take the process' memory management context mm and allocate a new virtual memory area vma:
struct mm_struct *mm = current->mm;
...
vma = vm_area_alloc(mm);
...
vma->vm_page_prot = vm_get_page_prot(vm_flags);
This memory area is not shared with other processes.
Since mprotect changes only the vm_page_prot on the per-process vma, it doesn't affect other processes that map the same memory space.
The POSIX specification for mprotect() suggests that changes in the protection of shared memory should affect all processes using that shared memory.
Two of the error conditions detailed are:
[EAGAIN]
The prot argument specifies PROT_WRITE over a MAP_PRIVATE mapping and there are insufficient memory resources to reserve for locking the private page.
[ENOMEM]
The prot argument specifies PROT_WRITE on a MAP_PRIVATE mapping, and it would require more space than the system is able to supply for locking the private pages, if required.
These strongly suggest that memory that is mapped with MAP_SHARED should not fail because of a lack of memory for making copies.
See also the POSIX specification for mmap().