I have a C program that generates large amounts of data in memory, and I need to share one particular section of this data in memory, so that another process can have read access to it.
I'm attempting to use mmap to do this, but I'm not having much success. Here is my code:
//Code above generates a pointer to the memory section I need to share, named addr
if (infoBlock->memory_size == 1073741824) { //This is the data block I need to share
int err, fd;
fd = open("/tmp/testOutput", (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
goto failedExit;
}
unsigned *p = mmap(addr, 1073741824, PROT_READ, (MAP_SHARED | MAP_FIXED), fd, 0);
if (!p) {perror("mmap failed"); goto failedExit; }
printf("p is now: %p\n", p); //This should point to the shared mapping
printf("%u\n", *p); //Try to print out some data from the mapping
}
After running the program, I can see the file /tmp/testOutput is there, but it's size is 0. I'm not sure if that's a normal thing with memory mappings, as it's not technically a file. Also all of the output within my program points to the same memory address.
I can also see the memory map present within the /proc/PID/maps, with a reference to /tmp/testOutput.
Everything seems to run, however when it comes to dereferencing the pointer, the program exits, I'm assuming this is because I've done the mapping wrong, and the pointer is pointing to something it shouldn't be.
If anyone can spot what I'm doing wrong, or can offer some advice, it would be greatly appreciated.
Thanks!
You've mapped the storage associated with that file (or tried to) into your process, and you've insisted that it be mapped at an address you're already using for something else (presumably, addr was allocated somehow).
You don't say whether p actually does have the address you requested, and as suspectus points out, your error checking is broken.
Your Confusion:
You can't associate arbitrary heap or other process memory pages with a file after the fact. You have to allocate them in the filesystem, and then map them. (There is a way to associate them with a UNIX pipe using vmsplice, although it isn't exactly what you asked for).
Note the MMAP_FIXED flag will just replace the page which was occupied by your data, with the new pages associated with the file. Without that flag, the address hint would be ignored and the mapping placed elsewhere.
The Solution:
ftruncate the file to your desired size before mapping it (this allocates storage in the filesystem)
map it and then populate it
fix your mmap error checking
If you can't change your allocation scheme, the best you can manage is to copy your process-local memory into the mapping, in which case you might as well just write it to the file.
The ideal case would look something like this:
void *alloc_mmap(const char *filename, size_t length)
{
int fd;
fd = open(filename, (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
return NULL;
}
if (ftruncate(fd, length)) {
perror("Couldn't grow output file\n");
close(fd);
return NULL;
}
void *p = mmap(NULL, length, PROT_READ, MAP_SHARED, fd, 0);
if (p == -1) {
perror("mmap failed");
close(fd);
return NULL;
}
close(fd);
return p;
}
// now you've allocated your memory, you can populate it and it will be
// reflected in the file
Here is an extract from the mmap man page.
On success, mmap() returns a pointer to the mapped area. On error, the value
MAP_FAILED (that is, (void *) -1) is returned, and errno is set appropriately.
On success, munmap() returns 0, on failure -1, and errno is set (probably to
EINVAL).
The test for success should be changed to test for -1 return value of mmap. Then check the errno
value. HTH.
Related
In the user space programm I am allocating some memory via mmap as the following function call:
void *memory;
int fd;
fd = open(filepath, O_RDWR);
if (fd < 0)
return errno;
memory = mmap(NULL, 4096, PROT_WRITE, MAP_SHARED, fd, 0);
if (memory == MAP_FAILED)
return -1;
//syscall() goes here
In the kernel space in my system call I am trying to copy data to the memory mapped region as follows:
copy_to_user(memory,src,4096);
EDIT: added error checking code to the post for clarification
The copy_to_user() call is repeatedly failing in this case, whereas if I would have done a memory = malloc() it was succeeding always.
Am I getting some permission flags wrong in this case for mmap ?
Does the open succeed? What about mmap? Is the target file big enough? Can you write to the file through the mapping in userspace?
Also, the repeated 4096 is a strong hit your code is wrong. Userspace should pass the expected size instead.
When I first made this project last semester, the code worked fine. Now I get a bus error when the mmapped memory to share between processes is being written to and I'm not sure why it is not working anymore.
Account_Info *mapData()
{
int fd;
//open/create file with read and write permission and check return value
if ((fd = open("accounts", O_RDWR|O_CREAT, 0644)) == -1)
{
perror("Unable to open account list file.");
exit(0);
}
//map data to be shared with different processes
Account_Info *accounts = mmap((void*)0, (size_t) 100*(sizeof(Account_Info)), PROT_WRITE,
MAP_SHARED, fd, 0);
int count= 0;
//loop to initialize values of Account_Info struct
while (count != 20)
{
//bus error occurs here
accounts[count].CurrBalance= 0;
accounts[count].flag = 0;
int i = 0;
while (i != 100)
{
//place NULL terminator into each element of AccName
accounts[count].AccName[i]= '\0';
i++;
}
count++;
}
close(fd);
return accounts;
}
A documented cause for SIGBUS with mmap is
Attempted access to a portion of the buffer that does not correspond to the file (for example, beyond the end of the file, including the case where another process has truncated the file).
My guess is that the accounts file didn't exist, so open with O_CREAT created it. But it has zero size, so any attempt to read or write through the mapping will fault. You need to fill the file with enough zeroes (or something else) to cover the mapping, for example using ftruncate.
You will get SIGBUS if you attempt to write past the mapped region of the file.
Chances are pretty good that your backing store file accounts is truncated/too short. (e.g.) if the file has space for 10 struct entries and you write to the 11th, you'll get SIGBUS
Do an fstat to get st_size and compare this against the length parameter you're giving to mmap
You may want to consider using ftruncate to extend the file before doing mmap
I'm using shared memory (shm_open/mmap) to keep track of some state. In my shared memory I have the struct:
typedef struct fim_t {
uint64_t num_procs;
uint64_t num_numa;
int64_t *numa_nodes[MAX_FIM_NUMA];
int64_t procs[MAX_FIM_PROC];
}fim_t;
What I want to do is load process IDs in the procs array and then have the numa_nodes array point to procs array values so I can manipulate the value in one spot and have it change across all the references. My understanding is that setting the numa_nodes references to addresses of the procs array should not be a memory access violation because their addresses are both entirely within the shared memory segment. However I get a seg fault when I try to access the value which tells me that my previous statement must be false.
Here is example code:
int main(){
int fd;
int init_flag = 0;
if((fd = shm_open("fim", O_RDWR | O_CREAT | O_EXCL, S_IRWXU)) > 0){
printf("creating shared memory\n");
init_flag = 1;
} else {
printf("opening shared memory\n");
fd = shm_open("fim", O_RDWR, S_IRWXU);
}
if (-1 == fd) {
printf("fd is negative\n");
abort();
}
if ((1 == init_flag) && -1 == ftruncate(fd, sizeof(fim_t))){
printf("ftruncate failed %d\n", errno);
abort();
}
fim_t *fim = mmap(NULL, sizeof(fim_t), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if(MAP_FAILED == fim){
printf("mmap failed\n");
abort();
}
if(init_flag){
fim->num_procs = 1;
my_rank = 0;
for(int x=0;x<MAX_FIM_PROC;x++){
fim->procs[x] = 0;
}
fim->numa_nodes[0] = &(fim->procs[0]);
} else {
my_rank = __sync_fetch_and_add(&(fim->num_procs),1);
fim->procs[my_rank] = my_rank;
fim->numa_nodes[0] = &(fim->procs[my_rank]);
}
printf("my rank is: %"PRId64"\n",my_rank);
sleep(5);
printf("my numa val is %"PRId64"\n",*fim->numa_nodes[0]);
printf("rank %"PRId64" is going down\n", my_rank);
// SHUTDOWN
uint64_t active = __sync_sub_and_fetch(&(fim->num_procs),1);
printf("num active is now %"PRId64"\n", active);
close(fd);
shm_unlink("fim");
return 0;
}
What I expect/hope to happen would be that I run one process then immediately start another and the first process prints "my numa val is 1" (due to the second process setting the numa_node[0] value) and both exit cleanly. However, the second process runs fine, but in the first process seg faults (memory access) at the print statement for numa_node[0] (after the sleep).
So here's my question: Am I doing something wrong or is my approach unworkable? If it is unworkable, is there another way to achieve the result I'm looking for?
You haven't done anything to arrange for all users of the shared memory to map it at the same virtual address. Some *nix systems will do this by default, but most will not.
Either try to map your segment at a fixed address (and deal with failure - this may not succeed) - or store offsets in the shared memory, not actual pointers.
My understanding is that setting the numa_nodes references to addresses of the procs array should not be a memory access violation because their addresses are both entirely within the shared memory segment.
The problem is that different processes map the shared memory to different addresses.
fim_t *fim = mmap(NULL, sizeof(fim_t), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
fim will have different values in different processes. Print it out to check this.
This causes the pointers to the int64_t procs[MAX_FIM_PROC] elements to be different in different processes.
fim is <addr1> in process 1
fim is <addr2> in process 2
&fim->procs[0] will be different in two processes
&fim->procs[0] is <addr1> + <offset> in process 1
&fim->procs[0] is <addr2> + <offset> in process 2
Because these are different values, they cannot be shared between processes. Valid pointer in one process will not be valid in another process.
There are two possible solutions to this.
Force the shared memory to map to the same address in all processes. mmap has an option to accomplish this. Then you can share pointers to elements in the shared memory across processes.
Do not share pointers in shared memory. Share indexes instead.
While trying to resolve some debugging issues , I added some printf-s to my code :
I used that code :
struct PipeShm
{
int init;
sem_t sema;
...
...
}
struct PipeShm * sharedPipe = NULL;
func2:
int func2()
{
if (!sharedPipe)
{
int myFd = shm_open ("/myregion", O_CREAT | O_TRUNC | O_RDWR, 0666);
if (myFd == -1)
error_out ("shm_open");
// allocate some memory in the region in the size of the struct
int retAlloc = ftruncate (myFd, sizeof * sharedPipe);
if (retAlloc < 0) // check if allocation failed
error_out("ftruncate");
// map the region and shared in with all the processes
sharedPipe = mmap (NULL, sizeof * sharedPipe,PROT_READ | PROT_WRITE,MAP_SHARED , myFd, 0);
if (sharedPipe == MAP_FAILED) // check if the allocation failed
error_out("mmap");
// put initial value
int value = -10;
// get the value of the semaphore
sem_getvalue(&sharedPipe->semaphore, &value);
if (sharedPipe->init != TRUE) // get in here only if init is NOT TRUE !
{
if (!sem_init (&sharedPipe->semaphore, 1, 1)) // initialize the semaphore to 0
{
sharedPipe->init = TRUE;
sharedPipe->flag = FALSE;
sharedPipe->ptr1 = NULL;
sharedPipe->ptr2 = NULL;
sharedPipe->status1 = -10;
sharedPipe->status2 = -10;
sharedPipe->semaphoreFlag = FALSE;
sharedPipe->currentPipeIndex = 0;
printf("\nI'm inside the critical section! my init is: %d\n" , sharedPipe->init);
}
else
perror ("shm_pipe_init");
printf("\nI'm out the critical section! my init is: %d\n" , sharedPipe->init);
}
}
return 1; // always successful
}
With that main :
int main()
{
int spd, pid, rb;
char buff[4096];
fork();
func2();
return 0;
}
And got this :
shm_pipe_mkfifo: File exists
I'm inside the critical section! my init is: 1
I'm out the critical section! my init is: 1
Output:hello world!
I'm inside the critical section! my init is: 1
I'm out the critical section! my init is: 1
It seems that the shared memory is not so shared , why ?
The segment is shared between all processes due to MAP_SHARED | MAP_ANONYMOUS , so why both processes have the same before and after values ?
It seems that each process has its own semaphore even though it was shared between them , so what went wrong ?
Thanks
Since you use the MAP_ANONYMOUS flag to mmap, the myFd argument is ignored, and you create two independent shared memory chunks, one in each process, which have no relation to each other.
MAP_ANONYMOUS
The mapping is not backed by any file; its contents are initialā
ized to zero. The fd and offset arguments are ignored; however,
some implementations require fd to be -1 if MAP_ANONYMOUS (or
MAP_ANON) is specified, and portable applications should ensure
this. The use of MAP_ANONYMOUS in conjunction with MAP_SHARED
is only supported on Linux since kernel 2.4.
If you get rid of MAP_ANONYMOUS you'll then only have one shared memory chunk, but you then have the problem of not calling sem_init. On Linux with NPTL it will actually work, as clearing a sem_t to all 0 bytes (the initial state here) is equivalent to sem_init(&sema, anything, 0); (NPTL ignores the pshared flag), but that's not portable to other systems.
Per Karoly's comment on another answer, there's also a race condition due O_TRUNC on the open call. If the second thread calls open after the first thread has already started modifying the semaphore, that TRUNC will clobber the semaphore state. Probably the best solution is to move the code creating, opening, and mmaping the shared memory to a different function that is called BEFORE calling fork.
edit
To fix the O_TRUNC problem, you can't have more than one process calling shm_open with O_TRUNC. But if you just get rid of the O_TRUNC, then you have the startup problem that if the shared memory object already exists (from a previous run of the program), it may not be in a predictable state. On possibility is to split off the beginning of func2:
main() {
func1();
fork();
func2();
}
func1() {
int myFd = shm_open ("/myregion", O_CREAT | O_TRUNC | O_RDWR, 0666);
if (myFd == -1)
error_out ("shm_open");
// allocate some memory in the region in the size of the struct
int retAlloc = ftruncate (myFd, sizeof *sharedPipe);
if (retAlloc < 0) // check if allocation failed
error_out("ftruncate");
// map the region and shared in with all the processes
sharedPipe = mmap (NULL, sizeof *sharedPipe, PROT_READ|PROT_WRITE, MAP_SHARED, myFd, 0);
if (sharedPipe == MAP_FAILED) // check if the allocation failed
error_out("mmap");
}
func2() {
// put initial value
int value = -10;
// get the value of the semaphore
sem_getvalue(&sharedPipe->semaphore, &value);
:
Alternately you could keep the same code (just get rid of O_TRUNC) and add a cleanup before the fork:
main() {
shm_unlink("/myregion");
fork();
func2();
In all cases you'll still have problem if you run multiple copies of your program at the same time.
A few thoughts...
I think this is a fundemental misunderstanding of how POSIX semaphores work. I don't see a call to sem_init or sem_open. You shouldn't be able to use them across processes without doing so much more explicitly than you've done.
I'm not so fresh on the implementation of mmap on Linux and how MAP_ANONYMOUS might affect this, but in general writes to mapped regions can't really be instantaneous. The manpage on linux.die says:
MAP_SHARED
Share this mapping. Updates to the mapping are visible to other processes that map this file, and are carried through to the underlying file. The file may not actually be updated until msync(2) or munmap() is called.
The reason for this is that your memory access gets trapped in a page fault and at that point the kernel will fill contents from the file descriptor, then let you do your write in RAM, then at some later point the kernel will flush back to the file descriptor.
I have an issue attempting to access shared memory using mmap for complex types.
So I allocate my memory as so in my parent process:
/* Create mmap file */
fid = open(TMP_FILE_NAME, O_RDWR | O_CREAT | O_EXCL, (mode_t) 0755);
if (fid < 0)
{
printf("Bad Open of mmap file <%s>\n", TMP_FILE_NAME);
die(-1);
}
/* Make mmap file Buffer Size */
status = ftruncate(fid, INPUT_BUFFER_SIZE);
if (status)
{
printf("Failed to ftruncate the file <%s>, status = %d\n", TMP_FILE_NAME, status);
die(-1);
}
/* Initialize Shared Memory */
mmap_ptr = mmap((caddr_t) 0,
INPUT_BUFFER_SIZE, // Default Buffer Size
PROT_WRITE | PROT_READ, // R/W Permissions
MAP_SHARED, // No file backing
fid,
(off_t) 0);
if (mmap_ptr == MAP_FAILED)
{
printf("Failed to perform mmap, Exiting\n");
die(-1);
}
Now the Struct that I'm passing in memory to my child process is as follows:
/* Data structue for IPC */
typedef struct {
int current_active_id;
int consume_remaining;
Queue buffer;
} input_buffer;
where Queue is a data structure class from the following:
http://www.idevelopment.info/data/Programming/data_structures/c/Queue/Queue.shtml
In my child process it's okay when I do this, it returns the correct value:
printf("Got here... Shared Mem: %d\n", input_queue->consume_remaining);
but when I do something like:
IsEmpty(input_queue->buffer)
it crashes and in the code of the Queue it's only doing this:
return Q->Size == 0;
Any help would be appreciated, thanks!!
Queue is a pointer to struct QueueRecord, and should be allocated as such, presumably using the same shared memory segment. note that this should also be mapped at the same address in both parent and child, or you will not be able to dereference it.
The structure you are putting in the map contains pointers. The pointers are all relative to the address space of the process that created them.
If the other process doesn't mmap at the same address, or if it does but the allocations made for the queue aren't taken from inside that buffer, the pointers will be invalid in the other process.