I'm using a queue created with a circle buffer in C by mapping a file to two halves of the same underlying buffer, one after the other. If I attempt to access the buffer immediately after creating it (both before and after mapping the file on top), a bus error is thrown.
I'm programming on a x86 machine, so (if I understand correctly), the only reason this would occur is if the memory location is physically inaccessible. If this is the case, why would mmap return a physically unavailable address?
My code for creating a new code can be seen below.
struct queue create_queue() {
size_t pagesize = getpagesize();
size_t sz = ((BUFFSIZE*sizeof(char *))/pagesize)*(pagesize+1); //align to page
int fd = fileno(tmpfile());
void *buffer = mmap(NULL, 2*sz, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
mmap(buffer, sz, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_SHARED | MAP_FIXED, fd, 0);
mmap(buffer+sz, sz, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_SHARED | MAP_FIXED, fd, 0);
struct queue new_queue;
new_queue.buffer = (char **)buffer;
new_queue.front = 0;
new_queue.back = 0;
sem_init(&(new_queue.sem), 0, 0);
return new_queue;
}
The problem is that:
size_t sz = ((BUFFSIZE*sizeof(char *))/pagesize)*(pagesize+1); //align to page
does set sz to a multiple of the page size, but to a muiliple of the page size plus one. An alternative approach is:
size_t sz = ((BUFFSIZE*sizeof(char *)+pagesize-1)/pagesize)*pagesize; //align to page
Related
i have few question based on below source:
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <stdio.h>
int g;
int main(void) {
int fd = shm_open("/myregion", O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
ftruncate(fd, sizeof(int)); // set size by sizeof(int)
int *p1 = mmap(NULL, 10*sizeof(int), PROT_READ | PROT_WRITE, MAP_SHARED, fd,0); //now map 10*sizeof(int).
if (p1== MAP_FAILED) {
printf("*****************error");
}
*p1 = mmap(NULL, 8*sizeof(int), PROT_READ | PROT_WRITE, MAP_SHARED, fd,0);
if (p1== MAP_FAILED) {
printf("*****************error");
}
*p1=89;
return g;
}
Question 1 :
why i don't see any error while i set size as size_of(int) and then map 10*size_of(int)
Question 2:
how many instace of shared mem is created here? i mean is there only one shared mem created or two as i did mmap twice?
Thanks
Given the code
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <stdio.h>
int g;
int main(void) {
int fd = shm_open("/myregion", O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
ftruncate(fd, sizeof(int)); // set size by sizeof(int)
int *p1 = mmap(NULL, 10*sizeof(int), PROT_READ | PROT_WRITE, MAP_SHARED, fd,0); //now map 10*sizeof(int).
*p1 = mmap(NULL, 8*sizeof(int), PROT_READ | PROT_WRITE, MAP_SHARED, fd,0);
if (!p1){
printf("*****************error");
}
*p1 = g;
*p1=89;
return g;
}
Question 1 : why i don't see any error while i set size as size_of(int) and then map 10*size_of(int)
Because you don't check the return value from mmap(), you don't know if an error happened or not. By immediately calling mmap() again with
*p1 = mmap(NULL, 8*sizeof(int), PROT_READ | PROT_WRITE, MAP_SHARED, fd,0);
you mask any potential errors from the first mmap() call, and you also leak any memory that was successfully allocated.
Question 2: how many instace of shared mem is created here? i mean is there only one shared mem created or two as i did mmap twice?
If the first call succeeded, you mapped two memory segments. If it failed, you only mapped one if the second once succeeded.
If the first call did succeed, you leaked the memory.
Note that if you tried to write to the mmap()'d segment past the end of the size of the file you set with ftruncate(), you will not cause the file to grow. Per the POSIX mmap() documentation:
Note that references beyond the end of the object do not extend the object as the new end cannot be determined precisely by most virtual memory hardware. Instead, the size can be directly manipulated by ftruncate().
On Linux, trying to access mmap()'d data beyond the end of the mapped file will likely result in your process receiving a SIGBUS signal.
If I have one shared memory segment of size 1024, how do I mmap three different sized regions of it? I tried the following, but got a seg fault. I think something is not aligned correctly but I can't seem to isolate where.
fd = shm_open(NAME, FLAGS, MODE);
ftruncate(fd, 1024);
addr0 = mmap(NULL, 50, PROTS, FLAGS, fd, 0);
addr1 = mmap(NULL, 100, PROTS, FLAGS, fd, 50);
addr2 = mmap(NULL, 874, PROTS, FLAGS, fd, 150);
As Adam Martin notes, offsets must be multiples of the page size. But you probably don't need three separate mmaps, and could just mmap once and use pointers to different offsets of that single mapping:
fd = shm_open(NAME, FLAGS, MODE);
ftruncate(fd, 1024);
addr0 = mmap(NULL, 1024, PROTS, FLAGS, fd, 0);
addr1 = (void*)((char*)addr0 + 50);
addr2 = (void*)((char*)addr1 + 100);
Just remember to only call munmap on addr0 (at which point all three pointers become invalid). Alternatively, you mmap from the same start point three times, and adjust each pointer individually (which would allow you to munmap each value individually, just make sure to do so on the original pointer, not the adjusted pointer):
fd = shm_open(NAME, FLAGS, MODE);
ftruncate(fd, 1024);
addr0 = mmap(NULL, 50, PROTS, FLAGS, fd, 0);
char *const baseaddr1 = mmap(NULL, 150, PROTS, FLAGS, fd, 0);
char *const baseaddr2 = mmap(NULL, 1024, PROTS, FLAGS, fd, 0);
addr1 = (void*)(baseaddr1 + 50);
addr2 = (void*)(baseaddr2 + 150);
Your offset (last argument to mmap) must be a multiple of page size man. Also fruncate should be ftruncate, though I assume you introduced that typo copying this over, otherwise your code shouldn't compile.
I have a simple program going this:
int main(void) {
int fd;
const char *text = "This is a test";
fd = open("/tmp/msyncTest", (O_CREAT | O_TRUNC | O_RDWR), (S_IRWXU | S_IRWXG | S_IRWXO) );
if ( fd < 0 ) {
perror("open() error");
return fd;
}
/* mmap the file. */
void *address;
off_t my_offset = 0;
address = mmap(NULL, 4096, PROT_WRITE, MAP_SHARED, fd, my_offset);
if ( address == MAP_FAILED ) {
perror("mmap error. " );
return -1;
}
/* Move some data into the file using memory map. */
strcpy( (char *)address, text);
/* use msync to write changes to disk. */
if ( msync( address, 4096 , MS_SYNC ) < 0 ) {
perror("msync failed with error:");
return -1;
}
else {
printf("%s","msync completed successfully.");
}
close(fd);
unlink("/tmp/msyncTest");
}
Anything wrong with my code? I have made some simple tests and it seems that the problem comes from strcpy. But according to the definition, I see no problem.
If
fd = open("/tmp/msyncTest", (O_CREAT | O_TRUNC | O_RDWR), (S_IRWXU | S_IRWXG | S_IRWXO) );
is successful, fd will refer to a zero-length file (O_TRUNC). The call to mmap()
address = mmap(NULL, 4096, PROT_WRITE, MAP_SHARED, fd, my_offset);
establishes a memory-mapping, but the pages do not correspond to an object.
http://pubs.opengroup.org/onlinepubs/7908799/xsh/mmap.html has the following to say about this situation:
The system always zero-fills any partial page at the end of an object. Further, the system never writes out any modified portions of the last page of an object that are beyond its end. References within the address range starting at pa and continuing for len bytes to whole pages following the end of an object result in delivery of a SIGBUS signal.
Similarly, man mmap on Linux notes
Use of a mapped region can result in these signals:[...] SIGBUS Attempted access to a portion of the buffer that does not correspond to the file (for example, beyond the end of the file, including the case where another process has truncated the file).
Consequently, you must ftruncate() the file to a non-zero length before mmap()ing it (unless you are mmap()ing anonymous memory).
So I am trying to port some code from Linux to Mac OSX. During porting I came across this code.
static void allocZeroMap(unsigned long size, int prot, const char *name)
{
struct map *newnode;
int fd;
char buf[11];
fd = open("/dev/zero", O_RDWR);
if (fd == -1) {
printf("couldn't open /dev/zero\n");
exit(EXIT_FAILURE);
}
newnode = zmalloc(sizeof(struct map));
newnode->name = strdup(name);
newnode->size = size;
newnode->prot = prot;
newnode->type = INITIAL_MAP;
newnode->ptr = mmap(NULL, size, prot, MAP_ANONYMOUS | MAP_SHARED, fd, 0);
if (newnode->ptr == MAP_FAILED) {
perror("mmap");
exit(EXIT_FAILURE);
}
newnode->name = zmalloc(80);
sprintf(newnode->name, "anon(%s)", name);
num_initial_mappings++;
list_add_tail(&newnode->list, &initial_mappings->list);
sizeUnit(size, buf);
printf("mapping[%d]: (zeropage %s) %p (%s)\n",
num_initial_mappings - 1, name, newnode->ptr, buf);
close(fd);
}
So I figured that I would need to just switch newnode->ptr = mmap(NULL, size, prot, MAP_ANONYMOUS | MAP_SHARED, fd, 0); to newnode->ptr = mmap(NULL, size, prot, MAP_ANON | MAP_SHARED, fd, 0); because Mac OSX does not have the MAP_ANONYMOUS flag and uses MAP_ANON instead.
But when I run the program, that mmap call fails with Invalid argument. So my next thought was check the man page for what causes Invalid arguments errors. The first cause is that MAP_FIXED is set and the addr argument is not page aligned, but I'm not calling MAP_FIXED. Second it's says it will return Invalid argument if you don't specify either MAP_PRIVATE or MAP_SHARED but I'm using MAP_SHARED so that shouldn't be it. Next it says the len argument must not be negative, so I printed the size variable and ran it and it's a positive value. The final reason is that the offset argument is not page aligned, but I am passing zero so that shouldn't be the problem either.
So next I checked stack overflow and saw this question mmap with /dev/zero . I tried using MAP_FILE instead of MAP_ANON but now I get Operation not supported by device, which the man page says means I am missing MAP_ANON. So I tried adding MAP_ANON so that I have mmap(NULL, size, prot, MAP_FILE | MAP_ANON| MAP_SHARED, fd, 0); but that returns Invalid argument as well.
So how do I mmap /dev/zero on Mac OSX or at least create a zero-filled memory map on Mac OSX?
You should pass -1 instead of fd as the value of the file descriptor as per mmap() man page. Also as you said MAP_ANON is the correct option for MacOS (at least older versions) and there is not need for MAP_FILE.
I tried using mmap like this:
off_t offset = 0x123456789;
long pagesize = sysconf(_SC_PAGE_SIZE);
int pageoffset = offset % pagesize;
//Open file, get file descriptor
fd = open("./test", O_RDONLY);
map_main = mmap(NULL, 106 + pageoffset, PROT_READ | PROT_EXEC,
MAP_SHARED, fd, offset - pageoffset);
Not behaving as expected.
Am I doing right, for reading file from specific location?
Reference: Segfault while using mmap in C for reading binary files