My program creates a file using
HANDLE_ERROR(fd = open(path/to/file,O_WRONLY|O_CREAT,0640)))
Mapped it
uint8_t *output_file_addr = (uint8_t *)mmap ( NULL, size , PROT_WRITE, MAP_SHARED, fd, 0 )
if(output_dic_addr == MAP_FAILED){
fprintf(stderr,"%s\n",strerror(errno));
exit(EXIT_FAILURE);
}
The mmap responds with Permission denied
The problem is solved by allocating space for file using posix_fallocate(...).
I thought the mmap will do the allocation as well (wrong judgment)
It is impossible to create a write only mapping on typical hardware and the file has been opened without read permission.
Related
I'd like to declare an array that can start small and grow to a terabyte or so without bumping into anything else or wasting physical memory. There are several of these "miniheaps" with different sized structs in them and I want them to spread themselves out through virtual memory. I can do this if I back each miniheap with a file and that's my main intention, but for testing purposes I'd like a forgetful variant. It certainly isn't allowed to zero the whole terabyte.
The code below works when fd is a file but says "Can't make file mapping" when fd==-1. What should I write instead?
(I'm using linux and have no interest in porting it to anything else.)
void * reserve;
// 'lim*PAGE' is 1TB ...
reserve = mmap(0, lim*PAGE, PROT_NONE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
if (reserve == (void*)-1) { fprintf(stderr, "Can't make reserve mapping\n"); quit(1); }
void * filemap;
// 'fd' is an open file or -1 ...
if (fd!=-1) {
filemap = mmap(reserve, fileSize(fd), PROT_READ|PROT_WRITE, MAP_SHARED_VALIDATE|MAP_FIXED, fd, 0);
} else {
// stp is 1 or 2 and PAGE is 4k ...
filemap = mmap(reserve, PAGE*stp, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_FIXED, fd, 0);
}
if (filemap == (void*)-1) { fprintf(stderr, "Can't make file mapping\n"); quit(1); }
// fileSize and quit are my own utilities for the obvious purposes.
I just added MAP_PRIVATE to the second mmap and it worked. Reading malloc.c is what gave it away.
In the user space programm I am allocating some memory via mmap as the following function call:
void *memory;
int fd;
fd = open(filepath, O_RDWR);
if (fd < 0)
return errno;
memory = mmap(NULL, 4096, PROT_WRITE, MAP_SHARED, fd, 0);
if (memory == MAP_FAILED)
return -1;
//syscall() goes here
In the kernel space in my system call I am trying to copy data to the memory mapped region as follows:
copy_to_user(memory,src,4096);
EDIT: added error checking code to the post for clarification
The copy_to_user() call is repeatedly failing in this case, whereas if I would have done a memory = malloc() it was succeeding always.
Am I getting some permission flags wrong in this case for mmap ?
Does the open succeed? What about mmap? Is the target file big enough? Can you write to the file through the mapping in userspace?
Also, the repeated 4096 is a strong hit your code is wrong. Userspace should pass the expected size instead.
I'm getting -EINVAL for some reason, and it's not clear to me why. Here's where I open and attempt to mmap the file:
if ((fd = open(argv[1], O_RDWR)) < 0)
{
fprintf(stderr, "Failed to open %s: %s\n", argv[1], strerror(errno));
return 1;
}
struct stat statbuf;
if (fstat(fd, &statbuf))
{
fprintf(stderr, "stat filed: %s\n", strerror(errno));
return 1;
}
char* fbase = mmap(NULL, statbuf.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
if (fbase == MAP_FAILED)
{
fprintf(stderr, "mmap failed: %s\n", strerror(errno));
return 1;
}
EDIT: I should add, the error is occurring in the mmap.
Turns out changing the MAP_SHARED to MAP_PRIVATE allows this to succeed.
This reason this was failing is subtle: My code is running inside a VirtualBox VM, and the file I was attempting to mmap was in a shared directory on my host machine. The VirtualBox virtual filesystem apparently doesn't implement mmap with the MAP_SHARED option across the boundary of the hypervisor.
If you'll read jxh's helpful comments on both my question and on his answer, it turns out that this code was working for him because he was likely attempting to mmap a host filesystem file into the host memory.
My observation that switching from MAP_SHARED to MAP_PRIVATE is also consistent with this: since privately mapped memory is invisible to other processes, the virtual filesystem driver will probably have no objection to mapping the memory.
The solution was to move the file I wanted to map into the guest's hard drive and perform manipulation from there.
Your statbuf.st_size is 0. mmap() will fail if the length parameter is 0.
There are 3 listed reasons for EINVAL error mmap():
void *mmap(void *addr, size_t length, int prot, int flags,
int fd, off_t offset);
...
We don't like addr, length, or offset (e.g., they are too large, or not aligned on a page boundary).
(since Linux 2.6.12) length was 0.
flags contained neither MAP_PRIVATE or MAP_SHARED, or contained both of these values.
edit grub to add iomem=relaxed and reboot, make sure cat /proc/cmdline shows entry for iomem=relaxed after boot, re-run your program and check
[root#fedora ~]# cat /proc/cmdline
BOOT_IMAGE=(hd0,gpt2)/vmlinuz-5.18.19-200.fc36.x86_64 root=/dev/mapper/fedora_fedora-root ro rd.lvm.lv=fedora_fedora/root iomem=relaxed rhgb quiet
I have a C program that generates large amounts of data in memory, and I need to share one particular section of this data in memory, so that another process can have read access to it.
I'm attempting to use mmap to do this, but I'm not having much success. Here is my code:
//Code above generates a pointer to the memory section I need to share, named addr
if (infoBlock->memory_size == 1073741824) { //This is the data block I need to share
int err, fd;
fd = open("/tmp/testOutput", (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
goto failedExit;
}
unsigned *p = mmap(addr, 1073741824, PROT_READ, (MAP_SHARED | MAP_FIXED), fd, 0);
if (!p) {perror("mmap failed"); goto failedExit; }
printf("p is now: %p\n", p); //This should point to the shared mapping
printf("%u\n", *p); //Try to print out some data from the mapping
}
After running the program, I can see the file /tmp/testOutput is there, but it's size is 0. I'm not sure if that's a normal thing with memory mappings, as it's not technically a file. Also all of the output within my program points to the same memory address.
I can also see the memory map present within the /proc/PID/maps, with a reference to /tmp/testOutput.
Everything seems to run, however when it comes to dereferencing the pointer, the program exits, I'm assuming this is because I've done the mapping wrong, and the pointer is pointing to something it shouldn't be.
If anyone can spot what I'm doing wrong, or can offer some advice, it would be greatly appreciated.
Thanks!
You've mapped the storage associated with that file (or tried to) into your process, and you've insisted that it be mapped at an address you're already using for something else (presumably, addr was allocated somehow).
You don't say whether p actually does have the address you requested, and as suspectus points out, your error checking is broken.
Your Confusion:
You can't associate arbitrary heap or other process memory pages with a file after the fact. You have to allocate them in the filesystem, and then map them. (There is a way to associate them with a UNIX pipe using vmsplice, although it isn't exactly what you asked for).
Note the MMAP_FIXED flag will just replace the page which was occupied by your data, with the new pages associated with the file. Without that flag, the address hint would be ignored and the mapping placed elsewhere.
The Solution:
ftruncate the file to your desired size before mapping it (this allocates storage in the filesystem)
map it and then populate it
fix your mmap error checking
If you can't change your allocation scheme, the best you can manage is to copy your process-local memory into the mapping, in which case you might as well just write it to the file.
The ideal case would look something like this:
void *alloc_mmap(const char *filename, size_t length)
{
int fd;
fd = open(filename, (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
return NULL;
}
if (ftruncate(fd, length)) {
perror("Couldn't grow output file\n");
close(fd);
return NULL;
}
void *p = mmap(NULL, length, PROT_READ, MAP_SHARED, fd, 0);
if (p == -1) {
perror("mmap failed");
close(fd);
return NULL;
}
close(fd);
return p;
}
// now you've allocated your memory, you can populate it and it will be
// reflected in the file
Here is an extract from the mmap man page.
On success, mmap() returns a pointer to the mapped area. On error, the value
MAP_FAILED (that is, (void *) -1) is returned, and errno is set appropriately.
On success, munmap() returns 0, on failure -1, and errno is set (probably to
EINVAL).
The test for success should be changed to test for -1 return value of mmap. Then check the errno
value. HTH.
Playing around with mmap for the fun of it, I have the following code:
(.. snip ..)
fd = open("/home/me/straight_a.txt", O_RDONLY);
if (fd == -1) {
perror("open");
exit(1);
}
m = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_FILE|MAP_PRIVATE, fd, 0);
if (m == MAP_FAILED) {
perror("mmap");
exit(1);
}
printf("m is %p\n", m);
printf("*m = %c\n", *m);
printf("*(m+1) = %c\n", *(m+1));
(.. snip ..)
This works as expected. But before I got to this, I tried...
m = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_FILE|MAP_SHARED, fd, 0);
... and mmap errored out with:
mmap: Permission denied
In general, what's the difference between the two flags (man page isn't to generous on this subject)? What sort of permission (and where) am I missing?
EDIT
Like it usually happens.. partially figured it out.
Turns out open needed an O_RDWR flag.
So, am I correct to assume that:
MAP_PRIVATE - changes are made in memory only, not saved to disk?
MAP_SHARED - changes would be saved to disk...
... but I'm not saving anything to disk anywhere, I thought? Just operating on memory.
You opened the file in read-only mode. Then you attempted to mmap part of it in read/write mode with MAP_SHARED set. In this context, MAP_SHARED implies that if you write to the mmap'd region your changes will be committed back to the mapped file itself. You can't do this because you opened the file in read-only mode.
MAP_PRIVATE works because writes to the mmap'd region are not committed back to the original file. When you write to the region, the pages that were written to are copied to a different region of memory, possibly backed by swap space.
Writes to a MAP_SHARED segment are carried through to the underlying file. You opened the file with O_RDONLY, which conflicts with the PROT_WRITE flag, thereby preventing MAP_SHARED from being able to write back to the file.
MAP_PRIVATE does not carry writes back to the underlying file, so the fact that you opened the file O_RDONLY is not an issue.