When I use mmaping for a large file(80-150 GB) valgrind failed with the error (mmap): Invalid argument, intuitively, I realized that the problem was probably with the file size, I tried to reduce the file size by half and it worked.
The question is, how to make mmaping in Valgrind "unlimited"?
Some details:
System details: ubuntu 20.04, x86, valgrind 3.21, 8 GB RAM, 2 CPU
Command: valgrind --tool=massif --pages-as-heap=yes ./app
mmap call: ptr = mmap(nullptr, len, PROT_READ, MAP_SHARED, fd, 0);
working mmap call: ptr = mmap(nullptr, len/2, PROT_READ, MAP_SHARED, fd, 0);
Related
In mallinfo structure there are two fields hblks and hblkhd. The man documentation says that they are responsible for the number of blocks allocated by mmap and the total number of bytes. But when I run next code
void * ptr = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
*(int *) ptr = 10;
Fields hblks and hblkhd are also zero. While the total number of free bytes in the blocks decreases. Could you please explain why this behavior is observed?
I also tried to allocate all free space and use mmap after it. But in this situation fields also equal to zero
Compiler: gcc 9.4.0
OS: Ubuntu 20.04.1
I did some experiments and they led me to the conclusion that this field is filled only when mmap occurred when calling malloc. A normal mmap call doesn't show up in this statistic, which is logical, because this is a system call, and the statistic is collected in user-space
In the user space programm I am allocating some memory via mmap as the following function call:
void *memory;
int fd;
fd = open(filepath, O_RDWR);
if (fd < 0)
return errno;
memory = mmap(NULL, 4096, PROT_WRITE, MAP_SHARED, fd, 0);
if (memory == MAP_FAILED)
return -1;
//syscall() goes here
In the kernel space in my system call I am trying to copy data to the memory mapped region as follows:
copy_to_user(memory,src,4096);
EDIT: added error checking code to the post for clarification
The copy_to_user() call is repeatedly failing in this case, whereas if I would have done a memory = malloc() it was succeeding always.
Am I getting some permission flags wrong in this case for mmap ?
Does the open succeed? What about mmap? Is the target file big enough? Can you write to the file through the mapping in userspace?
Also, the repeated 4096 is a strong hit your code is wrong. Userspace should pass the expected size instead.
I'm given a physical address, specifically 0x000000368d76c0. I'm trying to mmap it into my program. The code that I'm using is
void *mmap64;
off_t offset = 0x000000368d76c0;
int memFd = open("/dev/mem", O_RDWR);
if (-1 == memFd)
perror("Error ");
mmap64 = mmap(0, sizeof(uint64_t), PROT_WRITE | PROT_READ, MAP_SHARED, memFd, offset);
if (MAP_FAILED == mmap64) {
perror("Error ");
return -1;
}
For some reason when I run this code I get a failure on mmap. Specifically it says Error Invalid argument. I'm pretty sure it is because of the offset value, but I don't know what is wrong with it.
I would appreciate any help on it.
According to mmap(2) - Linux manual page,
offset must be a multiple of the page size as
returned by sysconf(_SC_PAGE_SIZE).
When the page size is 4096 (a page size used in x86 CPU), 0x000000368d76c0 is not a multiple of 4096 and will be considered as invalid.
For that reason, you will have to adjust the offset.
I'm trying to mmap an 1TB anonymous file under Fedora Linux x86_64 (4G RAM plus 16G swap). But I got ENOMEM "cannot allocate memory" and even for 32G as the following code. Am I missing anything? Appreciate any clue.
#define HEAP_SIZE (1UL << 35)
int main()
{
void *addr = mmap(0, HEAP_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
if (addr == MAP_FAILED)
{
perror(NULL);
return 1;
}
printf("mmap %d gbytes succeed\n", HEAP_SIZE/(1UL << 30));
return 0;
}
The default Linux overcommit policy prevents you from allocating this much memory. You don't have anywhere near 1TB of RAM, and the kernel will give you ENOMEM now rather than running the OOM killer later... but you can change this policy.
$ /sbin/sysctl vm.overcommit_memory
vm.overcommit_memory = 0
$ sudo /sbin/sysctl vm.overcommit_memory=1
vm.overcommit_memory = 1
Policy 1 is "always overcommit", which is useful for certain applications. Policy 2 is "never overcommit". The default policy, 0, allows some overcommit but uses heuristics to reject large allocations, like the one which failed on your computer.
Alternative
You could also use the MAP_NORESERVE flag. Note that the kernel will ignore this flag if its policy is to "never overcommit".
Has anybody succeeded in mmap'ing a /proc/pid/mem file with Linux kernel 2.6? I am getting an ENODEV (No such device) error. My call looks like this:
char * map = mmap(NULL, PAGE_SIZE, PROT_READ, MAP_SHARED, mem_fd, offset);
And I have verified by looking at the /proc/pid/maps file while debugging that, when execution reaches this call, offset has the value of the top of the stack minus PAGE_SIZE. I have also verified with ptrace that mmap is setting errno to ENODEV.
See proc_mem_operations in /usr/src/linux/fs/proc/base.c: /proc/.../mem does not support mmap.