mmap() fail with ENOMEM on a 1TB ANONYMOUS file? - c

Question

I'm trying to mmap an 1TB anonymous file under Fedora Linux x86_64 (4G RAM plus 16G swap). But I got ENOMEM "cannot allocate memory" and even for 32G as the following code. Am I missing anything? Appreciate any clue.
#define HEAP_SIZE (1UL << 35)
int main()
{
void *addr = mmap(0, HEAP_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
if (addr == MAP_FAILED)
{
perror(NULL);
return 1;
}
printf("mmap %d gbytes succeed\n", HEAP_SIZE/(1UL << 30));
return 0;
}

Answer 1

The default Linux overcommit policy prevents you from allocating this much memory. You don't have anywhere near 1TB of RAM, and the kernel will give you ENOMEM now rather than running the OOM killer later... but you can change this policy.
$ /sbin/sysctl vm.overcommit_memory
vm.overcommit_memory = 0
$ sudo /sbin/sysctl vm.overcommit_memory=1
vm.overcommit_memory = 1
Policy 1 is "always overcommit", which is useful for certain applications. Policy 2 is "never overcommit". The default policy, 0, allows some overcommit but uses heuristics to reject large allocations, like the one which failed on your computer.
Alternative
You could also use the MAP_NORESERVE flag. Note that the kernel will ignore this flag if its policy is to "never overcommit".