malloc under linux, implicit limit - c

Question

Sorry if the title isn't as descriptive as it should be, the problem is hard to put in a few words. I am trying to find out how much mem i have available by malloc'ing and if that worked, writing to that segment. On certain systems (all linux on x86_64) i see segfaults when writing to the 2049th mib. The code is:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <sys/mman.h>
int main (int argc, char **argv) {
void *addr;
int megasize = 2100
/// allocate the memory via mmap same result
//addr = mmap ((void *) 0, (size_t) megasize << 20, PROT_READ | PROT_WRITE,
// MAP_PRIVATE | MAP_ANONYMOUS, (int) -1, (off_t) 0);
addr = malloc(megasize << 20);
if (addr == MAP_FAILED) {
fprintf (stderr, "alloc of %d megabytes failed %s\n", megasize,
strerror (errno));
exit (1);
};
printf ("got %d megabytes at %p\n", megasize, addr);
{
int i;
char *p = addr;
printf("touching the %d Mb memory:\n", megasize);
for (i = 0; i < megasize; i++) {
p[i << 20] = 0;
putchar('.');
if (i%64==63) // commenting out this line i see that it really is the 2049th mb
printf(" #%d\n", i);
fflush(stdout);
};
putchar('\n');
};
/// free the memory
munmap (addr, (size_t) megasize << 20);
return 0;
}
It segfaults reliably on some systems, whereas on others it works fine. Reading the logs for the systems where it fails tells me it's not the oom killer. There are values for megasize that i can choose which will cause malloc to fail but those are larger.
The segfault occurs reliably for any size bigger than 2gib and smaller than the limit where malloc returns -1 for those systems.
I believe there is a limit i am hitting that isn't observed by malloc and i can't figure out what it is. I tried reading out a few of the limits via getrlimit that seemed relevant like RLIMIT_AS and RLIMIT_DATA but those were way bigger.
This is the relevant part of my valgrindlog
==29126== Warning: set address range perms: large range [0x39436000, 0xbc836000) (defined)
==29126== Invalid write of size 1
==29126== at 0x400AAD: main (in /home/max/source/scratch/memorytest)
==29126== Address 0xffffffffb9436000 is not stack'd, malloc'd or (recently) free'd
Can anybody please give me a clue as to what the problem is?

Answer 1

You'll be getting an overflow when counting via int i, as int is 4 bytes wide here:
p[i << 20] = ...
Change
int i;
to be
size_t i;
size_t is the preferred type when addressing memory.

Answer 2

An 32-bit int cannot store the value 2049 mb. You're invoking undefined behavior via signed integer overflow, and happen to be getting a negative number. On most 32-bit machines, when added to a pointer that wraps back around and ends up giving you the address you wanted, by accident. On 64-bit machines, that gives you an address roughly 2047 mb below the start of your block of memory (or wrapped around to the top of the 64-bit memory space).
Use the proper types. Here, i should have type size_t.