Reading into heap memory I shouldn't have access to? - c

Question

In the example below I have allocated 20 bytes of memory to extend an array by 5 integers. After that I have set the last element to 15 and then reallocated the pointer to 4 bytes (1 integer). Then I print the first 10 elements of the array (it only consists of 6 at this point) and the 9th (which I've previously set to 15) is printed without warnings or errors.
The code :
#include <stdlib.h>
#include <stdio.h>
int main()
{
int arr[5] = {0};
int *ptr = &arr[0];
ptr = malloc(5 * sizeof(int));
arr[9] = 15;
ptr = realloc(ptr, 1 * sizeof(int));
for (int i = 0; i < 10; ++i)
{
printf("%d\n", arr[i]);
}
free(ptr);
return 0;
}
The result after compiling and running :
0
0
0
0
0
32766
681279744
-1123562100
-1261131712
15
My question is as follows : Why is the 9th element of the array still 15? (why am I able to access it?; Shouldn't the allocated memory be at the first free block of memory my compiler finds and not connected to the array's buffer whatsoever?)

Answer 1

The behaviour of malloc() \ realloc() is irrelevant in this case because in the code in the question the content of arr rather than ptr is modified and displayed, and arr is not dynamically allocated or reallocated. So there is no out-of-bounds access in the dynamic memory. The out-of-bounds access to arr[] has undefined behaviour. You will be stomping on memory not allocated to arr. In some cases that will modify adjacent variables, but in this case you have none, so since stacks most often grow downward, you may be modifying the local variables of the calling function or corrupting the return address of the current function - this being main() even that might not cause any noticeable error. In other cases it will lead to a crash.
However, had you modified ptr[15] and reallocated, then displayed the content at ptr it is most likely that you see a similar result because avoid an unnecessary data move, realloc() reuses the same memory block when the allocation is reduced, and simply reduces its size, returning the remainder to the heap.
Returning memory to the heap, does not change its content or make it inaccessible, and C does not perform any bounds checking, so if you code to access memory that is not part of the allocation it will let you. It simply makes the returned block available for allocation.
Strictly it is undefined behaviour, so other behaviour is possible, but generally C does not generate code to do anything other than the bare minimum required - except possibly in some cases to support debugging.

Answer 2

Your description of what the program is doing is all wrong.
In the example below I have allocated 20 bytes of memory to extend an array by 5 integers
No, you don't. You can't extend arr. It's just impossible.
After that I have set the last element to 15
No - because you didn't extend the array so index 9 does not represent the last element. You simply write outside the array.
Look at these lines:
int *ptr = &arr[0];
ptr = malloc(5 * sizeof(int));
First you make ptr point to the first element in arr but rigth after you you make ptr point to some dynamic allocated memory which have absolutely no relation to arr. In other words - the first line can simply be deleted (and probably the compiler will).
In the rest of your program you never use ptr for anything. In other words - you can simply remove all code using ptr. It has no effect.
So the program could simply be:
int main()
{
int arr[5] = {0};
arr[9] = 15;
for (int i = 0; i < 10; ++i)
{
printf("%d\n", arr[i]);
}
return 0;
}
And it has undefined behavior because you access arr out of bounds.

Answer 3

Why is the 9th element of the array still 15?
The "most likely reality" is that the OS provides a way to allocate area/s of virtual pages (which aren't necessarily real memory and should be considered "pretend/fake memory"), and malloc() carves up the allocated "pretend/fake memory" (and allocates more area/s of virtual pages if/when necessary, and deallocates areas of virtual pages if/when convenient).
Freeing "pretend/fake memory that was carved up by malloc()" probably does no more than alter some meta-data used to manage the heap; and is unlikely to cause "pretend/fake memory" to be deallocated (and is even less likely to effect actual real physical RAM).
Of course all of this depends on the environment the software is compiled for, and it can be completely different; so as far as C is concerned (at the "C abstract machine" level) it's all undefined behavior (that might work like I've described, but may not); and even if it does work like I've described there's no guarantee that something you can't know about (e.g. a different thread buried in a shared library) won't allocate the same "pretend/fake memory that was carved up by malloc()" immediately after you free it and won't overwrite the data you left behind.
why am I able to access it?
This is partly because C isn't a managed (or "safe") language - for performance reasons; typically there are no checks for "array index out of bounds" and no checks for "used after it was freed". Instead, bugs cause undefined behavior (and may be critical security vulnerabilities).

Answer 4

int arr[5] = {0}; // these 5 integers are kept on the stack of the function
int *ptr = &arr[0]; // the pointer ptr is also on the stack and points to the address of arr[0]
ptr = malloc(5 * sizeof(int)); // malloc creates heap of size 5 * sizeof int and returns a ptr which points to it
// the ptr now points to the heap and not to the arr[] any more.
arr[9] = 15; //the array is of length 5 and arr[9] is out of the border of maximum arr[4] !
ptr = realloc(ptr, 1 * sizeof(int)); //does nothing here, since the allocated size is already larger than 1 - but it depends on implementation if the rest of 4 x integer will be free'd.
for (int i = 0; i < 10; ++i) // undefined behavior!
{
printf("%d\n", arr[i]);
}
free(ptr);
return 0;`

Answer 5

In short:
Whatever you do with/to a copy of the address of an array inside a pointer variable, it has no influence on the array.
The address copy creates no relation whatsoever between the array and memory allocated (and referenced by the pointer) by a later malloc.
The allocation will not be right after the array.
A realloc of a pointer with a copy of an array access does not work. Realloc only works with pointers which carry the result of a succesful malloc. (Which is probably why you inserted the malloc.)
Longer:
Here are some important facts on your code, see my comments:
#include <stdlib.h>
#include <stdio.h>
int main()
{
int arr[5] = {0}; /* size 5 ints, nothing will change that */
int *ptr = &arr[0]; /* this value will be ignored in the next line */
ptr = malloc(5 * sizeof(int)); /* overwrite value from previous line */
arr[9] = 15; /* arr still only has size 5 and this access beyond */
ptr = realloc(ptr, 1 * sizeof(int)); /* irrelevant, no influence on arr */
for (int i = 0; i < 10; ++i) /* 10 is larger than 5 ... */
{
printf("%d\n", arr[i]); /* starting with 5, this access beyond several times */
}
free(ptr);
return 0;
}
Now let us discuss your description:
In the example below I have allocated 20 bytes of memory ....
True, in the line ptr = malloc(5 * sizeof(int)); (assuming that an int has 4 bytes; not guaranteed, but let's assume it).
... to extend an array by 5 integers.
No. No attribute of the array is affected by this line. Especially not the size.
Note that with the malloc, the line int *ptr = &arr[0]; is almost completely ignored. Only the part int *ptr; remains relevant. The malloc determines the value in the pointer and there is no relation to the array whatsoever.
After that I have set the last element to 15 ...
No, you access memory beyond the array. The last useable array element is arr[4] noce code until now has changed that. Judgin from the output, which still contains "15", you got "lucky", the value has not killed anything and still is in memory. But it is practically unrelated to the array and is also practically guaranteed outside of the allocated memory referenced by ptr.
... and then reallocated the pointer to 4 bytes (1 integer).
True. But I do not really get the point you try to make.
Then I print the first 10 elements of the array ...
No, you print the first 5 elements of the array, i.e. all of them.
Then you print 3 values which happen to be inside memory which you should not access at all. Afterwards you print a fifth value outside of the array, which you also should not access, but which happens to be still be the 15 you wrote there earlier - and should not have in the first place either.
... (it only consists of 6 at this point) ...
You probabyl mean 5 values from the array and 1 from ptr, but they are unrelated and unlikely to be consecutive.
... and the 9th (which I've previously set to 15) is printed without warnings or errors.
There is no 9th, see above. Concerning the lack of errors, well, you are not always lucky enough to be told by the compiler or the runtime that you make a mistake. Life would be so much easier if they could notify you of reliably all mistakes.
Let us go on with your comments:
But isn't arr[9] part of the defined heap?
No. I am not sure what you mean by "the defined heap", but it is surely neither part of the array nor the allocated memory referenced by the pointer. The chance that the allocation is right after the array is as close to zero as it gets - maybe not precisely 0, but you simply are not allowed to assume that.
I have allocated 20 bytes, ...
On many current machines, but assuming that an int has four bytes is also not a afe assumption. However, yes, lets assume that 5 ints have 20 bytes.
... so arr should now consist of 10 integers, instead of 5.
Again no, whatever you do via ptr, it has no influence on the array and there is practically no chance that the ptr-referenced memory is right after the array by chance. It seems that you assume that copying the address of the array into the pointer has an influence on array. That is not the case. It had once a copy of the arrays address, but even that has been overwritten one line later. And even if it had not been overwritten, reallocing the ptr would make an error (that is why you inserted the malloc line, isn't it?) but still not have any effect on the array or its size.
... But I don't think I am passing the barrier of the defined heap.
Again, lets assume that by "the defined heap" you mean either the array or the allocated memory referenced by ptr. Neither can be assumed to contain the arr[9] you access. So yes, you ARE accessing outside of any memory you are allowed to access.
I shouldn't be able to access arr[9], right?
Yes and no. Yes, you are not allowed to do that (with or without the realloc to 1).
No, you cannot expect to get any helpful error message.
Let's look at your comment to another answer:
My teacher in school told me that using realloc() with a smaller size than the already allocated memory frees it until it becomes n bytes.
Not wrong. It is freed, which means you are not allowed to use it anymore.
It is also theoretically freed so that it could be used by the next malloc. That does however not mean that the next malloc will. In no case implies freeing memory any change to the content of that freed memory. It definitly could change, but you cannot expect it or even rely on it. Tom Kuschels answer to this comment is also right.