How the memory management unit(MMU) detects the double free of a pointer?
I know that its a good practice to make the pointer NULL just after freeing it, but suppose programmer does not do it. Is there any MMU mechanism to detect it?
The MMU has nothing to do with it. If you free a pointer allocated with malloc twice you will probably corrupt the C runtime heap. The heap (not the MMU) can in principle protect itself against such things, but most don't. Please note that this has nothing to do with the operating system - neither malloc() nor free() are system calls.
How the memory management unit(MMU) detects the double free of a pointer?
The MMU just does virtual address space -> physical memory mapping, it doesn't know anything about how the heap is organized/how the allocation works/..., that is operating system/allocator work.
How does OS detects the double free then? Whats the mechanism??
It walks the list/bitmap/... of allocated blocks, sees that there's no allocated block with the address you passed to it, so it detects that it's a double free.
However if that block has already been re-allocated, it finds it and correctly free it => but now the code that used the re-allocated block will go nuts, since the memory it has correctly acquired and that it didn't release has become unallocated.
If the allocator protects the unallocated memory marking it as no-read and no-write/removing it from the committed pages of the virtual address space the program will die as soon as that memory is accessed again (but the code that apparently caused the crash will be actually innocent, since it didn't do anything wrong).
Otherwise, the application may still work for some time, until that memory block will be given to some other piece of code that requested some memory. At that point, two pieces of the same application will try to work on the same block of memory, with all the mess that can originate from this.
(Thanks to Pascal Cuoq for pointing out my error.)
No, there is no MMU mechanism to detect it. It is common that calling free on an already free'd address causes the program to crash as the implentation of free does something unexpected and causes a segmentation fault.
Running valgrind is a good way of checking for memory management problems, such as double freeing a pointer.
Setting it to NULL isn't actually a good practice, it hides bugs. Particularly double free()s. Check the OS memory map, something like 0xfeeefeee or 0xdeadbeef is usually good.
You can diagnose double free()s with a debug allocator. Most any decent CRT has one.
Related
Say I have the following program for demonstration purposes only:
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
char *my_memory = malloc(50000);
int *my_int = malloc(sizeof(int));
//Do other things but never free my_memory or my_int
return 0;
}
According to Wikipedia:
In computer science, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations1 in such a way that memory which is no longer needed is not released. A memory leak may also happen when an object is stored in memory but cannot be accessed by the running code.[2]
Sentence #1 implies that a memory leak can be unneeded, unfreed memory. However, many, many programmers I've worked with have stated that "there is no memory leak because the OS frees the memory." These programmers believe that memory leaks only occur if there is no longer a reference point or handle to some allocation, thus making it no longer accessible and no longer able to be freed.
As far as I know, it is true that the OS does free the memory, in regards to modern macOS, Windows, and Linux.
I've used AddressSanitizer, Dr. Memory, and Valgrind; they flag this type of program as having "memory leaks."
So my question is, is an example such as the above, where memory is allocated and not freed before program termination, a memory leak or not?
Yes and no.
Most programs are written with the expectation that they free their memory before termination. But most operating systems are written to not assume anything about the memory used by applications, so they will indeed reclaim that memory after a program has terminated.
It's not that freeing memory in your application code always directly returns it to the operating system anyway--typical memory allocators will just mark the memory as unused and available for use in a subsequent call to malloc().
There are also programs written with the express intent of not freeing their allocated memory. This is a simple, effective way to implement very high performance deallocation, because the operating system sees the allocated memory as a small number of large chunks and takes much less time to deallocate those chunks than if the application were to explicitly free each of them (assuming they were allocated as a much larger number of small objects from the application perspective). This is a valid approach, though it does complicate the use of some analysis tools like valgrind, so some programs avoid it by explicitly allocating and deallocating a small number of large "pools" of memory and internally dividing those up to store small objects. This "pool allocator" approach is analogous to what the C standard malloc() and free() normally do for you.
yes this is a memory leak.
each time you allocate space you must free that space when you don't need it any more.
since allocated memory can be shared between threads, it's hard for OS to detect that there is no more programs that doesn't use that space any more and it will still allocated after the end of the program.
For example, I have a block of memory allocated on C.
void* block = malloc(1024*10);
In runtime, I never change it manually. But, it might be changed because of memory corruption, stray pointer or wild pointer for example.
memset(straypointer, 1, 1);
This will happen in very very rare, BUT, it still has the chance.
So, I wonder whether is possible to know my memory block has be changed unexpected.
I guess some kind of memory pool can do it, but I don't have further idea.
If you are on Windows : don't use malloc but VirtualAlloc. Then fill the memory with whatever you want, then use VirtualProtect to protect that memory.
Then as soon as someone writes to that memory region, your program will crash (or crash into the debugger if debugged). For other systems use a similar method (depending on the system).
struct element {
unsigned long int ip;
int type;
int rtt;
struct element * next;
struct element * edge;
};
I have a linked list. I create new nodes using malloc.
I tried to free up memory using free (ptr to node)
but when I run the traverse function again, I can traverse the linked list and the rtt value is correct as well as the next and edge pointers as I can follow the linked list. ONly the ip value is corrupted. why is this?
The behaviour of malloc() and free() depends heavily on the operating system and C library that you are using. In most implementations there are actually two memory allocators in play:
The OS memory allocator, which uses the virtual memory facilities of the processor to provide a process with its own address space and maps physical memory pages into that address space for use.
The C library memory allocator, which is in fact part of the application code and uses the pages provided by the OS to provide fine-grained memory management facilities, as provided by malloc() and free().
In general, calling free() does one or more of the following:
It marks the pointed-to memory area as free in the C memory allocator. This allows that memory to be reused. free() does not zero-out freed memory.
It may return memory to the OS, depending on the settings of the C memory allocator and whether it is actually possible to free that part of the heap. If memory is not returned to the OS, it can be reused by future malloc() calls by the same application.
If you try to access memory that has been freed, usually one of three things will happen:
The memory has been returned to the OS, and your program, typically, crashes. If you ask me, that is probably the best scenario - you have a problem, sure, but you know it.
The memory has not been reused, therefore your old data is still there. Your program goes on as if nothing was wrong. This is in my opinion the worst case scenario. Your code appears to work correctly and, if Murphy has a say in this, it will continue to do so until it reaches your end users - then it will fail spectacularly.
The memory has been reused by your program, and your code will start messing around with its own data. If you are careful (and lucky?), you will probably notice that the results are off. If not, well...
If you are on Linux/Unix Valgrind is a good tool to catch memory management problems like this. There are also replacement libraries for the C memory allocator, such as DUMA that will also allow you to detect such issues.
Memory is not wiped when you free it - that would be a waste of processor time. It is just allocated to the "free list". That is why your data is still there
Whenever you free a block, you should set the corresponding pointer to NULL, so you don't accidentally reference it - it could be reused at any time.
Actually free doesn't delete anything, it just tells the OS it can use that memory again, for example next time you call malloc() it could overwrite some of your nodes.
Freeing the memory releases it for reuse. It doesn't necessarily destroy the data that was at that location. It is up to you not to access a memory region that has been released because the behavior is undefined (i.e. usually very bad).
If you want the data destroyed for some strange reason, then overwrite the memory area prior to freeing it (e.g. memset(buf, 0, len)).
Most architectures have a memory map where the user application grows towards
lower memory where the stack grows to the opposite direction. I am wondering what
is happening when I write such a big C-program that all the space for the application
and the is taken away, ie, that stack pointer and application try to write to the same
region in memory? I assume in C something like a segmentation fault occurs? Is there
any processor support that tries to avoid such a problem from happening?
Thanks
No, in C you can get out of memory, which you only notice directly if you actually check the return value of malloc et al. If not, you probably get to dereference a null pointer somewhere, making your app crash. But possibly there may be no immediate visible signs, only your memory gets silently corrupted. Since the memory space for the application is managed by the app itself, the processor/OS can't detect such errors. In modern OSs memory space of the OS itself and of other apps is protected from your app, so if you accidentally try to write to memory outside your own memory space, you are likely to get segmentation fault. But inside your own memory space, it is up to yourself to protect your memory.
The stack pointer is limited. Once it tries to go further than allowed, you typically get a StackOverflow exception or interrupt, leading to program termination. This most commonly happens with runaway recursive functions.
Similarly, space for the stack is reserved, and not accessible to the heap allocator. When you try to do a heap allocation (malloc or new) without enough space, the allocator will typically return NULL or throw an OutOfMemory exception.
I disagree with the answer that says "there will be no immediate visible signs, only your memory gets silently corrupted."
You'll either get a StackOverflow or OutOfMemory, depending on which resource was exhausted first.
Abelenky is correct, modern architectures will trap the stack growing past some limit much smaller than all available address space (this is easy to test with a simple recursive function)
Also, "App grows down, stack grows up" doesn't really describe the memory mapping of multithreaded systems anyway, each thread has it's own stack, which has a pre-set maximium size, and the heap is one or more seperately mapped areas of address space.
The easiest way to figure this stuff out is to attach a debugger to a simple test program; you can see the memory regions used by your process in any decent one. be sure to look where your libraries and code are loaded, as well as more than one thread worth of stack.
Can accessing (for read only) memory freed cause an access violation, and, if so, under what circumstances?
Yes, it can. "Access violation" ("segmentation fault", etc) is the response that is normally generated by OS/hardware when the process attempts to access (even just for reading) memory that is known to OS as "empty", "freed" or inaccessible for some other reason. The key moment here is that the OS/hardware must know that the memory is free. Memory management functions of C Standard Library don't necessarily return the 'free'd memory back to OS. They might (and will) keep it for future allocations. So in some cases accessing 'free'd memory will not result in "Access Violation" since from the OS's/hardware's point of view this memory has not been really freed. However, at some point the Standard Library might decide to return the collected free memory back to OS, after which an attempt to access that memory will normally result in "Access Violation".
You're asking "can" and not "will", so your answer is yes. It is undefined behavior to point to memory not owned by your program, therefore anything could happen.
Will it? Depends. That is very OS specific. You might be able to get away with it, but obviously you cannot depend on this. Trying to dereference it could cause an exception, because the OS has reclaimed the memory for it's own uses. (again, OS specific).
On Windows: Managing Virtual Memory in Win32
Free, Reserved, and Committed Virtual Memory
Every address in a process can be
thought of as either free, reserved,
or committed at any given time. A
process begins with all addresses
free, meaning they are free to be
committed to memory or reserved for
future use. Before any free address
may be used, it must first be
allocated as reserved or committed.
Attempting to access an address that is either reserved or free generates
an access violation exception.
Unlikely
Memory managers can theoretically return space to the OS but rarely if ever do so. And without returning the space all the way to the kernel, the MMU will never be involved and so a fault is not possible.
The problem is fragmentation. Allocation of variably-sized blocks is inefficient, and general purpose allocators cannot move allocated blocks around because they don't know what points to what, so they can't coalesce blocks unless they happen to be adjacent.
Measurements indicate that fragmentation overhead tends to be about 50% in steady-state processes, so with every-other-block untouchable it's impossible to return pages unless they are much smaller than blocks, and they generally are not.
Also, the book-keeping challenge of returning pages embedded within the heap is daunting, so most memory managers don't even have the capability, even in the unlikely case that they would have the opportunity.
Finally, the traditional process model was not a sparse object. This kind of low-level software is conservatively developed and lives for a long time. An allocator developed today just might attempt sparse allocation but most subsystems just use whatever is already in the C library, and that is not something that's wise to rewrite casually.
It's certainly allowed to; the C standard says straightforwardly that behavior is undefined if "The value of a pointer that refers to space deallocated by a call to the free or realloc function is used". In practice, owing to the way OSs work, you're more likely to get garbage than a crash, but quite simply you're not allowed to assume anything about what happens when you invoke undefined behavior.
freed memory doesn't belong to you anymore, exactly, corresponding physical memory page is out of your process address space which might have been remapped to other process address space already after your freeing, and that address you accessing have not been allocated physical page and do mapping yet; so "Access violation" or "segfault" will happen if access it even for reading only. it is triggered by the processor hardware in general, e.g. GP#, not by OS.
though if the specific physical page which owns your freed memory is still under controlling of your task context, say partial of the page is still used by your process, then "Access violation" or "segfault" may not occur.
If you're asking this because you've profiled your code and found that accessing freed memory would provide a significant performance boost, then the answer is very rarely, if the freed block is small. If you want to be sure, provide your own alternative implementation of malloc() and free().
We can access it but not encouraged. for example
void main()
{
char *str, *ptr;
str = (char *)malloc(10);
ptr = str;
strcpy(str, "Hello");
printf("%s", str);
free(str);
printf("%s", ptr);
}