* glibc detected ./load: double free or corruption (!prev): ADDRESS **
When using glibc, how does it know that I am double-freeing? Does it keep track of everything I malloced and freed? Is it contained in metadata like how free knows how much space to free (How does free know how much to free?)
For each allocation, memory manager keeps some 'header' (most likely tree node or linked list). When you passed to free something that doesn't contain valid header - well, it couldn't correctly be freed. As for where this information is being kept - it's up to implementation, but usually it placed right before address you got from malloc - however, size and structure is very likely to be unknown, but at least it gives an idea how easily this header could be broken/corrupted/overwritten/etc.
When you malloc something, you get a pointer on a memory bloc. You already know that ^^. The memory management also reserve an (hidden) header before* your bloc (which tracks the bloc size for instance)
When you free your pointer, the header is red to check if it is a valid pointer. The free operation also erase the header.
If you free twice, the header will no longer be valid on the second free. Hence the detection.
The C language standard says that freeing a pointer a second time is undefined behavior. What you see in glibc is one particular case of this undefined behavior--a helpful one--where a message is issued. Most allocators keep track of what is allocated and up to a certain amount it can keep track of what has been freed. But you cannot count on this behavior.
A C program is also allowed to silently crash or ignore the situation (or any other action it deems necessary).
The memory allocated by malloc(), calloc() or realloc() does have a metadata for the allocation, which is used by free() to de-allocate the allocated memory.
However, one should not make any assumptions about how or whether the double free is detected as the behavior is undefined in the standard as stated below.
free() frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc(), calloc() or realloc(). Otherwise, or if free(ptr) has already been called before, undefined behavior occurs. If ptr is NULL, no operation is performed.
Related
I've read this question and answer: How do malloc() and free() work?
A friend asked me how can I be sure a free worked. I replied that if it didn't work then the OS is probably crashing and it won't matter for much longer.
But I'm more interested in the nature of the answer to this question.
Once I've freed memory how can I be sure it has been freed? Do I just assume it did?
This is a purely theoretical question with no actual code behind it, and mostly when thinking about I decided "well it doesn't matter anyway", but I'm uneasy with this answer.
For instance if memory is a problem and I want to make sure that a large structure was freed after calling free, otherwise I'll try cleaning up again, is there a way to do this?
Edit:
To those stating my question answered here: Function free() in C isn't working for me
The answer provided there simply states that I "cannot actually test if free() worked". I'm trying to understand why this cannot be done. What is the nature behind free.
Edit2:
After reading through the answers provided it seems that the answer I gave my friend is accepted, that "it just works".
The only way free "doesn't work" is if you pass it an invalid pointer or one that's already been freed. Doing so invokes undefined behavior.
From the man page:
The free() function frees the memory space pointed to by ptr, which
must have been returned by a previous call to malloc(), calloc() or
realloc(). Otherwise, or if free(ptr) has already been called before,
undefined behavior occurs. If ptr is NULL, no operation is performed.
If you run your program under a memory checking tool like valgrind, it will tell you if you're calling free incorrectly.
if memory is a problem and I want to make sure that a large structure was freed after calling free, otherwise I'll try cleaning up again, is there a way to do this?
No, there is no legal way to free the same memory twice. In fact, calling free on a freed pointer leads to undefined behavior. That is why you should set freed pointers to NULL if they are to remain in scope.
There is also no API to see if a memory address has been freed or not, so your code must keep track of what is currently allocated in order to free it at some later point.
You can only do so much returning the memory to the operating system: once you call free, it's out of your hands, so you need to rely on the OS and on the memory manager of your C library to account for the freed memory correctly.
The best you can do on your side is to run your code through a memory profiler under its typical workload, and fix all errors.
You cannot verify it for specific memory allocation. There is no system call to ask if an allocation is still allocated or has already been freed.
However, as memory management and in particular memory leaks (allocations not freed), references to already freed memory, double frees and memory corruption are a serious issue in all programming languages with explicit memory management, there are tools to debug these kind of problems, such as Purify, Insure++, Valgrind or Boundschecker.
free does not return any value, or set errno to be anything (barring whatever implementation-specific features your compiler might have). The documentation for free says:
The free() function frees the memory space pointed to by ptr, which must have been returned by a previous call to malloc(), calloc() or realloc(). Otherwise, or if free(ptr) has already been called before, undefined behavior occurs. If ptr is NULL, no operation is performed.
If you call free on memory that was not allocated using the malloc family of functions, the result is undefined. If your program crashes on calling free (or operator delete in C++), it's a sign of poor memory management elsewhere in your code.
as mentioned above free doesn't return any value but just to verify I initialized one of the members from the structure & printed it before and after using free.(FYI- this is just an experiment)
struct BookList {
int BookNumber;
char BookName[4];
struct BookList * ptr_next;
}BookListVar,*ptr_BookList;
ptr_current = (struct ptr_BookList *)malloc(sizeof(struct BookList));
ptr_current->BookNumber = 123;
printf("BookNumber Stored : %d\n", ptr_current->BookNumber); //got 123 as output
free(ptr_current);
printf("BookNumber Stored : %d\n", ptr_current->BookNumber); //Got some garbage value
I have been taught in lectures, that calling free() on a pointer twice is really, really bad. I know that it is good practice, to set a pointer to NULL, right after having freed it.
However, I still have never heard any explanation as to why that is. From what I understand, the way malloc() works, it should technically keep track of the pointers it has allocated and given you to use. So why does it not know, whether a pointer it receives through free() has been freed yet or not?
I would love to understand, what happens internally, when you call free() on a location that has previously already been freed.
When you use malloc you are telling the PC that you want to reserve some memory location on the heap just for you. The computer gives back a pointer to the first byte of the addressed space.
When you use free you are actually telling the computer that you don't need that space anymore, so it marks that space as available for other data.
The pointer still points to that memory address. At this point that same space in the heap can be returned by another malloc call. When you invoke free a second time, you are not freeing the previous data, but the new data, and this may not be good for your program ;)
To answer your first question,
So why does it not know, whether a pointer it receives through free() has been freed yet or not?
because, the specification for malloc() in C standard does not mandate this. When you call malloc() or family of functions, what it does is to return you a pointer and internally it stores the size of the memory location allocated in that pointer. That is the reason free() does not need a size to clean up the memory.
Also, once free()-d, what happens with the actually allocated memory is still implelentation dependent. Calling free() is just a marker to point out that the allocated memory is no longer in use by the process and can be reclaimed and e re-allocated, if needed. So, keeping track of the allocated pointer is very needless at that point. It will be an unnecessary burden on the OS to keep all the backtracks.
For debugging purpose, however, some library implementations can do this job for you, like DUMA or dmalloc and last but not the least, memcheck tool from Valgrind.
Now, technically, the C standard does not specify any behaviour if you call free() on an already free-ed pointer. It is undefined behavior.
C11, chapter §7.22.3.3, free() function
[...] if
the argument does not match a pointer earlier returned by a memory management
function, or if the space has been deallocated by a call to free() or realloc(), the
behavior is undefined.
C standard only says that calling free twice on a pointer returned by malloc and its family function invoke undefined behavior. There is no further explanation why it is so.
But, why it is bad is explained here:
Freeing The Same Chunk Twice
To understand what this kind of error might cause, we should remember how the memory manager normally works. Often, it stores the size of the allocated chunk right before the chunk itself in memory. If we freed the memory, this memory chunk might have been allocated again by another malloc() request, and thus this double-free will actually free the wrong memory chunk - causing us to have a dangling pointer somewhere else in our application. Such bugs tend to show themselves much later than the place in the code where they occured. Sometimes we don't see them at all, but they still lurk around, waiting for an opportunity to rear their ugly heads.
Another problem that might occure, is that this double-free will be done after the freed chunk was merged together with neighbouring free chunks to form a larger free chunk, and then the larger chunk was re-allocated. In such a case, when we try to free() our chunk for the 2nd time, we'll actually free only part of the memory chunk that the application is currently using. This will cause even more unexpected problems.
When you are calling malloc you are getting a pointer. The runtime library needs to keep track of the malloced memory. Typically malloc does not store the memory management structures separated from the malloc ed memory but in one place. So a malloc for x bytes in fact takes x+n bytes, where one possible layout is that the first n bytes are containing a linked list struct with pointers to the next (and maybe previous) allocated memory block.
When you free a pointer then the function free could walk through it's internal memory management structures and check if the pointer you pass in is a valid pointer that was malloced. Only then it could access the hidden parts of the memory block. But doing this check would be very time consuming, especially if you allocate a lot. So free simply assumes that you pass in a valid pointer. That means it directly access the hidden parts of the memory block and assumes that the linked list pointers there are valid.
If you free a block twice then you might have the problem that someone did a new malloc, got the memory you just freed, overwrites it and the second free reads invalid pointers from it.
Setting a freed pointer to NULL is good practice because it helps debugging. If you access freed memory your program might crash, but it might also just read suspicious values and maybe crash later. Finding the root cause then might be hard. If you set freed pointers to NULL your program will immediately crash when you try to access the memory. That helps massively during debugging.
Suppose that I have used a free() function to free a memory that,for many reasons, I'm not allowed to.
How can I stop my C application from crashing and just generate an error and continue the execution? I don't have try-catch kind of provision here (like C++/java...). Is there any way to ignore this error and continue execution?
If yes,
How do you do that?
More importantly, is it advisable to do so (continuing execution considering this memory error occurred)?
Thank you
It's certainly not advisable. Even if your program's version of free correctly detects that the memory you're trying to free cannot be freed, and thus doesn't do any harm itself, you still have a bug in your program — part of the program thought it owned that memory. Who knows what it might have tried to do with that memory before freeing it? Find and fix the bug. Don't just sweep it under the rug.
There is nothing in the C standard that you can use to do what you want. The description of the free function is very clear on that (§7.20.3.2 in C99):
The free function causes the space pointed to by ptr to be deallocated, that is, made
available for further allocation. If ptr is a null pointer, no action occurs. Otherwise, if
the argument does not match a pointer earlier returned by the calloc, malloc, or
realloc function, or if the space has been deallocated by a call to free or realloc,
the behavior is undefined.
Freeing invalid memory is a serious bug and should be fixed, as it's possible to corrupt the state of your program. Try using a tool like valgrind to spot what's going wrong.
The only pointers you should be using free on are those you receive from malloc, calloc or realloc or NULL pointers. Further you shouldn't use free on the same pointer more than once.
My question is quite simple. We generally allocate memory by declaring a pointer and then assigning a block of memory to that pointer. Suppose somewhere in the code I happen to use
ptr = ptr + 1
and then I use
free(ptr)
can someone tell what will happen. The entire memory block will get deallocated or something else. Can I partially deallocate the memory?
You must always pass exactly the same pointer to free that you got from malloc (or realloc.) If you don't, the "behavior is undefined", which is a term of art that means you can't rely on the program behaving in any predictable way. In this case, though, you should expect it to crash immediately. (If you get unlucky, it will instead corrupt memory, causing a crash some time later, or worse, incorrect output.)
The only way to partially deallocate memory is realloc with a smaller size, but that's only good for trimming at the end and isn't guaranteed to make the trimmed-off chunk available for some other allocation.
It's impossible to deallocate part of a memory block. The only thing you can do is reallocate the block giving it a different size. However this does not guarantee that the block will land in the same place in memory (it might be copied somewhere else).
You must pass to free() the same pointer to the same location malloc() returned. That's because the allocator keeps a sort of list of the blocks you allocated (#everyone: feel free to add/modify if I'm wrong) and if the pointer you pass to free doesn't compare it this list free() complains ("bad memory block" maybe?). To partly deallocate the memory you should use realloc(), which changes the dimensions of that block of memory, but is slow and inefficient. You should use it only when you're sure of the new size of the block or leaving more space to fill in the future.
malloc() , free() and realloc() are not part of C language.
These are functions defines in standard library. Code in standard library that deals with it is usually called "allocator". So, actual answer is "It depends on C library".
On Linux, glibc would crash the program.
On VC++, C runtime would corrupt allocator's state
Source code is available for these libraries, so you can actually put a breakpoint and step into free.
I'm trying to free up the memory I've allocated with malloc, but the free command doesn't seem to do its job properly according to Eclipse's debugger. How's this possible?
Below is a screenshot of my debugger after it supposedly freed up seCurrent->student->year, which is clearly not the case. year was allocated using malloc.
alt text http://img693.imageshack.us/img693/7840/codeo.png
free() does not normally change any values in your program - it just makes adjustments to the C runtime heap. This means that the values in the memory that was just freed are retained. However, attempts to access them from your code lead to undefined behaviour.
What makes you think it hasn't freed it? Freeing memory means that accessing it from the program thereafter is undefined behavior, and the memory is available for re-use next time you call malloc. It does not promise to overwrite the data that was stored in the memory you freed, or to prevent the debugger from reading the unallocated memory.
Free will return the allocated space to the heap to be reused by subsequent mallocs but it does not change the values of any pointers that previously referenced that memory. In your case, no other mallocs have yet been performed so the memory just freed is still the same as it was just prior to the call to free. In order for your code to know that there is no longer any data associated with the pointer, you may want to set it to null after freeing the memory associated with it.
when you malloc() some memory, all it does is searching for some free space in memory, and keeping track it is now used. It doesn't initialize it or whatever.
when you call free(), all it does is clearing this memory block out of the list of used memory blocks. Again it doesn't modify the contents.