I have been recently trying to learn how to program in the C programming language.
I am currently having trouble understanding how memory is deallocated by free() in C.
What does it mean to free or release the memory?
For instance, if I have the following pointer:
int *p = malloc(sizeof(int));
When I deallocate it using free(p), what does it do? Does it somehow flag it as "deallocated", so the application may use it for new allocations?
Does it deallocates only the pointer address, or the address being pointed is also deallocated too?
I would do some experiments myself to better understand this, but I am so newbie in the subject that I don't know even how to debug a C program yet (I'm not using any IDE).
Also, what if int *p is actually a pointer to an array of int?
If I call free(p), does it deallocate the whole array or only the element it is pointing to?
I'm so eager to finally understand this, I would very much appreciate any help!
What does it mean to free or release the memory?
It means that you're done with the memory and are ready to give it back to the memory allocator.
When I deallocate it using free(p), what does it do?
The specifics are implementation dependent, but for a typical allocator it puts the block back on the free list. The allocator maintains a list of blocks that are available for use. When you ask for a chunk of memory (by calling malloc() or similar) the allocator finds an appropriate block in the list of free blocks, removes it (so it's no longer available), and gives you a pointer to the block. When you call free(), the process is reversed -- the block is put back on the free list and thereby becomes available to be allocated again.
Importantly, once you call free() on a pointer, you must not dereference that pointer again. A common source of memory-related errors is using a pointer after it has been freed. For that reason, some consider it a helpful practice to set a pointer to nil immediately after freeing it. Similarly, you should avoid calling free() on a pointer that you didn't originally get from the allocator (e.g. don't free a pointer to a local variable), and it's never a good idea to call free() twice on the same pointer.
Does it deallocates only the pointer address, or the address being pointed is also deallocated too?
When you request a block of memory from the allocator, you specify the size of the block you want. The allocator keeps track of the size of the block so that when you free the block, it knows both the starting address and the block size. When you call free(p), the block that p points to is deallocated; nothing happens to the pointer p itself.
Also, what if int *p is actually a pointer to an array of int?
An array in C is a contiguous block of memory, so a pointer to the first element of the array is also a pointer to the entire block. Freeing that block will properly deallocate the entire array.
I'm so eager to finally understand this, I would very much appreciate any help!
There are a number of good pages about memory allocation in C that you should read for a much more detailed understanding. One place you could start is with the GNU C Library manual section on memory allocation.
As alluded to above and in the other answers, the actual behavior of the allocator depends on the implementation. Your code shouldn't have any particular expectations about how memory allocation works beyond what's documented in the standard library, i.e. call malloc(), calloc(), etc. to get a block of memory, and call free() to give it back when you're done so that it can be reused.
malloc and free do whatever they want. Their expected behaviour is that malloc allocates a block of desired size in dynamic memory and returns a pointer to it. free must be able to receive one such pointer and correctly deallocate the block. How they keep track of the block size is irrelevant.
Is int *p a pointer to an array of ints ? Maybe. If you allocated sufficient space for several ints, yes.
There is a fixed and limited amount of memory in your computer, and everybody wants some. The Operating system is charged with the task of assigning ownership to pieces of memory and keeping track of it all to assure that no one messes with anyone else's.
When you ask for memory with malloc(), you're asking the system (the C runtime and the OS) to give you the address of a block of memory that is now yours. You are free to write to it and read from it at will, and the system promises that no one else will mess with it while you own it. When you de-allocate it with free(), nothing happens to the memory itself, it's just no longer yours. What happens to it is none of your business. The system may keep it around for future allocations, it may give it to some other process.
The details of how this happens vary from one system to another, but they really don't concern the programmer (unless you're the one writing the code for malloc/free). Just use the memory while it's yours, and keep your hands off while it's not.
Related
I'm trying to figure out how many bytes in a block are taken up by the boundary tags. I have been told that when trying to malloc an adjacent block of memory, a "jump" will appear in assembly code, and I can use that to determine the size of the boundary tag. I've tried this:
int* arr = malloc(8);
arr++;
arr = malloc(8);
But there isn't any jump in assembly code. Am I "trying to malloc an adjacent block of memory"?
EDIT: I think he means a jump will appear between address value. I use the beginning of the second block of memory subtract the payload size of the first block. But I'm still confused, how could I malloc an adjacent block of memory?
Unless you're writing an actual memory allocator, you can't actually allocate two consecutive chunks of memory. If you want to see some pretty gnarly code which does this, have a look at the Illumos malloc https://github.com/illumos/illumos-gate/blob/master/usr/src/lib/libc/port/gen/malloc.c.
If you want to see how Illumos (and Solaris) handle the redzone between allocated blocks, you should trawl through https://github.com/illumos/illumos-gate/tree/master/usr/src/lib/libumem/common.
The memory consumed by malloc(3) requires, for proper management of the actually used memory, of some structures that must be dynamically allocated also. For this reason, many allocators just do allocate the space required for the management data adjacent to the block space dedicated to the user. This makes that normally two consecutive junks of memory allocated by malloc(2) show some gap in their addresses.
There are other reasons to see gaps, one fundamental is that malloc normally gives you aligned memory addresses, so it is warranted that the data you store on that memory will be properly aligned.
And of course, there can be implementations (normally when heap allocation should be more robust in respect to buffer overruns) that the memory dedicated to storage of management data is completely unrelated and apart off the final given memory. In this case you could observe no gaps between memory allocations on some cases.
Anyway, your code has serious bugs, let's see:
int* arr = malloc(8);
You had better here to acquire just the memory you need, using the sizeof operator, as in int *arr = malloc(sizeof *arr); instead.
arr++;
this statement is useless, as you are going to overwrite the value of arr (the pointer) with new assignment statement after it from malloc(), so it is of no use to increment the pointer value. You also are somewhat losing the returned value of the previous malloc() (which is essential in order to return the allocated memory, but read below).
arr = malloc(8);
Until here, you had the chance to --arr decrementing the value of arr in order to be capable of free(3) that block. But this statement overwrites the value stored in arr so the previous pointer value is overwritten by the new pointer. Memory you acquired on the first malloc has no way to be accessed again. This is what is commonly known as a memory leak, and is normally a serious error (very difficult to catch) on long run programs (like servers or system daemons). The program allocates a bunch of memory in the inner part of a loop, that is not returned back with a call to free(3), so the program begins growing and growing until it exhausts all the available memory.
A final note, I don't understand what did you mean with malloc adjacent block of memory. Did you believe that incrementing the pointer would make malloc() to give you a special block of memory?
First, malloc has no idea of what are you going to do with the pointer it gives to you.
But also, it doesn't know anything about the variable contents of the pointer you are assigning to (you can even not store it in a variable, and pass it as another parameter to another functions) So there's no possibility for malloc to know that you have incremented the pointer value, or even the pointer location, from its body.
I cannot guess your interpretation. It would be nice to know what has made you to think that you can control how malloc(3) selects the block of memory to give to you. You have no control on the internals of malloc() You just specify the amount of continous memory you want, and mallocs provides it, giving you a pointer pointing to the start of that block. You cannot assume that the next time you call malloc (with the same or different amount of memory) it will give you an adjacent block. It just can be completely unrelated (above or below in memory) to the previous given block. And you cannot modify that pointer, because you need it to call free(3) once you don't need the block anymore, with exactly the same pointer value that malloc(3) gave to you. If, for some reason you modify the pointer, you need to restore it to the original value to be capable of calling free(3). Lack to do so, you'll probably crash your program at the next call to free(3).
I just see a memory leak. Malloc 2 times into different vars 8 bytes of space and see if the difference is more than 8 bytes or 2 int.
I have been taught in lectures, that calling free() on a pointer twice is really, really bad. I know that it is good practice, to set a pointer to NULL, right after having freed it.
However, I still have never heard any explanation as to why that is. From what I understand, the way malloc() works, it should technically keep track of the pointers it has allocated and given you to use. So why does it not know, whether a pointer it receives through free() has been freed yet or not?
I would love to understand, what happens internally, when you call free() on a location that has previously already been freed.
When you use malloc you are telling the PC that you want to reserve some memory location on the heap just for you. The computer gives back a pointer to the first byte of the addressed space.
When you use free you are actually telling the computer that you don't need that space anymore, so it marks that space as available for other data.
The pointer still points to that memory address. At this point that same space in the heap can be returned by another malloc call. When you invoke free a second time, you are not freeing the previous data, but the new data, and this may not be good for your program ;)
To answer your first question,
So why does it not know, whether a pointer it receives through free() has been freed yet or not?
because, the specification for malloc() in C standard does not mandate this. When you call malloc() or family of functions, what it does is to return you a pointer and internally it stores the size of the memory location allocated in that pointer. That is the reason free() does not need a size to clean up the memory.
Also, once free()-d, what happens with the actually allocated memory is still implelentation dependent. Calling free() is just a marker to point out that the allocated memory is no longer in use by the process and can be reclaimed and e re-allocated, if needed. So, keeping track of the allocated pointer is very needless at that point. It will be an unnecessary burden on the OS to keep all the backtracks.
For debugging purpose, however, some library implementations can do this job for you, like DUMA or dmalloc and last but not the least, memcheck tool from Valgrind.
Now, technically, the C standard does not specify any behaviour if you call free() on an already free-ed pointer. It is undefined behavior.
C11, chapter ยง7.22.3.3, free() function
[...] if
the argument does not match a pointer earlier returned by a memory management
function, or if the space has been deallocated by a call to free() or realloc(), the
behavior is undefined.
C standard only says that calling free twice on a pointer returned by malloc and its family function invoke undefined behavior. There is no further explanation why it is so.
But, why it is bad is explained here:
Freeing The Same Chunk Twice
To understand what this kind of error might cause, we should remember how the memory manager normally works. Often, it stores the size of the allocated chunk right before the chunk itself in memory. If we freed the memory, this memory chunk might have been allocated again by another malloc() request, and thus this double-free will actually free the wrong memory chunk - causing us to have a dangling pointer somewhere else in our application. Such bugs tend to show themselves much later than the place in the code where they occured. Sometimes we don't see them at all, but they still lurk around, waiting for an opportunity to rear their ugly heads.
Another problem that might occure, is that this double-free will be done after the freed chunk was merged together with neighbouring free chunks to form a larger free chunk, and then the larger chunk was re-allocated. In such a case, when we try to free() our chunk for the 2nd time, we'll actually free only part of the memory chunk that the application is currently using. This will cause even more unexpected problems.
When you are calling malloc you are getting a pointer. The runtime library needs to keep track of the malloced memory. Typically malloc does not store the memory management structures separated from the malloc ed memory but in one place. So a malloc for x bytes in fact takes x+n bytes, where one possible layout is that the first n bytes are containing a linked list struct with pointers to the next (and maybe previous) allocated memory block.
When you free a pointer then the function free could walk through it's internal memory management structures and check if the pointer you pass in is a valid pointer that was malloced. Only then it could access the hidden parts of the memory block. But doing this check would be very time consuming, especially if you allocate a lot. So free simply assumes that you pass in a valid pointer. That means it directly access the hidden parts of the memory block and assumes that the linked list pointers there are valid.
If you free a block twice then you might have the problem that someone did a new malloc, got the memory you just freed, overwrites it and the second free reads invalid pointers from it.
Setting a freed pointer to NULL is good practice because it helps debugging. If you access freed memory your program might crash, but it might also just read suspicious values and maybe crash later. Finding the root cause then might be hard. If you set freed pointers to NULL your program will immediately crash when you try to access the memory. That helps massively during debugging.
So I have allocated 256 blocks in heap:
char* ptr1 = malloc(128);
char* ptr2 = malloc(128);
Now after I free ptr2 which I assume currently lies on top of the heap, the program break(the current location of the heap) does not decrease. However if I do another malloc the address returned by malloc is the same as the one that is freed.
So I have the following questions:
When I free a block why does not the program break decrease?
When I call free what exactly happens?How does it keep track of the freed memory so that next time I declare malloc the address is the same?
It's unspecified behavior. You can not rely on any single answer, unless you only care about one particular platform/os/compiler/libc combination. You did not specify an OS, and the C standard does not describe, or require any particular implementation. From C99 (I don't have the final published version of C11 yet):
7.20.3
The order and contiguity of storage allocated by successive calls to
the calloc, malloc, and realloc functions is unspecified. The pointer
returned if the allocation succeeds is suitably aligned so that it may
be assigned to a pointer to any type of object and then used to access
such an object or an array of such objects in the space allocated
(until the space is explicitly deallocated). The lifetime of an
allocated object extends from the allocation until the deallocation.
Each such allocation shall yield a pointer to an object disjoint from
any other object. The pointer returned points to the start (lowest
byte address) of the allocated space. If the space cannot be
allocated, a null pointer is returned. If the size of the space
requested is zero, the behavior is implementation- defined: either a
null pointer is returned, or the behavior is as if the size were some
nonzero value, except that the returned pointer shall not be used to
access an object.
This manual of GNU libc , might be of help.
Here's the gist
Occasionally, free can actually return memory to the operating system
and make the process smaller. Usually, all it can do is allow a later
call to malloc to reuse the space. In the meantime, the space remains
in your program as part of a free-list used internally by malloc.
When I free a block why does not the program break decrease?
I believe it doesn't decrease because that memory has already been given to the program.
When I call free() what exactly happens?
That section of memory is marked as allocatable, and its previous contents can be overwritten.
Consider this example...
[allocatedStatus][sideOfAllocation][allocatedMemory]
^-- Returned pointer
Considering this, the free() can then mark the [allocatedStatus] to false, so future allocations on the heap can use that memory.
How does it keep track of the free()d memory so that next time I declare
malloc() the address is the same?
I don't think it does. It just scanned for some free memory and found that previous block that had been marked as free.
Here is a rough idea how memory allocators work:
You have an allocator that has a bunch of "bins" ("free lists") which are just linked lists of free memory blocks. Each bin has a different block size associated with it (I.e.: you can have a list for 8 byte blocks, 16 byte blocks, 32 byte blocks, etc... Even arbitrary sizes like 7 or 10 byte blocks). When your program requests memory (usually through malloc()) the allocator goes to the smallest bin that would fit your data and checks to see if there are any free memory blocks in it. If not then it will request some memory from the OS (usually called a page) and cuts the block it gets back into a bunch of smaller blocks to fill the bin with. Then it returns one of these free blocks to your program.
When you call free, the allocator takes that memory address and puts it back into the bin (aka free list) it came from and everybody is happy. :)
The memory is still there to use so you don't have to keep paging memory, but with respect to your program it is free.
I believe it's entirely up to the operating system once you call free(), it may choose to immediately reclaim that memory or not care and just mark that memory segment as a possible acquisition for a later time (likely the same thing). To my knowledge that memory (if significant) shows up as available in the task manager right after free() on windows.
Keep in mind that the memory we are talking about here is virtual. So that means the operating system can tell you anything it wants and is likely not an accurate representation of the physical state of the machine.
Think about how you would manage memory allocation if you were writing an OS, you likely wouldn't want to do anything hasty that may waste resources. We are talking about 128 bytes here, would you want to waste valuable processing time handling it alone? It may be the reason for that behavior or not, at least plausible.
Do it in a loop and then free() in another loop or just allocate big chunks of memory, see what happens, experiment.
I am struggling to wrap my head around malloc in c - specifically when it needs to be free()'d. I am getting weird errors in gcc such as:
... free(): invalid next size (fast): ...
when I try to free a char pointer. For example, when reading from an input file, it will crash on certain lines when doing the following:
FILE *f = fopen(file,"r");
char x[256];
while(1) {
if(fgets(x,sizeof x,f)==NULL) break;
char *tmp = some_function_return_char_pointer(x); //OR malloc(nbytes);
// do some stuff
free(tmp); // this is where I get the error, but only sometimes
}
I checked for obvious things, such as x being NULL, but it's not; it just crashes on random lines.
But my REAL question is - when do I need to use free()? Or, probably more correctly, when should I NOT use free? What if malloc is in a function, and I return the var that used malloc()? What about in a for or while loop? Does malloc-ing for an array of struct have the same rules as for a string/char pointer?
I gather from the errors I'm getting in gcc on program crash that I'm just not understanding malloc and free. I've spent my quality time with Google and I'm still hitting brick walls. Are there any good resources you've found? Everything I see says that whenever I use malloc I need to use free. But then I try that and my program crashes. So maybe it's different based on a variable's scope? Does C free the memory at the end of a loop when a variable is declared inside of it? At the end of a function?
So:
for(i=0;i<100;i++) char *x=malloc(n); // no need to use free(x)?
but:
char *x;
for(i=0;i<100;i++) {
x=malloc(n);
free(x); //must do this, since scope of x greater than loop?
}
Is that right?
Hopefully I'm making sense...
malloc() is C's dynamic allocator. You have to understand the difference between automatic (scoped) and dynamic (manual) variables.
Automatic variables live for the duration of their scope. They're the ones you declare without any decoration: int x;
Most variables in a C program should be automatic, since they are local to some piece of code (e.g. a function, or a loop), and they communicate via function calls and return values.
The only time you need dynamic allocation is when you have some data that needs to outlive any given scope. Such data must be allocated dynamically, and eventually freed when it is no longer necessary.
The prime usage example for this is your typical linked list. The list nodes cannot possibly be local to any scope if you are going to have generic "insert/erase/find" list manipulation functions. Thus, each node must be allocated dynamically, and the list manipulation functions must ensure that they free those nodes that are no longer part of the list.
In summary, variable allocation is fundamentally and primarily a question of scope. If possible keep everything automatic and you don't have to do anything. If necessary, use dynamic allocation and take care to deallocate manually whenever appropriate.
(Edit: As #Oli says, you may also want to use dynamic allocation in a strictly local context at times, because most platforms limit the size of automatic variables to a much smaller limit than the size of dynamic memory. Think "huge array". Exceeding the available space for automatic variables usually has a colourful name such as "pile overrun" or something similar.)
In general, every call to malloc must have one corresponding call to free.* This has nothing to do with scope (i.e. nothing to do with functions or loops).
* Exceptions to this rule include using functions like strdup, but the principle is the same.
Broadly speaking, every pointer that is ever returned by malloc() must eventually be passed to free(). The scope of the variable that you store the pointer in does not affect this, because even after the variable is no longer in scope, the memory that the pointer points to will still be allocated until you call free() on it.
Well, the scope of the malloc'd memory lays between calls to malloc and free or otherwise until process is stopped (that is when OS cleans up for the process). If you never call free you get a memory leak. That could happen when address that you can pass to free goes out of scope before you actually used it - that is like loosing your keys for the car, car is still there but you can't really drive it. The error you are getting is most likely either because function returns a pointer to some memory that was not allocated using malloc or it returns a null pointer which you pass to free, which you cannot do.
You should free memory when you will no longer be accessing it. You should not free memory if you will be accessing it. This will give you a lot of pain.
If you don't want memory leak, you have to free the memory from malloc.
It can be very tricky. For example, if the // do some stuff has a continue, the free will be skipped and lead to memory leak. It is tricky, so we have shared_ptr in C++; and rumor has it salary of C programmer is higher than C++ programmer.
Sometimes we don't care memory leak. If the memory holds something that is needed during the whole lifetime of execution, you can choose not to free it. Example: a string for environment variable.
PS: Valgrind is a tool to help detect memory bugs. Especially useful for memory leak.
malloc(n) allocates n bytes of memory from a memory location named heap and then returns a void* type of pointer to it. The memory is allocated at runtime. Once you have allocated a memory dynamically, scope does not matter as long as you keep a pointer to it with you(or the address of it specifically). For example:
int* allocate_an_integer_array(int n)
{
int* p = (int*) (malloc(sizeof(int)*n));
return p;
}
This functions simply allocates memory from heap equal to n integers and returns a pointer to the first location. The pointer can be used in the calling function as you want to. The SCOPE does not matter as long as the pointer is with you..
free(p) returns the memory to heap.
The only thing you need to remember is to free it as if you don't free it and lose the value of its address, there will bw a memory leak. It is so because according to OS, you are still using the memory as you have not freed it and a memory leak will happen..
Also after freeing just set the value of the pointer to null so that u don't use it again as the same memory may be allocated again at any other time for a different purpose....
So, all you need to do is to be careful...
Hope it helps!
My question is quite simple. We generally allocate memory by declaring a pointer and then assigning a block of memory to that pointer. Suppose somewhere in the code I happen to use
ptr = ptr + 1
and then I use
free(ptr)
can someone tell what will happen. The entire memory block will get deallocated or something else. Can I partially deallocate the memory?
You must always pass exactly the same pointer to free that you got from malloc (or realloc.) If you don't, the "behavior is undefined", which is a term of art that means you can't rely on the program behaving in any predictable way. In this case, though, you should expect it to crash immediately. (If you get unlucky, it will instead corrupt memory, causing a crash some time later, or worse, incorrect output.)
The only way to partially deallocate memory is realloc with a smaller size, but that's only good for trimming at the end and isn't guaranteed to make the trimmed-off chunk available for some other allocation.
It's impossible to deallocate part of a memory block. The only thing you can do is reallocate the block giving it a different size. However this does not guarantee that the block will land in the same place in memory (it might be copied somewhere else).
You must pass to free() the same pointer to the same location malloc() returned. That's because the allocator keeps a sort of list of the blocks you allocated (#everyone: feel free to add/modify if I'm wrong) and if the pointer you pass to free doesn't compare it this list free() complains ("bad memory block" maybe?). To partly deallocate the memory you should use realloc(), which changes the dimensions of that block of memory, but is slow and inefficient. You should use it only when you're sure of the new size of the block or leaving more space to fill in the future.
malloc() , free() and realloc() are not part of C language.
These are functions defines in standard library. Code in standard library that deals with it is usually called "allocator". So, actual answer is "It depends on C library".
On Linux, glibc would crash the program.
On VC++, C runtime would corrupt allocator's state
Source code is available for these libraries, so you can actually put a breakpoint and step into free.