I wish to put pointers allocated via GC_MALLOC in memory allocated with the standard malloc(). The manual says to not do this, but this memory is allocated in a library I do not have control over. Is it possible to designate the memory allocated by malloc() as uncollectable in the same way that GC_MALLOC_UNCOLLECTABLE() does for memory that it allocates? That is, so that it will know to scan that memory for pointers, but not attempt to free it.
The only alternative I can think of is to allocate some memory via GC_MALLOC_UNCOLLECTABLE() in addition to the malloc-allocated memory, just to place the pointer in both places, but I am hoping for a less awkward solution.
Related
I want to initialize a memory region and thus can use malloc (in glibc) to allocate memory only from this region. There will be multiple malloc used and every malloc operation should only use this pre-defined region. This region requires contiguous memory addresses with a specific size.
My thought is to create a region_init function that uses sbrk or shared memory object (using shm_open?) to initialize a memory region that can be used by (multiple) malloc. BTW, I think mmap() doesn't work for this scenario because mmap() doesn't always return contiguous addresses. The problem is that I don't know how to make every malloc use this memory region I created (using sbrk? shared memory? and/or others?). Can I achieve this by modifying the source codes of malloc in glibc? If I can, how to modify glibc (any idea is appreciated)? What should I do for implementing this functionality?
Before this question, I asked a related question on StackOverflow. In the previous question, some answer/comment suggested that I can use a single malloc to allocate a large memory buffer/region and implement my own API to manage this memory (e.g., m_malloc to allocate part of the memory from this memory buffer/region). However, the problem of this design is that it is hard to free especially when there are multiple variables in this region. The free in glibc can only free the whole region that is malloc'd. If I want to free a variable (when there are many variables in this malloc'd region), I can only free the whole region which means all of the variables are freed. I really want to use free to free the memory of any variable I allocated in the memory region created by only one malloc.
I hope my question is clear. Thanks for your help and ideas.
The VirtualAlloc() will allocate a virtual memory page that have some attributes (by the parameter "fdwProtect").
What about the memory that allocated by malloc() ?Is that have same attributes?
Is the memory by malloc() have attributes that "commintting" or "reserving" ?
Further more, what about other C/C++ lib function?
VirtualAlloc
This function allows you to specify additional options for memory allocation. But it allocates memory in large page with a minimum indicated by GetLargePageMinimum, you can commit, reserve with it. It's not for general use. Memory allocated by this function is automatically initialized to zero.
malloc
The standard C version to allocate memory. Prefer it if you are writing in C rather than C++, and your code needs to work on on other platforms, or someone specifically says that you need to use it. It's quite possible that, on Windows, malloc would be implemented on top of HeapAlloc. malloc can allocate any chunk of memory, it doesn't have any concept to commit and reserve by current standard. Memory allocated by this function is not initialized.
I have been taught in lectures, that calling free() on a pointer twice is really, really bad. I know that it is good practice, to set a pointer to NULL, right after having freed it.
However, I still have never heard any explanation as to why that is. From what I understand, the way malloc() works, it should technically keep track of the pointers it has allocated and given you to use. So why does it not know, whether a pointer it receives through free() has been freed yet or not?
I would love to understand, what happens internally, when you call free() on a location that has previously already been freed.
When you use malloc you are telling the PC that you want to reserve some memory location on the heap just for you. The computer gives back a pointer to the first byte of the addressed space.
When you use free you are actually telling the computer that you don't need that space anymore, so it marks that space as available for other data.
The pointer still points to that memory address. At this point that same space in the heap can be returned by another malloc call. When you invoke free a second time, you are not freeing the previous data, but the new data, and this may not be good for your program ;)
To answer your first question,
So why does it not know, whether a pointer it receives through free() has been freed yet or not?
because, the specification for malloc() in C standard does not mandate this. When you call malloc() or family of functions, what it does is to return you a pointer and internally it stores the size of the memory location allocated in that pointer. That is the reason free() does not need a size to clean up the memory.
Also, once free()-d, what happens with the actually allocated memory is still implelentation dependent. Calling free() is just a marker to point out that the allocated memory is no longer in use by the process and can be reclaimed and e re-allocated, if needed. So, keeping track of the allocated pointer is very needless at that point. It will be an unnecessary burden on the OS to keep all the backtracks.
For debugging purpose, however, some library implementations can do this job for you, like DUMA or dmalloc and last but not the least, memcheck tool from Valgrind.
Now, technically, the C standard does not specify any behaviour if you call free() on an already free-ed pointer. It is undefined behavior.
C11, chapter ยง7.22.3.3, free() function
[...] if
the argument does not match a pointer earlier returned by a memory management
function, or if the space has been deallocated by a call to free() or realloc(), the
behavior is undefined.
C standard only says that calling free twice on a pointer returned by malloc and its family function invoke undefined behavior. There is no further explanation why it is so.
But, why it is bad is explained here:
Freeing The Same Chunk Twice
To understand what this kind of error might cause, we should remember how the memory manager normally works. Often, it stores the size of the allocated chunk right before the chunk itself in memory. If we freed the memory, this memory chunk might have been allocated again by another malloc() request, and thus this double-free will actually free the wrong memory chunk - causing us to have a dangling pointer somewhere else in our application. Such bugs tend to show themselves much later than the place in the code where they occured. Sometimes we don't see them at all, but they still lurk around, waiting for an opportunity to rear their ugly heads.
Another problem that might occure, is that this double-free will be done after the freed chunk was merged together with neighbouring free chunks to form a larger free chunk, and then the larger chunk was re-allocated. In such a case, when we try to free() our chunk for the 2nd time, we'll actually free only part of the memory chunk that the application is currently using. This will cause even more unexpected problems.
When you are calling malloc you are getting a pointer. The runtime library needs to keep track of the malloced memory. Typically malloc does not store the memory management structures separated from the malloc ed memory but in one place. So a malloc for x bytes in fact takes x+n bytes, where one possible layout is that the first n bytes are containing a linked list struct with pointers to the next (and maybe previous) allocated memory block.
When you free a pointer then the function free could walk through it's internal memory management structures and check if the pointer you pass in is a valid pointer that was malloced. Only then it could access the hidden parts of the memory block. But doing this check would be very time consuming, especially if you allocate a lot. So free simply assumes that you pass in a valid pointer. That means it directly access the hidden parts of the memory block and assumes that the linked list pointers there are valid.
If you free a block twice then you might have the problem that someone did a new malloc, got the memory you just freed, overwrites it and the second free reads invalid pointers from it.
Setting a freed pointer to NULL is good practice because it helps debugging. If you access freed memory your program might crash, but it might also just read suspicious values and maybe crash later. Finding the root cause then might be hard. If you set freed pointers to NULL your program will immediately crash when you try to access the memory. That helps massively during debugging.
I have been recently trying to learn how to program in the C programming language.
I am currently having trouble understanding how memory is deallocated by free() in C.
What does it mean to free or release the memory?
For instance, if I have the following pointer:
int *p = malloc(sizeof(int));
When I deallocate it using free(p), what does it do? Does it somehow flag it as "deallocated", so the application may use it for new allocations?
Does it deallocates only the pointer address, or the address being pointed is also deallocated too?
I would do some experiments myself to better understand this, but I am so newbie in the subject that I don't know even how to debug a C program yet (I'm not using any IDE).
Also, what if int *p is actually a pointer to an array of int?
If I call free(p), does it deallocate the whole array or only the element it is pointing to?
I'm so eager to finally understand this, I would very much appreciate any help!
What does it mean to free or release the memory?
It means that you're done with the memory and are ready to give it back to the memory allocator.
When I deallocate it using free(p), what does it do?
The specifics are implementation dependent, but for a typical allocator it puts the block back on the free list. The allocator maintains a list of blocks that are available for use. When you ask for a chunk of memory (by calling malloc() or similar) the allocator finds an appropriate block in the list of free blocks, removes it (so it's no longer available), and gives you a pointer to the block. When you call free(), the process is reversed -- the block is put back on the free list and thereby becomes available to be allocated again.
Importantly, once you call free() on a pointer, you must not dereference that pointer again. A common source of memory-related errors is using a pointer after it has been freed. For that reason, some consider it a helpful practice to set a pointer to nil immediately after freeing it. Similarly, you should avoid calling free() on a pointer that you didn't originally get from the allocator (e.g. don't free a pointer to a local variable), and it's never a good idea to call free() twice on the same pointer.
Does it deallocates only the pointer address, or the address being pointed is also deallocated too?
When you request a block of memory from the allocator, you specify the size of the block you want. The allocator keeps track of the size of the block so that when you free the block, it knows both the starting address and the block size. When you call free(p), the block that p points to is deallocated; nothing happens to the pointer p itself.
Also, what if int *p is actually a pointer to an array of int?
An array in C is a contiguous block of memory, so a pointer to the first element of the array is also a pointer to the entire block. Freeing that block will properly deallocate the entire array.
I'm so eager to finally understand this, I would very much appreciate any help!
There are a number of good pages about memory allocation in C that you should read for a much more detailed understanding. One place you could start is with the GNU C Library manual section on memory allocation.
As alluded to above and in the other answers, the actual behavior of the allocator depends on the implementation. Your code shouldn't have any particular expectations about how memory allocation works beyond what's documented in the standard library, i.e. call malloc(), calloc(), etc. to get a block of memory, and call free() to give it back when you're done so that it can be reused.
malloc and free do whatever they want. Their expected behaviour is that malloc allocates a block of desired size in dynamic memory and returns a pointer to it. free must be able to receive one such pointer and correctly deallocate the block. How they keep track of the block size is irrelevant.
Is int *p a pointer to an array of ints ? Maybe. If you allocated sufficient space for several ints, yes.
There is a fixed and limited amount of memory in your computer, and everybody wants some. The Operating system is charged with the task of assigning ownership to pieces of memory and keeping track of it all to assure that no one messes with anyone else's.
When you ask for memory with malloc(), you're asking the system (the C runtime and the OS) to give you the address of a block of memory that is now yours. You are free to write to it and read from it at will, and the system promises that no one else will mess with it while you own it. When you de-allocate it with free(), nothing happens to the memory itself, it's just no longer yours. What happens to it is none of your business. The system may keep it around for future allocations, it may give it to some other process.
The details of how this happens vary from one system to another, but they really don't concern the programmer (unless you're the one writing the code for malloc/free). Just use the memory while it's yours, and keep your hands off while it's not.
I am trying to undestand the C functions malloc and free. I know this has been discussed a lot on StackOverflow. However, I think I kind of know what these functions do by now. I want to know why to use them. Let's take a look at this piece of code:
int n = 10;
char* array;
array = (char*) malloc(n * sizeof(char));
// Check whether memory could be allocated or not...
// Do whatever with array...
free(array);
array = NULL;
I created a pointer of type char which I called array. Then I used malloc to find a chunk of memory that is currently not used and (10 * sizeof(char)) bytes large. That address I casted to type char pointer before assigning it to my previously created char pointer. Now I can work with my char array. When I am done, I'll use free to free that chunk of memory since it's not being used anymore.
I have one question: Why wouldn't I just do char array[10];? Wikipedia has only one small sentence to give to answer that, and that sentence I unfortunately don't understand:
However, the size of the array is fixed at compile time. If one wishes to allocate a similar array dynamically...
The slide from my university is similarily concise:
It is also possible to allocate memory from the heap.
What is the heap? I know a data structure called heap. :)
However, I've someone could explain to me in which case it makes sense to use malloc and free instead of the regular declaration of a variable, that'd be great. :)
C provides three different possible "storage durations" for objects:
Automatic - local storage that's specific to the invocation of the function it's in. There may be more than one instance of objects created with automatic storage, if a function is called recursively or from multiple threads. Or there may be no instances (if/when the function isn't being called).
Static - storage that exists, in exactly one instance, for the entire duration of the running program.
Allocated (dynamic) - created by malloc, and persists until free is called to free it or the program terminates. Allocated storage is the only type of storage with which you can create arbitrarily large or arbitrarily many objects which you can keep even when functions return. This is what malloc is useful for.
First of all there is no need to cast the malloc
array = malloc(n * sizeof(char));
I have one question: Why wouldn't I just do char array[10];?
What will you do if you don't know how many storage space do you want (Say, if you wanted to have an array of arbitrary size like a stack or linked list for example)?
In this case you have to rely on malloc (in C99 you can use Variable Length Arrays but for small memory size).
The function malloc is used to allocate a certain amount of memory during the execution of a program. The malloc function will request a block of memory from the heap. If the request is granted, the operating system will reserve the requested amount of memory.
When the amount of memory is not needed anymore, you must return it to the operating system by calling the function free.
In simple: you use an array when you know the number of elements the array will need to hold at compile time. you use malloc with pointers when you don't know how many elements the array will need to be at compile time.
For more detail read Heap Management With malloc() and free().
Imagine you want to allocate 1,000 arrays.
If you did not have malloc and free... but needed a declaration in your source for each array, then you'd have to make 1,000 declarations. You'd have to give them all names. (array1, array2, ... array1000).
The idea in general of dynamic memory management is to handle items when the quantity of items is not something you can know in advance at the time you are writing your program.
Regarding your question: Why wouldn't I just do char array[10];?. You can, and most of the time, that will be completely sufficient. However, what if you wanted to do something similar, but much much bigger? Or what if the size of your data needs to change during execution? These are a few of the situations that point to using dynamically allocated memory (calloc() or malloc()).
Understanding a little about how/when the stack and heap are used would be good: When you use malloc() or calloc(), it uses memory from the heap, where automatic/static variables are given memory on the stack, and are freed when you leave the scope of that variable, i.e the function or block it was declared in.
Using malloc and calloc become very useful when the size of the data you need is not known until run-time. When the size is determined, you can easily call one of these to allocate memory onto the heap, then when you are finished, free it with free()
Regarding What is the heap? There is a good discussion on that topic here (slightly different topic, but good discussion)
In response to However, I've someone could explain to me in which case it makes sense to use malloc() and free()...?
In short, If you know what your memory requirements are at build time (before run-time) for a particular variable(s), use static / automatic creation of variables (and corresponding memory usage). If you do not know what size is necessary until run-time, use malloc() or calloc() with a corresponding call to free() (for each use) to create memory. This is of course a rule-of-thumb, and a gross generalization. As you gain experience using memory, you will find scenarios where even when size information is known before run-time, you will choose to dynamically allocate due to some other criteria. (size comes to mind)
If you know in advance that you only require an array of 10 chars, you should just say char array[10]. malloc is useful if you don't know in advance how much storage you need. It is also useful if you need storage that is valid after the current function returns. If you declare array as char array[10], it will be allocated on the stack. This data will not be valid after your function returns. Storage that you obtain from malloc is valid until you call free on it.
Also, there is no need to cast the return value of malloc.
Why to use free after malloc can be understood in the way that it is a good style to free memory as soon as you don't need it. However if you dont free the memory then it would not harm much but only your run time cost will increase.
You may also choose to leave memory unfreed when you exit the program. malloc() uses the heap and the complete heap of a process is freed when the process exits. The only reason why people insist on freeing the memory is to avoid memory leaks.
From here:
Allocation Myth 4: Non-garbage-collected programs should always
deallocate all memory they allocate.
The Truth: Omitted deallocations in frequently executed code cause
growing leaks. They are rarely acceptable. but Programs that retain
most allocated memory until program exit often perform better without
any intervening deallocation. Malloc is much easier to implement if
there is no free.
In most cases, deallocating memory just before program exit is
pointless. The OS will reclaim it anyway. Free will touch and page in
the dead objects; the OS won't.
Consequence: Be careful with "leak detectors" that count allocations.
Some "leaks" are good!
Also the wiki has a good point in Heap base memory allocation:-
The heap method suffers from a few inherent flaws, stemming entirely
from fragmentation. Like any method of memory allocation, the heap
will become fragmented; that is, there will be sections of used and
unused memory in the allocated space on the heap. A good allocator
will attempt to find an unused area of already allocated memory to use
before resorting to expanding the heap. The major problem with this
method is that the heap has only two significant attributes: base, or
the beginning of the heap in virtual memory space; and length, or its
size. The heap requires enough system memory to fill its entire
length, and its base can never change. Thus, any large areas of unused
memory are wasted. The heap can get "stuck" in this position if a
small used segment exists at the end of the heap, which could waste
any magnitude of address space, from a few megabytes to a few hundred.