This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
C programming : How does free know how much to free?
Hello All,
How OS will come to know how much size i have to free when we define free(pointer).
I mean we are not providing any size , only pointer to free statement.
How's internally handle the size ?
Thanks,
Neel
The OS won't have a clue, as free is not a system call. However, your C libraries memory allocation system will have recorded the size in some way when the memory was originally allocated by malloc(), so it knows how much to free.
The size is stored internally in the allocator, and the pointer you pass to free is used to reach that data. A very basic approach is to store the size 4 bytes before the pointer, so substracting 4 from the pointer gives you a pointer to it's size.
Notice that the OS doesn't handle this directly, it's implemented by your C/C++ runtime allocator.
When you call malloc, the C library will automatically carve a space out for you on the heap. Because things created on the heap are created dynamically, what is on the heap at any given point in time is not known as it is for the stack. So the library will keep track of all the memory that you have allocated on the heap.
At some point your heap might look like this:
p---+
V
---------------------------------------
... | used (4) | used (10) | used (8) | ...
---------------------------------------
The library will keep track of how much memory is allocated for each block. In this case, the pointer p points to the start of the middle block.
If we do the following call:
free(p);
then the library will free this space for you on the heap, like so...
p---+
V
----------------------------------------
... | used (4) | unused(10) | used (8) | ...
----------------------------------------
Now, the next time that you are looking for some space, say with a call like:
void* ptr = malloc(10);
The newly unused space may be allocated to your program again, which will allow us to reduce the overall amount of memory our program uses.
ptr---+
V
----------------------------------------
... | used (4) | used(10) | used (8) | ...
----------------------------------------
The way your library might handle internally managing the sizes is different. A simple way to implement this, would be to just add an additional amount of bytes (we'll say 1 for the example) at the beginning of each block allocated to hold the size of each block. So our previous block of heap memory would look like this:
bytes: 1 4 1 10 1 8
--------------------------------
... |4| used |10| used |8| used | ...
--------------------------------
^
+---ptr
Now, if we say that block sizes will be rounded up to be divisible by 2, they we have an extra bit at the end of the size (because we can always assume it to be 0, which we can conveniently use to check whether the corresponding block is used or unused.
When we pass a pointer in free:
free(ptr);
The library would move the pointer given back one byte, and change the used/unused bit to unused. In this specific case, we don't even have to actually know the size of the block in order to free it. It only becomes an issue when we try to reallocate the same amount of data. Then, the malloc call would go down the line, checking to see if the next block was free. If it is free, then if it is the right size that block will be returned back to the user, otherwise a new block will be cut at the end of the heap, and more space allocated from the OS if necessary.
Related
So I know that a call free() on a variable allocated in the stack would cause an invalid pointer error.
In a malloced pointer, malloc() allocates 8 bytes before the actual pointer to leave information about its size. So I was wondering if I had made a long before a struct and then called free on that struct if it would be possible to free the struct (of course this is going off the assumption that the allocation of those 8 bytes is the only thing extra that malloc does).
I guess my final question would be if there is any real difference between stack variable allocation and heap allocation (in terms of the backend calls to the kernel).
Some C implementations might use data before the allocated space to help them manage the space. Some do not. Some do that for allocations of certain sizes and not others. If they do it, it might be eight bytes, or it might be some other amount. You should not rely on any behavior in this regard.
When you declare a long object and a struct of some sort in a block, the compiler might or might not put them next to each other on the stack. It might put the long before the struct or vice-versa, or, because it optimizes your program, it might keep the long in a register and never put it on the stack at all, and it might do other things. In some C implementations, a long is eight bytes. In some, it is not. There is no good way for you to ensure two separate objects are put in adjacent memory. (You can make them not separate by putting them in a larger struct.)
Even if you are able to cobble together a long followed by a struct, how would you know what value to put into the long? Did the C implementation put the length of the allocation in there? Or is it a pointer to another block? Or to some other part of the database the C implementation uses to track allocated memory? If malloc and free are using memory just before the allocated space, that memory is not empty. It needs to have some value in it, and you do not know what that is.
If you get lucky, passing the address of the struct to free might not crash your program right away. But then you have freed a part of the stack, in some sense. When you call malloc again, the pointer you get back might be for that memory, and then your program presumably will write to that space. Then what happens when your program calls other routines, causing the stack to grow into that space? You will have overlapping uses of the same memory. Some of your data will be stomping over other data, and your program will not work.
Yes, there are differences between memory allocated on the stack and memory allocated from the heap. This is outside of the model that C presents to your program. However, in systems where processes have stack and heap, they are generally in different places in the memory of your process. In particular, the stack memory must remain available for use as the stack grows and shrinks. You cannot mix it with the heap without breaking things.
It is good to ask questions about what happens when you try various things. However, modern implementations of malloc and free are quite complicated, and you pretty much have to accept them as a service that you cannot peer into easily. Instead, to help you learn, you might think about this:
How would you write your own malloc and free?
Write some code that allocates a large amount of memory using malloc, say a megabyte, and write two routines called MyMalloc and MyFree that work like malloc and free, except they use the memory you allocated. When MyMalloc is called, it will carve out a chunk of the memory. When MyFree is called, it will return the chunk to make it available again.
Write some experimental code that somewhat randomly calls MyMalloc with various sizes and MyFree, in somewhat random orders.
How can you make all of this work? How do you divide the megabyte into chunks? How do you remember which chunks are allocated and which are free? When somebody calls MyFree, how do you know how much they are giving back? When neighboring chunks are returned with MyFree, how do you put them back together into bigger pieces again?
I think your real question is how does the stack work.
The stack is one big memory block allocated when your program starts. There is a pointer to the top of the stack. The name is suggestive: think of a stack of magazines.
When a function is called, the parameters are placed on top of the stack. The function itself then places its local variables on top of that. When the function exits, the stack pointer is simply moved back to where it was before the function was called. This frees all the local variables and input arguments used by the function.
The heap manager has nothing to do with this block of memory. Tricking free to put some of the stack in the heap manager’s memory is going to wreak havoc on your program. The memory would likely be used again as you call other functions, and used simultaneously if you malloc memory, leading to data corruption at best, and stack corruption (read crash) at worst.
When you speak of memory being allocated on the stack, you have to understand that in most implementations, the stack is allocted in a block -- variables are not allocated individually or separately.
+-+ +--------------------------------------------------+
| | Stack frame data section; local variables and |
| | |
| | function arguments in order determined by the |
| | |
| | calling convention of the target platform |
Stack frame for | | |
function call; +---+ | (size is implementation dependent) |
block allocated | | |
| | |
| +--------------------------------------------------+
| |Instruction pointer (return address) |
| +--------------------------------------------------+
| |Space for return value (if not in a CPU register) |
+-+ +--------------------------------------------------+
| |
| |
| |
| (stack frame of previously called function) |
| |
| |
+--------------------------------------------------+
Each function call is allocated its own stack frame, with the required size to hold the return value (if necessary), the instruction pointer of the return address, and all of the local variables and function arguments. So while memory for the stack frame is allocated, it's not allocated with respect to any individual variable -- only in regard to the sum of the individual sizes.
In C programming, you can pass any kind of pointer you like as an argument to free, how does it know the size of the allocated memory to free? Whenever I pass a pointer to some function, I have to also pass the size (ie an array of 10 elements needs to receive 10 as a parameter to know the size of the array), but I do not have to pass the size to the free function. Why not, and can I use this same technique in my own functions to save me from needing to cart around the extra variable of the array's length?
When you call malloc(), you specify the amount of memory to allocate. The amount of memory actually used is slightly more than this, and includes extra information that records (at least) how big the block is. You can't (reliably) access that other information - and nor should you :-).
When you call free(), it simply looks at the extra information to find out how big the block is.
Most implementations of C memory allocation functions will store accounting information for each block, either in-line or separately.
One typical way (in-line) is to actually allocate both a header and the memory you asked for, padded out to some minimum size. So for example, if you asked for 20 bytes, the system may allocate a 48-byte block:
16-byte header containing size, special marker, checksum, pointers to next/previous block and so on.
32 bytes data area (your 20 bytes padded out to a multiple of 16).
The address then given to you is the address of the data area. Then, when you free the block, free will simply take the address you give it and, assuming you haven't stuffed up that address or the memory around it, check the accounting information immediately before it. Graphically, that would be along the lines of:
____ The allocated block ____
/ \
+--------+--------------------+
| Header | Your data area ... |
+--------+--------------------+
^
|
+-- The address you are given
Keep in mind the size of the header and the padding are totally implementation defined (actually, the entire thing is implementation-defined (a) but the in-line accounting option is a common one).
The checksums and special markers that exist in the accounting information are often the cause of errors like "Memory arena corrupted" or "Double free" if you overwrite them or free them twice.
The padding (to make allocation more efficient) is why you can sometimes write a little bit beyond the end of your requested space without causing problems (still, don't do that, it's undefined behaviour and, just because it works sometimes, doesn't mean it's okay to do it).
(a) I've written implementations of malloc in embedded systems where you got 128 bytes no matter what you asked for (that was the size of the largest structure in the system), assuming you asked for 128 bytes or less (requests for more would be met with a NULL return value). A very simple bit-mask (i.e., not in-line) was used to decide whether a 128-byte chunk was allocated or not.
Others I've developed had different pools for 16-byte chunks, 64-bytes chunks, 256-byte chunks and 1K chunks, again using a bit-mask to decide what blocks were used or available.
Both these options managed to reduce the overhead of the accounting information and to increase the speed of malloc and free (no need to coalesce adjacent blocks when freeing), particularly important in the environment we were working in.
From the comp.lang.c FAQ list: How does free know how many bytes to free?
The malloc/free implementation remembers the size of each block as it is allocated, so it is not necessary to remind it of the size when freeing. (Typically, the size is stored adjacent to the allocated block, which is why things usually break badly if the bounds of the allocated block are even slightly overstepped)
This answer is relocated from How does free() know how much memory to deallocate? where I was abrubtly prevented from answering by an apparent duplicate question. This answer then should be relevant to this duplicate:
For the case of malloc, the heap allocator stores a mapping of the original returned pointer, to relevant details needed for freeing the memory later. This typically involves storing the size of the memory region in whatever form relevant to the allocator in use, for example raw size, or a node in a binary tree used to track allocations, or a count of memory "units" in use.
free will not fail if you "rename" the pointer, or duplicate it in any way. It is not however reference counted, and only the first free will be correct. Additional frees are "double free" errors.
Attempting to free any pointer with a value different to those returned by previous mallocs, and as yet unfreed is an error. It is not possible to partially free memory regions returned from malloc.
On a related note GLib library has memory allocation functions which do not save implicit size - and then you just pass the size parameter to free. This can eliminate part of the overhead.
The heap manager stored the amount of memory belonging to the allocated block somewhere when you called malloc.
I never implemented one myself, but I guess the memory right in front of the allocated block might contain the meta information.
The original technique was to allocate a slightly larger block and store the size at the beginning, then give the application the rest of the blog. The extra space holds a size and possibly links to thread the free blocks together for reuse.
There are certain issues with those tricks, however, such as poor cache and memory management behavior. Using memory right in the block tends to page things in unnecessarily and it also creates dirty pages which complicate sharing and copy-on-write.
So a more advanced technique is to keep a separate directory. Exotic approaches have also been developed where areas of memory use the same power-of-two sizes.
In general, the answer is: a separate data structure is allocated to keep state.
malloc() and free() are system/compiler dependent so it's hard to give a specific answer.
More information on this other question.
To answer the second half of your question: yes, you can, and a fairly common pattern in C is the following:
typedef struct {
size_t numElements
int elements[1]; /* but enough space malloced for numElements at runtime */
} IntArray_t;
#define SIZE 10
IntArray_t* myArray = malloc(sizeof(intArray_t) + SIZE * sizeof(int));
myArray->numElements = SIZE;
to answer the second question, yes you could (kind of) use the same technique as malloc()
by simply assigning the first cell inside every array to the size of the array.
that lets you send the array without sending an additional size argument.
When we call malloc it's simply consume more byte from it's requirement. This more byte consumption contain information like check sum,size and other additional information.
When we call free at that time it directly go to that additional information where it's find the address and also find how much block will be free.
I've been trying to learn the basics of a heap overflow attack. I'm mostly interested in using a corruption or modification of the chunk metadata for the basis of the attack, but I'm also open to other suggestions. I know that my goal of the exploit should be do overwrite the printf() function pointer with that of the challenge() function pointer, but I can't seem to figure out how to achieve that write.
I have the following piece of code which I want to exploit, which is using malloc from glibc 2.11.2 :
void challenge()
{
puts("you win\n");
}
int main(int argc, char **argv)
{
char *inputA, *inputB, *inputC;
inputA = malloc(32);
inputB = malloc(32);
inputC = malloc(32);
strcpy(inputA, argv[1]);
strcpy(inputB, argv[2]);
strcpy(inputC, argv[3]);
free(inputC);
free(inputB);
free(inputA);
printf("execute challenge to win\n");
}
Obviously, achieving an actual overwrite of an allocated chunk's metadata is trivial. However, I have not been able to find a way to exploit this code using any of the standard techniques.
I have read and attempted to implement the techniques from:
The paper: w00w00 on Heap Overflows
Although the paper is very clear, the unlink technique has been obsolete for some time.
Malloc Maleficarum.txt
This paper expands upon the exploit techniques from the w00w00 days, and accounts for the newer versions of glibc. However, I have not found that given the 5 techniques detailed in the paper, that the code above matches any of the prerequisites for those techniques.
Understanding the Heap By Breaking it(pdf)
The pdf gives a pretty good review of how the heap works, but focuses on double free techniques.
I originally tried to exploit this code by manipulating the size value of the chunk for inputC, so that it pointed back to the head of the inputC chunk. When that didn't work, I tried pointing further back to the chunk of inputB. That is when I realized that the new glibc performs a sanity check on the size value.
How can a user craft an exploit to take advantage of a free, assuming he has the ability to edit the allocated chunk's metadata to arbitrary values, and user it to overwrite a value in the GOT or write to any other arbitrary address?
Note: When I write 'arbitrary address' I understand that memory pages may be read only or protected, I mean an address that I can assume I can write to.
Note: I will say before I answer that this is purely an academic answer, not intended to be used for malicious purposes. I am aware of the exercises OP is doing and they are open source and not intended to encourage any users to use these techniques in unapproved circumstances.
I will detail the technique below but for your reference I would take a look at the Vudo malloc tricks (It's referenced in one of your links above) because my overview is going to be a short one: http://www.phrack.com/issues.html?issue=57&id=8
It details how malloc handles creating blocks of memory, pulling memory from lists and other things. In particular the unlink attack is of interest for this attack (note: you're correct that glibc now performs a sanity check on sizes for this particular reason, but you should be on an older libc for this exercise... legacy bro).
From the paper, an allocated block and a free block use the same data structure, but the data is handled differently. See here:
chunk -> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| prev_size: size of the previous chunk, in bytes (used |
| by dlmalloc only if this previous chunk is free) |
+---------------------------------------------------------+
| size: size of the chunk (the number of bytes between |
| "chunk" and "nextchunk") and 2 bits status information |
mem -> +---------------------------------------------------------+
| fd: not used by dlmalloc because "chunk" is allocated |
| (user data therefore starts here) |
+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
| bk: not used by dlmalloc because "chunk" is allocated |
| (there may be user data here) |
+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
| |
| |
| user data (may be 0 bytes long) |
| |
| |
next -> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| prev_size: not used by dlmalloc because "chunk" is |
| allocated (may hold user data, to decrease wastage) |
+---------------------------------------------------------+
Allocated blocks don't use the fd or bk pointers, but free ones will. This is going to be important later. You should know enough programming to understand that "blocks" in Doug Lea's malloc are organized into a doubly-linked list; there's one list for free blocks and another for allocated ones (technically there are several lists for free depending on sizes but it's irrelevant here since the code allocates blocks of the same size). So when you're freeing a particular block, you have to fix the pointers to keep the list in tact.
e.g. say you're freeing block y from the list below:
x <-> y <-> z
Notice that in the diagram above the spots for bk and fd contain the necessary pointers to iterate along the list. When malloc wants to take a block p off of the list it calls, among other things, a macro to fix the list:
#define unlink( y, BK, FD ) {
BK = P->bk;
FD = P->fd;
FD->bk = BK;
BK->fd = FD;
}
The macro itself isn't hard to understand, but the important thing to note in older versions of libc is that it doesn't perform sanity checks on the size or the pointers being written to. What it means in your case is that without any sort of address randomization you can predictably and reliably determine the status of the heap and redirect an arbitrary pointer to an address of your choosing by overflowing the heap (via the strncopy here) in a specific way.
There's a few things required to get the attack to work:
the fd pointer for your block is pointing to the address you want to overwrite minus 12 bytes. The offset has to do with malloc cleaning up the alignment when it modifies the list
The bk pointer of your block is pointing to your shellcode
The size needs to be -4. This accomplishes a few things, namely it sets the status bits in the block
So you'll have to play around with the offsets in your specific example, but the general malicious format that you're trying to pass with the strcpy here is of the format:
| junk to fill up the legitimate buffer | -4 | -4 | addr you want to overwrite -12 (0x0C) | addr you want to call instead
Note the negative number sets the prev_size field to -4, which makes the free routing believe that the prev_size chunk actually starts in the current chunk that you control/are corrupting.
And yes, a proper explanation wouldn't be complete without mentioning that this attack doesn't work on current versions of glibc; the size has a sanity check done and the unlink method just won't work. That in combination with mitigations like address randomization make this attack not viable on anything but legacy systems. But the method described here is how I did that challenge ;)
Note that most of the techniques explained in Malloc Malleficarum are now protected. The glibc has improved a lot all that double free scenarios.
If you want to improve your knowledge about the Malloc Malleficarum techniques read the Malloc Des-Malleficarum and the House of Lore: Reloaded written by blackngel. You can find these texts in phrack.
Malloc Des-Malleficarum
I'm also working on it, and I can say to you that, for example, House of Mind is no longer exploitable, at least, as is explained in the texts. Although it might be possible to bypass the new restrictions added to the code.
Add that the easiest way to execute your code is to overwrite the .dtors address therefore your code will always be executed once the program finish.
If you download the glibc code and study the critic zones of malloc., etc you will find code checks that are not documented in the documents previously mentioned. These check were included to stop the double free party.
On the other hand, the presentation of Justin N. Ferguson (Understanding the Heap by breaking it) that you could find in youtube (BlackHat 2007) is perfect in order to understand all the heap mechanics, but I must admit that the techniques shown are far from being reliable, but at least, he opens a new field to heap exploitation.
Understanding the heap by breaking it
Anyways, I'm also working on it, so if you want to contact me, we can share our advances. You can reach me in the overflowedminds.net domain as newlog (build the mail address yourself ^^ ).
Heap overflows are tricky to pull off, and are very heavilly heap-layout dependent, although it looks like you're going after the Windows CRT heap, which has lots of mitigations in place specifically to stop this type of attack.
If you really do want to do this kind of thing, you need to happy jumping into WinDbg and stepping into functions like free to see exactly what is going on inside free, and hence what kind of control you might be able to achieve via the heap overflow of the previous value.
I won't give you any more specific help than that for the simple reason that demonstrating a heap overflow is usually enough for defensive security - defensive security experts can report a heap overflow without needing to actually fully exploit it. The only people who do need to fully exploit a heap-overflow all the way to remote code execution are people exploiting bugs offensively, and if you want to do that, you're on your own.
In C programming, you can pass any kind of pointer you like as an argument to free, how does it know the size of the allocated memory to free? Whenever I pass a pointer to some function, I have to also pass the size (ie an array of 10 elements needs to receive 10 as a parameter to know the size of the array), but I do not have to pass the size to the free function. Why not, and can I use this same technique in my own functions to save me from needing to cart around the extra variable of the array's length?
When you call malloc(), you specify the amount of memory to allocate. The amount of memory actually used is slightly more than this, and includes extra information that records (at least) how big the block is. You can't (reliably) access that other information - and nor should you :-).
When you call free(), it simply looks at the extra information to find out how big the block is.
Most implementations of C memory allocation functions will store accounting information for each block, either in-line or separately.
One typical way (in-line) is to actually allocate both a header and the memory you asked for, padded out to some minimum size. So for example, if you asked for 20 bytes, the system may allocate a 48-byte block:
16-byte header containing size, special marker, checksum, pointers to next/previous block and so on.
32 bytes data area (your 20 bytes padded out to a multiple of 16).
The address then given to you is the address of the data area. Then, when you free the block, free will simply take the address you give it and, assuming you haven't stuffed up that address or the memory around it, check the accounting information immediately before it. Graphically, that would be along the lines of:
____ The allocated block ____
/ \
+--------+--------------------+
| Header | Your data area ... |
+--------+--------------------+
^
|
+-- The address you are given
Keep in mind the size of the header and the padding are totally implementation defined (actually, the entire thing is implementation-defined (a) but the in-line accounting option is a common one).
The checksums and special markers that exist in the accounting information are often the cause of errors like "Memory arena corrupted" or "Double free" if you overwrite them or free them twice.
The padding (to make allocation more efficient) is why you can sometimes write a little bit beyond the end of your requested space without causing problems (still, don't do that, it's undefined behaviour and, just because it works sometimes, doesn't mean it's okay to do it).
(a) I've written implementations of malloc in embedded systems where you got 128 bytes no matter what you asked for (that was the size of the largest structure in the system), assuming you asked for 128 bytes or less (requests for more would be met with a NULL return value). A very simple bit-mask (i.e., not in-line) was used to decide whether a 128-byte chunk was allocated or not.
Others I've developed had different pools for 16-byte chunks, 64-bytes chunks, 256-byte chunks and 1K chunks, again using a bit-mask to decide what blocks were used or available.
Both these options managed to reduce the overhead of the accounting information and to increase the speed of malloc and free (no need to coalesce adjacent blocks when freeing), particularly important in the environment we were working in.
From the comp.lang.c FAQ list: How does free know how many bytes to free?
The malloc/free implementation remembers the size of each block as it is allocated, so it is not necessary to remind it of the size when freeing. (Typically, the size is stored adjacent to the allocated block, which is why things usually break badly if the bounds of the allocated block are even slightly overstepped)
This answer is relocated from How does free() know how much memory to deallocate? where I was abrubtly prevented from answering by an apparent duplicate question. This answer then should be relevant to this duplicate:
For the case of malloc, the heap allocator stores a mapping of the original returned pointer, to relevant details needed for freeing the memory later. This typically involves storing the size of the memory region in whatever form relevant to the allocator in use, for example raw size, or a node in a binary tree used to track allocations, or a count of memory "units" in use.
free will not fail if you "rename" the pointer, or duplicate it in any way. It is not however reference counted, and only the first free will be correct. Additional frees are "double free" errors.
Attempting to free any pointer with a value different to those returned by previous mallocs, and as yet unfreed is an error. It is not possible to partially free memory regions returned from malloc.
On a related note GLib library has memory allocation functions which do not save implicit size - and then you just pass the size parameter to free. This can eliminate part of the overhead.
The heap manager stored the amount of memory belonging to the allocated block somewhere when you called malloc.
I never implemented one myself, but I guess the memory right in front of the allocated block might contain the meta information.
The original technique was to allocate a slightly larger block and store the size at the beginning, then give the application the rest of the blog. The extra space holds a size and possibly links to thread the free blocks together for reuse.
There are certain issues with those tricks, however, such as poor cache and memory management behavior. Using memory right in the block tends to page things in unnecessarily and it also creates dirty pages which complicate sharing and copy-on-write.
So a more advanced technique is to keep a separate directory. Exotic approaches have also been developed where areas of memory use the same power-of-two sizes.
In general, the answer is: a separate data structure is allocated to keep state.
malloc() and free() are system/compiler dependent so it's hard to give a specific answer.
More information on this other question.
To answer the second half of your question: yes, you can, and a fairly common pattern in C is the following:
typedef struct {
size_t numElements
int elements[1]; /* but enough space malloced for numElements at runtime */
} IntArray_t;
#define SIZE 10
IntArray_t* myArray = malloc(sizeof(intArray_t) + SIZE * sizeof(int));
myArray->numElements = SIZE;
to answer the second question, yes you could (kind of) use the same technique as malloc()
by simply assigning the first cell inside every array to the size of the array.
that lets you send the array without sending an additional size argument.
When we call malloc it's simply consume more byte from it's requirement. This more byte consumption contain information like check sum,size and other additional information.
When we call free at that time it directly go to that additional information where it's find the address and also find how much block will be free.
When using malloc, if it produces a core dump with the error:
malloc(): memory corruption: ....... ***
Does this mean that malloc tried to allocate memory that was not free to allocate? IF so what are the causes of this?
It completely depends on your malloc implementation, but usually what this is means is that at some point prior to that malloc something wrote more data to a malloced buffer than its size.
A lot of malloc implementations store some of their data inline with their memory, in other words:
+--------------------------------+
|14 bytes -> Padding |
+--------------------------------+
|2 bytes -> Internal malloc info |
+--------------------------------+
|6 bytes -> Your data |
+--------------------------------+
|8 bytes -> Padding |
+--------------------------------+
|2 bytes -> Internal malloc info |
+--------------------------------+
So if some code of yours or a library wrote 16 bytes to that 6 byte buffer it would overwrite the padding and the 2 bytes of internal malloc info. The next time you call malloc it will try to walk through its data to find space, hit the overwritten space, and it will be nonsensical since you overwrote it, corrupting the heap.
Depending on the implementation such an error could also be caused by making a double free.
Most likely, this is not a problem in malloc itself. Rather, this is a problem with your application modifying parts of the heap that it shouldn't.
If you are running on Linux, try using Valgrind to see which code is trashing your heap.
The usual cause of this is that you wrote over data that malloc() did not give you permission to write over - either buffer overrun (writing beyond the end of the space you were given) or buffer underrun (writing before the start of the buffer).
It can sometimes be caused by freeing a pointer that was not allocated by malloc() et al, or by re-freeing (double freeing) a pointer that was allocated by malloc(). For example, freeing a static buffer is a bad idea; you will get corruption.
You should assume that the problem is in your code - it is extremely unlikely to be a problem in malloc() et al, and rather unlikely to be in any other library you are using.
There are several things that are usual causes of heap corruption:
overrunning the memory allocation (writing past the end of the allocated block)
double freeing a block
using a pointer after it's been freed
and of course something writing erroneously through a pointer that has nothing to do with a previous allocation (a 'ram hit' or rogue pointer) - this is the general case that includes all of the above.
These problems can be difficult to debug because the cause and effect are often separated by time and space (different area of code). So the bug doesn't get noticed until an eternity (in computer time) passes after the bug that caused the problem executes.
Using a debug heap can be very helpful in debugging these issues. Microsoft's compilers have a CrtDebug heap that's enabled in debug builds (but can have additional configuration items set). I'm not sure what GCC has out of the box, but there are tools I'm familiar with in passing such as Valgrind and Electric Fence that might help. Finally there a ton of home-grown heap debug libraries that might be helpful (Google around).
Could you please provide your malloc() statement?
Also, I wanted to double check that the return value is not null?
Outside of not having the memory to allocate to begin with, the problems I have encountered when using malloc() or new similar the nature you mentioned where actually resultant of a corrupted heap. I usually found some "interesting" code elsewhere in the program doing soomething like memcpy() with a character buffer causing a buffer overrun and a mangled address space.
-bn