This question already has answers here:
Determine size of dynamically allocated memory in C
(15 answers)
Closed 5 years ago.
I have read in: How does free know how much to free? that when one has some memory allocation denoted with a pointer such as
float (*ptr)[10]=malloc(sizeof(float)*100)
for a 10x10 array, ptr has a "head" to it with "accounting" information telling of the "step size" and what not so that you can properly perform pointer arithmetic and use free and whatnot.
Is there a consistent (not architecture dependent) and reliable (defined behavior) that can allow one to get their hands on this information?
I have read elsewhere that the de facto way to track array length when there are casts and dynamic memory allocations about is to manually allocate a slot to store the size. This naturally leads me to believe the answer to my question is 'no' yet I think I'd rather not make assumptions or I'll get my own sort of memory leakage.
Converting comments into an answer.
There is no defined standard way to get at the 'size of the block of allocated memory'. Each implementation has to have a way of knowing the size of each block it allocates, but there's no way for a programmer using the implementation to know the size (in general).
So it is dependent on some number of things, but if all is known, system, architecture, compiler, you're saying there is no resource to find out how things are formatted in memory?
There is no standard (neither de jure nor de facto standard) way to get at the information about the size of a block of memory allocated. All else apart, the size allocated by the library is usually bigger than the size requested (definitely because of the housekeeping data, but even the data portion may be rounded up to a multiple of 8 or a multiple of 16) — should the code report the size requested or the size allocated?
And, as 1201ProgramAlarm noted, one option on open source systems is to look at the C library's implementation of malloc() and free() to see what it does and devise a mechanism to provide the answer to the programmer. However, any such research is specific to that system — different systems will do it differently, in general — and the whole idea runs into a stone wall if the system is a closed source system.
Related
I'm using WinAPI's HeapAlloc() function for allocating memory, and I want to find out the size of it somewhere else in my code. Do I have to keep track of the sizes myself or is there another way?
HeapAlloc rounds up allocations to the nearest alignment. If you ask for 2 bytes, it will give you at least two bytes, but may give you more. As the documentation says:
If the HeapAlloc function succeeds, it allocates at least the amount of memory requested.
The specific alignment that HeapAlloc uses is not documented, but if I remember correctly, all of the Heap Manager APIs use an 8-byte alignment on 32-bit x86 and a 16-byte alignment on 64-bit x86. This old knowledge base article jives with my recollection. Of course, because it is not explicitly documented and subject to change in future versions of Windows and/or on different architectures, you should not rely on hard-coded alignment values.
However, if these functions do allocate more memory than request, the caller is free to use all of that memory. To determine the actual size of the allocation, you call the HeapSize function. Raymond Chen blogged about this some time ago.
So the behavior you're seeing actually makes sense, even though you are going about making the determination in entirely the wrong way. As has been pointed out in the comments already, sizeof doesn't tell you the size of the allocation. You need HeapSize to do that. All sizeof tells you is the size of the element matches[lastMeal] at compile time, which is also 8 bytes instead of 2 for alignment reasons.
As for your edit: best practice is to track this information yourself. Whenever you pass a pointer, pass the size of the allocation along with it. Note that this should be the expected size of the allocation (the 2 bytes that you asked for), not the actual size of the allocation (the 8 bytes that the Heap Manager returned for internal alignment considerations). When you free the memory by calling HeapFree, it knows how big the actual allocation was and will free it as necessary. There is, however, no way for your client code to determine the size of the allocation that you initially requested, which is why you need to track it yourself.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
c difference between malloc and calloc
Why does calloc require two parameters and malloc just one?
I've noticed this with many C functions calls particularly ones that deal with memory or file operations, but not all of them use both parameters. For instance malloc is passed one parameter, the size in bytes of the memory space needed. Calloc on the other hand is passed two parameters, the size in bytes of an element and the number of elements (size and nmem). There are other functions that use these size and nmem parameters as well.
Essentially the calloc call would allocate the same amount of memory as calling malloc(nmemsize) so all that's really happening is the asterisk () is replaced with a comma (,). At least this is all I can tell from the higher level that I am working at. I don't see a difference from calling calloc(1, nmemsize), calloc(nmemsize, 1), or calloc(nmem, size).
Is there something actually happening at a lower level that makes calling for instance calloc(1, nmem*size) fundamentally different from calloc(nmem, size)?
Edit: I know the functional difference between calloc and malloc. I'm interested in why there are differences in the parameters. There are other functions that use 2 size parameters for the total size (fread, fwrite, etc). I'm not concerned with the specific functions but rather why there are two parameters for the total size used in the function when essentially the total size becomes the two parameters multiplied together. I find most of the time when I use these functions I use the size that I need in the "size" parameter and a '1' for the "nmem" (sometimes "count" etc.) parameter.
In a comment to the question, I wrote that calloc() allows better memory alignment for platforms where it matters. I haven't been able to find anything to support that (yet). I am pretty sure it was a feature of the VMS/VAXC compiler, but source for that is scarce.
However, I did find that calloc() and alloc() appeared at the same time, with the release of Unix V6 in May 1975. In V5, released 11 months earlier, neither function is present; the kernel and runtime library (and assembler and C compiler) were written in assembly.
In the V6 release, calloc is implemented as the four line source code module:
calloc(n, s)
{
return(alloc(n*s));
}
calloc() does not clear the allocated memory; see alloc(), and there was no man page for calloc() in V6; however the man page for alloc():
DESCRIPTION
Alloc and free provide a simple general-purpose core management package.
Alloc is given a size in bytes; it returns a pointer to an area at least that size which
is even and hence can hold an object of any type. The argument to free
is a pointer to an area previously allocated by alloc; this space is made available for further allocation.
Needless to say, grave disorder will result if the space
assigned by alloc is overrun or if some random number is handed to free.
The routine uses a first-fit algorithm which coalesces blocks being freed with other
blocks already free. It calls sbrk (see "break (II))"
to get more core from the system when there is no suitable space already free.
DIAGNOSTICS
Returns -1 if there is no available core.
BUGS
Allocated memory contains garbage instead of being cleared.
Not even NULL is returned in the case of memory exhaustion!
calloc() first formally appears in UNIX V7, January 1979, along with several other improvements:
calloc() clears the memory returned.
alloc() was renamed to malloc()
realloc() appeared
in the case of memory exhaustion or a heap error, the functions "return a null pointer (0)"
Is there something actually happening at a lower level that makes calling for instance calloc(1, nmem*size) fundamentally different from calloc(nmem, size)?
This attempt to explain things is purely dependent from the libc implementation - and therefore left at the appreciation of a specific libc author:
Since calloc() is zeroing memory, the rationale might have been that it could (potentially) waste some more cycles at doing a mult.
In contrast, malloc() is given a chance to use a precalculated value, potentially reducing the overhead in a call that migh be simpler to satisfy.
Don't forget that C was designed at a time when each CPU cycle was costing a lot - hence a very lean design as compared to many other 'higher-level' languages.
This question could probably be better answered by the author of C Dennis Ritchie.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
C programming : How does free know how much to free?
A random thought occured to me that how comes free(myPtr) doesn't need a length parameter? How does it know how much memory to free?
I can only guess that it keeps track of how much memory it allocates for each particular start address.
This is because malloc saves the information about the length of the allocated chunk, usually in a spot that precedes the address returned to your program. It is not unusual for implementations to allocate an extra storage for a size_t, put the size there, add sizeof(size_t), and return it to you as the malloc-ed pointer. The standard does not call for this implementation, thoug, so alternative implementations are possible, e.g. based on a hash table.
When C allocates memory, it records the length associated with the pointer it gives you. (Often in the area just before the block of memory. But that's an implementation detail.) It keeps some kind of table or list of memory blocks it's handed out, and when you free that memory, C looks up the length of that block based on the value of the pointer.
That's part of why the pointer you pass to free has to be exactly equal to the one you get back from malloc. If it's not, C gets confused and can't find the correct memory block (or its length), and may very well end up "freeing" some memory it was never meant to touch. (If it does that, you may end up with a condition called "heap corruption", which is really bad -- from then on, C might do all kinds of wacky stuff, like trying to allocate some memory in the middle of an existing block and mangling whatever's there.)
This question already has answers here:
How does free know how much to free?
(11 answers)
Closed 6 years ago.
I have a sort of theoretical question about freeing a pointer.
I think some guy (the program itself, or OS) have to keep track of the pointers and the length of them. Why?
float* pointer1 = malloc( 100 * sizeof(float) );
For example: I have a float pointer here. Can somebody tell me what is the length of my pointer1 array? There is no way to get the length of that pointer with calling a function/method at run-time right? So how the program (or the OS) is able to free that memory part of the pointer without knowing the real length of it at run-time:
free(pointer1);
If the program knew the length of the pointer to free, why we are not able to get that information and using some other ways to keep track of the lengths. For example:
struct floatArray
{
float* pointer;
int length;
}
So, I am really curious about this issue. I really want to learn how the OS take cares of the pointers lenghts.
Thanks,
Sait.
The memory manager does indeed maintain metadata about each allocated block of memory. This will likely contain information in addition to the size of the block.
One common way to implement this is to store the metadata in memory just below the memory address which the user program uses to refer to the block. So if the call to malloc returns address p, and the meta data contains n bytes, then the meta data will be stored in addresses p-n to p-1.
It is usually not directly the OS, it is the malloc() implementation - the memory manager, which gets larger blocks of memory from the OS. Usually, this information will be stored in some linked-list but I suggest you look at your libc's malloc() to see how this information is really stored.
As pointed out, there is not a standard way to do it, because the implementation varies between different compilers/standard C libraries-
It is not necessarily the case that the OS tracks the exact length that you requested. Often the length is altered (increased) for various internal reasons, or stored in a way that is slow to access. It may have additional information encoded in it (typically the size is aligned to some multiple of a power of two and the low bits used as flags). Some malloc implementations (particularly embedded ones) simply don't implement free() at all, and have no need to store this information.
The C language designers wanted to allow this kind of freedom to the C library implementation, and so they don't allow the program to retrieve the size of a malloc'd buffer.
With MSVC or MinGW you can use the (non-standard!) function _msize
float* pointer1 = malloc( 100 * sizeof(float) );
printf("%lu bytes with %lu elements", (unsigned long)_msize(pointer1),(unsigned long)_msize(pointer1)/sizeof*pointer1);
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
C programming : How does free know how much to free?
Hi,
When using malloc(), we specify the size of the allocation, so it knows how much to allocate. However, how does free() knows how many bytes to release? The pointer contains only the starting address of the memory block, not the length of memory block.
Thanks and Regards,
Tazim.
The answer to this is implementation-dependent; the malloc library keeps track of the length somehow, but exactly how it does so is not specified by the C language standard.
A typical approach is to store some header information (including length) before the "starting address" that malloc returns to the caller.
Malloc saves the size of the allocated pointer in some kind of data structure. When you call free it looks up the entry in this data structure and free's that much memory.
When you allocate memory, the run-time library also maintains some internal structures. It has to in order to keep track of what parts of the heap have been allocated. This information also tells it the size of a block of memory given a pointer to that memory.
It depends on the implementation but there's a good chance this information is stored just before the pointer returned.
It's implementation-specific. Some techniques:
There may be more than one pointer. An arbitrarily complex structure could be allocated and you just get a pointer to the user-payload area. The library knows the fixed offset between the pointer given to you and the pointer to the origin of the structure. The other fields could be the size and the links that thread free blocks together.
There may be a separate dictionary. This can have memory-management advantages. One problem with using the allocated block for book-keeping is that the library itself ends up writing to many if not most of the allocated pages. This keeps them dirty (in an MMU sense) and can also prevent them from being shared following a fork. This is a big problem for web servers and has led to specialized implementations of web language systems ("Ruby Enterprise") that differ mainly in memory management core.