This question already has answers here:
How does free know how much to free?
(11 answers)
Closed 6 years ago.
I have a sort of theoretical question about freeing a pointer.
I think some guy (the program itself, or OS) have to keep track of the pointers and the length of them. Why?
float* pointer1 = malloc( 100 * sizeof(float) );
For example: I have a float pointer here. Can somebody tell me what is the length of my pointer1 array? There is no way to get the length of that pointer with calling a function/method at run-time right? So how the program (or the OS) is able to free that memory part of the pointer without knowing the real length of it at run-time:
free(pointer1);
If the program knew the length of the pointer to free, why we are not able to get that information and using some other ways to keep track of the lengths. For example:
struct floatArray
{
float* pointer;
int length;
}
So, I am really curious about this issue. I really want to learn how the OS take cares of the pointers lenghts.
Thanks,
Sait.
The memory manager does indeed maintain metadata about each allocated block of memory. This will likely contain information in addition to the size of the block.
One common way to implement this is to store the metadata in memory just below the memory address which the user program uses to refer to the block. So if the call to malloc returns address p, and the meta data contains n bytes, then the meta data will be stored in addresses p-n to p-1.
It is usually not directly the OS, it is the malloc() implementation - the memory manager, which gets larger blocks of memory from the OS. Usually, this information will be stored in some linked-list but I suggest you look at your libc's malloc() to see how this information is really stored.
As pointed out, there is not a standard way to do it, because the implementation varies between different compilers/standard C libraries-
It is not necessarily the case that the OS tracks the exact length that you requested. Often the length is altered (increased) for various internal reasons, or stored in a way that is slow to access. It may have additional information encoded in it (typically the size is aligned to some multiple of a power of two and the low bits used as flags). Some malloc implementations (particularly embedded ones) simply don't implement free() at all, and have no need to store this information.
The C language designers wanted to allow this kind of freedom to the C library implementation, and so they don't allow the program to retrieve the size of a malloc'd buffer.
With MSVC or MinGW you can use the (non-standard!) function _msize
float* pointer1 = malloc( 100 * sizeof(float) );
printf("%lu bytes with %lu elements", (unsigned long)_msize(pointer1),(unsigned long)_msize(pointer1)/sizeof*pointer1);
Related
This question already has answers here:
Determine size of dynamically allocated memory in C
(15 answers)
Closed 5 years ago.
I have read in: How does free know how much to free? that when one has some memory allocation denoted with a pointer such as
float (*ptr)[10]=malloc(sizeof(float)*100)
for a 10x10 array, ptr has a "head" to it with "accounting" information telling of the "step size" and what not so that you can properly perform pointer arithmetic and use free and whatnot.
Is there a consistent (not architecture dependent) and reliable (defined behavior) that can allow one to get their hands on this information?
I have read elsewhere that the de facto way to track array length when there are casts and dynamic memory allocations about is to manually allocate a slot to store the size. This naturally leads me to believe the answer to my question is 'no' yet I think I'd rather not make assumptions or I'll get my own sort of memory leakage.
Converting comments into an answer.
There is no defined standard way to get at the 'size of the block of allocated memory'. Each implementation has to have a way of knowing the size of each block it allocates, but there's no way for a programmer using the implementation to know the size (in general).
So it is dependent on some number of things, but if all is known, system, architecture, compiler, you're saying there is no resource to find out how things are formatted in memory?
There is no standard (neither de jure nor de facto standard) way to get at the information about the size of a block of memory allocated. All else apart, the size allocated by the library is usually bigger than the size requested (definitely because of the housekeeping data, but even the data portion may be rounded up to a multiple of 8 or a multiple of 16) — should the code report the size requested or the size allocated?
And, as 1201ProgramAlarm noted, one option on open source systems is to look at the C library's implementation of malloc() and free() to see what it does and devise a mechanism to provide the answer to the programmer. However, any such research is specific to that system — different systems will do it differently, in general — and the whole idea runs into a stone wall if the system is a closed source system.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
free() only needs the pointer value to release allocated memory. Which means that C knows how big assigned memory blocks are. Then how come there isn't some built-in function to find the size of a pointer array?
I know the convention is to keep track of array sizes, but given that there already is some memory management happening natively, why don't we harness that to have a convenient size() function for arrays?
Such a function would be possible. The question is why the C standard doesn't require it.
The GNU C library implementation provides a function malloc_usable_size() that's similar to what you're suggesting. It returns the number of usable bytes for a malloc()ed pointer -- which may be greater than the requested size for various reasons.
It's not 100% clear why no such function is in the standard. It's not mentioned in the 1989 ANSI C Rationale or in the ISO C99 Rationale.
One thing to keep in mind is that an implementation doesn't need to keep track of the number of bytes requested by a malloc() call. It will often round up the request to some larger value, and it only needs to keep track of that.
For that matter, free() doesn't necessarily need to know the size of the block being deallocated. It might just call some lower-level function that doesn't provide that information. Or, for example, allocated blocks might b organized into linked lists, one list for each allocated size; free() might simply release a block from that list without having to know the size.
Finally, C programmers have gotten along without such a function for decades. Adding a requirement to provide it would impose some (probably fairly small) overhead on all implementations. I think the attitude is that you can simply remember how much memory you asked for, and use that information as needed.
If you allocate a single object:
some_type *ptr = malloc(sizeof *ptr);
then sizeof *ptr gives you the size of the object. If you allocate an array:
some_type *ptr = malloc(count * sizeof *ptr);
then sizeof *ptr only gives you the size of a single element of the allocated array -- but if you remember the value of count you can compute the total requested size easily enough.
Bottom line: The C standard could have required such a function, but it's not really necessary.
UPDATE: Kerrek SB makes an excellent point in a comment, one that I hadn't thought of. I'll take the liberty of summarizing it here.
A function that operates on an array via a pointer to its initial element (and there are a lot of such functions) shouldn't have to care how the array was allocated. The proposed size() function, like the GNU-specific malloc_usable_size(), works only when the argument points to a heap-allocated array. This means that the function either has to assume that the array is heap-allocated (and be right about that assumption!) or be given extra information. In the latter case, it might as well be given the size of the array, making size() superfluous.
free() may use internal data to reclaim the block of memory being released, but be aware that this data does not necessarily contain the exact size passed to malloc(), calloc(), or realloc() to allocate the block. The C Standard does not specify a function to retrieve this information.
Most malloc() implementations provide a non-standard function to retrieve the available size of the allocated block: in the Glibc, this function is size_t malloc_usable_size(void *ptr);. Other libraries may have a different function or no function at all to retrieve this information.
As for a generic solution to retrieve the size of the array to which you have a pointer, this is usually not possible. In efficient implementations, pointers do not carry size information. It is possible to implement fat pointers that would carry this information, but the whole system needs to be compiled this way. Some integrated compilers such as tcc support this approach to provide runtime pointer checking.
Because basically, the address will point on a chunk of memory, which contains meta-data (such as the size of the chunk). Freeing that entry will actually mark the block available (if the pointer is valid).
If the caller access that memory location afterward, that's undefined behaviour. So even from that point of view, free will have its job done.
free() only needs the pointer value,because you can only pass the pointer that malloc() return.The malloc() will write the size of this assign in the front address of return pointer.When you pass the pointer to free(),free() will read the size,so free() knows how many space to release.Therefor,there not used any function to find the size of a pointer array.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
C programming : How does free know how much to free?
A random thought occured to me that how comes free(myPtr) doesn't need a length parameter? How does it know how much memory to free?
I can only guess that it keeps track of how much memory it allocates for each particular start address.
This is because malloc saves the information about the length of the allocated chunk, usually in a spot that precedes the address returned to your program. It is not unusual for implementations to allocate an extra storage for a size_t, put the size there, add sizeof(size_t), and return it to you as the malloc-ed pointer. The standard does not call for this implementation, thoug, so alternative implementations are possible, e.g. based on a hash table.
When C allocates memory, it records the length associated with the pointer it gives you. (Often in the area just before the block of memory. But that's an implementation detail.) It keeps some kind of table or list of memory blocks it's handed out, and when you free that memory, C looks up the length of that block based on the value of the pointer.
That's part of why the pointer you pass to free has to be exactly equal to the one you get back from malloc. If it's not, C gets confused and can't find the correct memory block (or its length), and may very well end up "freeing" some memory it was never meant to touch. (If it does that, you may end up with a condition called "heap corruption", which is really bad -- from then on, C might do all kinds of wacky stuff, like trying to allocate some memory in the middle of an existing block and mangling whatever's there.)
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
C programming : How does free know how much to free?
Hi,
When using malloc(), we specify the size of the allocation, so it knows how much to allocate. However, how does free() knows how many bytes to release? The pointer contains only the starting address of the memory block, not the length of memory block.
Thanks and Regards,
Tazim.
The answer to this is implementation-dependent; the malloc library keeps track of the length somehow, but exactly how it does so is not specified by the C language standard.
A typical approach is to store some header information (including length) before the "starting address" that malloc returns to the caller.
Malloc saves the size of the allocated pointer in some kind of data structure. When you call free it looks up the entry in this data structure and free's that much memory.
When you allocate memory, the run-time library also maintains some internal structures. It has to in order to keep track of what parts of the heap have been allocated. This information also tells it the size of a block of memory given a pointer to that memory.
It depends on the implementation but there's a good chance this information is stored just before the pointer returned.
It's implementation-specific. Some techniques:
There may be more than one pointer. An arbitrarily complex structure could be allocated and you just get a pointer to the user-payload area. The library knows the fixed offset between the pointer given to you and the pointer to the origin of the structure. The other fields could be the size and the links that thread free blocks together.
There may be a separate dictionary. This can have memory-management advantages. One problem with using the allocated block for book-keeping is that the library itself ends up writing to many if not most of the allocated pages. This keeps them dirty (in an MMU sense) and can also prevent them from being shared following a fork. This is a big problem for web servers and has led to specialized implementations of web language systems ("Ruby Enterprise") that differ mainly in memory management core.
This question already has answers here:
What's the point of using malloc when you can use pointer? [duplicate]
(3 answers)
Closed 5 years ago.
#include<stio.h>
main()
{
int *p,i;
p = (int*)malloc(sizeof(int));
printf("Enter:");
scanf("%d",p);
for(i=1;i<3;i++)
{
printf("Enter");
scanf("%d",p+i);
}
for(i=0;i<3;i++)
{
printf("No:%d\n",*(p+i));
}
getch();
return 0;
}
In this C program memory is accessed without allocation.The program works.Will any problem arise by accessing memory without allocation?If yes then what is the solution for storing a collection of integer data which the size is not known in advance?
Yes, it leads to undefined behavior. The problem is working here purely becuase of luck and may crash any time. The solution is to allocate the memory using malloc For example if you want to allocate memory for count number of elements then you can use int* p = (int*)malloc(sizeof(int)*count);. From here on you can access p as an array of count elements.
It likely works because the memory immediately after *p is both accessible (allocated in the VM system and has the right bits set), and not in use for anything else. This could all change if malloc finds you some bytes immediately before an inaccessible page; or if you move to a malloc implementation that uses the trailing space for bookkeeping.
So it's not really safe.
Accessing unallocated memory leads to undefined behavior. Exactly what happens will depends on a variety of conditions. It may "work" now but you could see problems when you extend your program.
If you don't know how many items you want to read, there are a couple of strategies to use.
Use realloc to grow the buffer as you need more space.
Use a linked list instead of an array
Most definitely yes. Its just pure luck that you can access without allocating. malloc does not what memory you are using and that could result in serious problems.
Hence its a compulsion (i don't want to use the word better here) to allocate memory according to your needs and then use it.
Some problems which could result are:
Segmentation fault
Memory corruption
and it may result in giving you headache for hours when the behavior is undefined.
For eg: the location of a crash may not be the exact place of origin of the problem.
The reason this code works is that the kernel never gives you a fraction of the system page size (which should be 4k). This means the memory after the first sizeof(int) bytes is actually owned by the process you run, but not allocated to you by the second layer of abstraction which is malloc.
"Segmentation fault" happens when you try and access memory outside the pages allocated to you by the kernel. You won't see it until you step out of your page.
The problem that may arise here is that you use malloc again and you will receive a pointer to a memory you used without malloc being aware of it. This will cause hellish bugs since you will change data used in different contexts without knowing.
As for your second question, the right way is very program dependent.
If the number of elements can be bounded reasonably, it might be OK to always allocate the same size using a constant defined in your program. This is ALWAYS the secure way (you need to make sure you don't let the user give you more than what you allocated).
If you really have a broad range of array sizes here, you might want to use a linked list which is built for that exactly.
There are two levels of memory allocation that take usually take place. At operating system level, you map memory pages to your address space. A page is the basic unit of memory management and is usually something like 1K or 4K bytes (but can be much larger or as small as 512 bytes, depending upon the system). It is possible to do that mapping yourself by making the appropriate system calls. However, applications generally only do that when they need large blocks of memory.
Standard libraries generally maintain a pool of pages. When you call malloc, the library looks to see if there is available memory in the pool. If so, it returns a block of memory from pages already mapped by the operating system. If not, the library make the system call to map more pages to the process and adds them to the managed pool.
Mapping and unmapping pages is a rather time consuming process. By using pooling, the library can speed things up significantly.
Invariable, the standard library functions allocate a few bytes in front of the memory returned by malloc and the like so that they can know how much memory is in the block when it is free'd. Many will also add memory add the end of the block as well for error checking.
When you are doing what you are doing, you could be reading this extra data or you could be reading some data that was mapped to the memory pool by the library.
What you are doing is bad.
IF you do not know the number of items in advance, you can use a data structure, such a linked list where new entries are created with each new number.