I thought that I couldn't retrieve the length of an allocated memory block like the simple .length function in Java. However, I now know that when malloc() allocates the block, it allocates extra bytes to hold an integer containing the size of the block. This integer is located at the beginning of the block; the address actually returned to the caller points to the location just past this length value. The problem is, I can't access that address to retrieve the block length.
#include <stdlib.h>
#include <stdio.h>
int main(void)
{
char *str;
str = (char*) malloc(sizeof(char)*1000);
int *length;
length = str-4; /*because on 32 bit system, an int is 4 bytes long*/
printf("Length of str:%d\n", *length);
free(str);
}
**Edit:
I finally did it. The problem is, it keeps giving 0 as the length instead of the size on my system is because my Ubuntu is 64 bit. I changed str-4 to str-8, and it works now.
If I change the size to 2000, it produces 2017 as the length. However, when I change to 3000, it gives 3009. I am using GCC.
You don't have to track it by your self!
size_t malloc_usable_size (void *ptr);
But it returns the real size of the allocated memory block!
Not the size you passed to malloc!
What you're doing is definitely wrong. While it's almost certain that the word just before the allocated block is related to the size, even so it probably contains some additional flags or information in the unused bits. Depending on the implementation, this data might even be in the high bits, which would cause you to read the entirely wrong length. Also it's possible that small allocations (e.g. 1 to 32 bytes) are packed into special small-block pages with no headers, in which case the word before the allocated block is just part of another block and has no meaning whatsoever in relation to the size of the block you're examining.
Just stop this misguided and dangerous pursuit. If you need to know the size of a block obtained by malloc, you're doing something wrong.
I would suggest you create your own malloc wrapper by compiling and linking a file which defines my_malloc() and then overwiting the default as follows:
// my_malloc.c
#define malloc(sz) my_malloc(sz)
typedef struct {
size_t size;
} Metadata;
void *my_malloc(size_t sz) {
size_t size_with_header = sz + sizeof(Metadata);
void* pointer = malloc(size_with_header);
// cast the header into a Metadata struct
Metadata* header = (Metadata*)pointer;
header->size = sz;
// return the address starting after the header
// since this is what the user needs
return pointer + sizeof(Metadata);
}
then you can always retrieve the size allocated by subtracting sizeof(Metadata), casting that pointer to Metadata and doing metadata->size:
Metadata* header = (Metadata*)(ptr - sizeof(Metadata));
printf("Size allocated is:%lu", header->size); // don't quote me on the %lu ;-)
You're not supposed to do that. If you want to know how much memory you've allocated, you need to keep track of it yourself.
Looking outside the block of memory returned to you (before the pointer returned by malloc, or after that pointer + the number of bytes you asked for) will result in undefined behavior. It might work in practice for a given malloc implementation, but it's not a good idea to depend on that.
This is not Standard C. However, it is supposed to work on Windows operatings systems and might to be available on other operating systems such as Linux (msize?) or Mac (alloc_size?), as well.
size_t _msize( void *memblock );
_msize() returns the size of a memory block allocated in the heap.
See this link:
http://msdn.microsoft.com/en-us/library/z2s077bc.aspx
This is implementation dependent
Every block you're allocating is precedeed by a block descriptor. Problem is, it dependends on system architecture.
Try to find the block descriptor size for you own system. Try take a look at you system malloc man page.
Related
Is there a max size for a char buffer? I have a program that is collecting strings for a char buffer and writing it to a proc file. After a certain point it appears to stop writing things - is there too much in there? What is that max size so I can work around this?
Here is code. This is an LKM - is limits.h available from kernel space?
Foremost:
const char* input = "hooloo\n";
Next:
int read_info( char *page, char **start, off_t off, int count, int *eof, void *data )
{
unsigned int mem;
char answer_buf[strlen(input) + 1 + 14];
name_added = vmalloc(strlen(input) + 1 + 14);
strcpy(name_added, input);
strcat(name_added, extension);
mem = sprintf(answer_buf, "%s\n", name_added);
memcpy(page, answer_buf, mem);
return strlen(answer_buf) + 1;
}
All in my code are things like this are things that remalloc the buffer and add to it. Also, that read_info is for the procfile. This issue is I keep adding to that buffer with the code above over and over and over - eventually I ca my procfile and the text cuts off - it doesn't go on forever like i want )-=.
There's no concrete maximum size "in C" specifically. Theoretical (or "potential") maximum size of any object on a C platform is determined by the implementation and is usually derived from the properties of the underlying machine platform and OS.
On platforms with flat memory model it will typically be limited by the size of the address space in theory, and by the size of the available free memory (or that specific kind) in practice.
On platforms with segmented memory model it might be limited by the segment size, which is smaller than the address space size. Although implementations are free to breach that limit by "emulating" flat memory model in the code. For that reason on such platforms the maximum object size can also depend on compilation settings.
The only maximum size for a dynamically allocated char buffer will be available system memory.
A buffer on the stack will have its size constrained by maximum stack size. This will vary greatly depending on host OS.
When writing data to file, are you checking the size returned by fwrite and calling it repeatedly to write the remainder of the buffer if necessary?
You have a memory leak in your code!
The following memory is never freed:
name_added = vmalloc(strlen(input) + 1 + 14);
I don't understand why you allocate memory for the output at all.
And you do it twice, both on the stack and on the heap.
The caller has provided a buffer for the output. Use it!
Don't create copies!
I'd say it's at least able to handle 1,000 unique characters
I am working on an embedded system (ARM Cortex M3) where I do not have access to any sort of "standard library". In particular, I do not have access to malloc.
I have a function void doStuff(uint8_t *buffer) that accepts a pointer to a 512 bits buffer. I have tried doing the following:
uint8_t buffer[64] = {0};
doStuff((uint8_t *) &buffer));
but I'm not getting the expected results. Am I doing something wrong? Is there any alternative approach?
doStuff(buffer) shall be ok since buffer is already an uint8_t*.
Aside of this, you're closing one bracket too much after &buffer in your example.
If buffer is of variable size, you should pass the size into doStuff too, if it's of constant size, I'd also pass the size just in case that you change the size one day.
This being said, you should do it the following way:
uint8_t buffer[64] = {0};
int len = 64;
doStuff(buffer, len);
a simplemalloc(): have a char mem[MAXMEM]; and a struct freetable. Then write your own simplemalloc(), that finds in freetable a big enough junk of memory, and returns the offset into mem. simplefree() would then adjust the freetable.
EDITH:
if you need a lot of malloc()s, you may even devide your static mem into different chunks for different tasks (one chunk for exactly 100byte allocs, one fort the size of your favorite struct, and so on) this will speed up finding free mem.
if you are short of mem, you should implement a bestmatch() in simplemalloc(), which as a bad sideeffect slows the execution down.
if you have enough memory, you can implement a debugversion, which "allocates" a bit more memory and puts XXX before the start and after the end of the simplemalloc()ed mem. on free() you can check, if this XXX is broken, so you know you have some buffer-over or under-flow, which you can be aware of.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
How do free and malloc work in C?
How does free know how many bytes of memory to be free'd when called in a program?
This is implementation specific, but when malloc is called, the size of the allocated memory is kept somewhere (usually offset from the pointer itself). When free is called, it will use that stored size.
This is exactly why you should only ever call free on a pointer that was returned by malloc.
It's done automatically. The corresponding "malloc" has saved the size in a secret place (typically stored at a negative offset from the pointer).
This, of course, mean that you can only free memory that corresponds to a block previously allocated by "malloc".
Asking how it knows "how many bytes to free" is a mistake. It's not like each byte individually has a free/not-free status bit attached to it (well, it could, but this would be an awful implementation). In many implementations the number of bytes in an allocation may be completely irrelevant; it's the data structures used to manage it that are relevant.
It's an implementation detail than can and will vary between different platforms. Here's one example though of how it could be implemented.
Every free call must be paired with a malloc / realloc call which knows the size request. The implementation of malloc could choose to store this size at an offset of the returned memory. Say by allocating a larger buffer than requested, stuffing the size in the front and then returning an offset into the allocated memory. The free function could then simply use the offset of the provided pointer to discover the size to free.
For example
void* malloc(size_t size) {
size_t actualSize = size + sizeof(size_t);
void* buffer = _internal_allocate(actualSize);
*((size_t*)buffer) = size;
return ((size_t*)buffer) + 1;
}
void free(void* buffer) {
size_t* other = buffer;
other--;
size_t originalSize = *other;
// Rest of free
...
}
The answer is implementation-specific.
malloc might keep a dictionary mapping addresses to data records
malloc might allocate a slightly larger block than requested and store metadata before or after the block it actually returns.
In some special cases, not intended for general use, free() is completely a no-op and it doesn't actually keep track.
I know that it's a common convention to pass the length of dynamically allocated arrays to functions that manipulate them:
void initializeAndFree(int* anArray, size_t length);
int main(){
size_t arrayLength = 0;
scanf("%d", &arrayLength);
int* myArray = (int*)malloc(sizeof(int)*arrayLength);
initializeAndFree(myArray, arrayLength);
}
void initializeAndFree(int* anArray, size_t length){
int i = 0;
for (i = 0; i < length; i++) {
anArray[i] = 0;
}
free(anArray);
}
but if there's no way for me to get the length of the allocated memory from a pointer, how does free() "automagically" know what to deallocate when all I'm giving it is the very same pointer? Why can't I get in on the magic, as a C programmer?
Where does free() get its free (har-har) knowledge from?
Besides Klatchko's correct point that the standard does not provide for it, real malloc/free implementations often allocate more space then you ask for. E.g. if you ask for 12 bytes it may provide 16 (see A Memory Allocator, which notes that 16 is a common size). So it doesn't need to know you asked for 12 bytes, just that it gave you a 16-byte chunk.
You can't get it because the C committee did not require that in the standard.
If you are willing to write some non-portable code, you may have luck with:
*((size_t *)ptr - 1)
or maybe:
*((size_t *)ptr - 2)
But whether that works will depend on exactly where the implementation of malloc you are using stores that data.
After reading Klatchko's answer, I myself tried it and ptr[-1] indeed stores the actual memory (usually more than the memory we asked for probably to save against segmentation fault).
{
char *a = malloc(1);
printf("%u\n", ((size_t *)a)[-1]); //prints 17
free(a);
exit(0);
}
Trying with different sizes, GCC allocates the memory as follows:
Initially memory allocated is 17 bytes.
The allocated memory is atleast 5 bytes more than requested size, if more is requested, it allocates 8 bytes more.
If size is [0,12], memory allocated is 17.
If size is [13], memory allocated is 25.
If size is [20], memory allocated is 25.
If size is [21], memory allocated is 33.
While it is possible to get the meta-data that the memory allocator places preceding the allocated block, this would only work if the pointer is truly a pointer to a dynamically allocated block. This would seriously affect the utility of function requiring that all passed arguments were pointers to such blocks rather than say a simple auto or static array.
The point is there is no portable way from inspection of the pointer to know what type of memory it points to. So while it is an interesting idea, it is not a particularly safe proposition.
A method that is safe and portable would be to reserve the first word of the allocation to hold the length. GCC (and perhaps some other compilers) supports a non-portable method of implementing this using a structure with a zero length array which simplifies the code somewhat compared to a portable solution:
typedef struct
{
size_t length ;
char alloc[0] ; // Compiler specific extension!!!
} tSizedAlloc ;
// Allocating a sized block
tSizedAlloc* blk = malloc( sizeof(tSizedAlloc) + length ) ;
blk->length = length ;
// Accessing the size and data information of the block
size_t blk_length = blk->length ;
char* data = blk->alloc ;
I know this thread is a little old, but still I have something to say. There is a function (or a macro, I haven't checked the library yet) malloc_usable_size() - obtains size of block of memory allocated from heap. The man page states that it's only for debugging, since it outputs not the number you've asked but the number it has allocated, which is a little bigger. Notice it's a GNU extention.
On the other hand, it may not even be needed, because I believe that to free memory chunk you don't have to know its size. Just remove the handle/descriptor/structure that is in charge for the chunk.
A non-standard way is to use _msize(). Using this function will make your code unportable. Also the documentation is not very clear on wheteher it will return the number passed into malloc() or the real block size (might be greater).
It's up to the malloc implementor how to store this data. Most often, the length is stored directly in front of the allocated memory (that is, if you want to allocate 7 bytes, 7+x bytes are allocated in reality where the x additional bytes are used to store the metadata). Sometimes, the metadata is both stored before and after the allocated memory to check for heap corruptions. But the implementor can as well choose to use an extra data structure to store the metadata.
You can allocate more memory to store size:
void my_malloc(size_t n,size_t size )
{
void *p = malloc( (n * size) + sizeof(size_t) );
if( p == NULL ) return NULL;
*( (size_t*)p) = n;
return (char*)p + sizeof(size_t);
}
void my_free(void *p)
{
free( (char*)p - sizeof(size_t) );
}
void my_realloc(void *oldp,size_t new_size)
{
// ...
}
int main(void)
{
char *p = my_malloc( 20, 1 );
printf("%lu\n",(long int) ((size_t*)p)[-1] );
return 0;
}
To answer the question about delete[], early versions of C++ actually required that you call delete[n] and tell the runtime the size, so it didn't have to store it. Sadly, this behaviour was removed as "too confusing".
(See D&E for details.)
I know that it's a common convention to pass the length of dynamically allocated arrays to functions that manipulate them:
void initializeAndFree(int* anArray, size_t length);
int main(){
size_t arrayLength = 0;
scanf("%d", &arrayLength);
int* myArray = (int*)malloc(sizeof(int)*arrayLength);
initializeAndFree(myArray, arrayLength);
}
void initializeAndFree(int* anArray, size_t length){
int i = 0;
for (i = 0; i < length; i++) {
anArray[i] = 0;
}
free(anArray);
}
but if there's no way for me to get the length of the allocated memory from a pointer, how does free() "automagically" know what to deallocate when all I'm giving it is the very same pointer? Why can't I get in on the magic, as a C programmer?
Where does free() get its free (har-har) knowledge from?
Besides Klatchko's correct point that the standard does not provide for it, real malloc/free implementations often allocate more space then you ask for. E.g. if you ask for 12 bytes it may provide 16 (see A Memory Allocator, which notes that 16 is a common size). So it doesn't need to know you asked for 12 bytes, just that it gave you a 16-byte chunk.
You can't get it because the C committee did not require that in the standard.
If you are willing to write some non-portable code, you may have luck with:
*((size_t *)ptr - 1)
or maybe:
*((size_t *)ptr - 2)
But whether that works will depend on exactly where the implementation of malloc you are using stores that data.
After reading Klatchko's answer, I myself tried it and ptr[-1] indeed stores the actual memory (usually more than the memory we asked for probably to save against segmentation fault).
{
char *a = malloc(1);
printf("%u\n", ((size_t *)a)[-1]); //prints 17
free(a);
exit(0);
}
Trying with different sizes, GCC allocates the memory as follows:
Initially memory allocated is 17 bytes.
The allocated memory is atleast 5 bytes more than requested size, if more is requested, it allocates 8 bytes more.
If size is [0,12], memory allocated is 17.
If size is [13], memory allocated is 25.
If size is [20], memory allocated is 25.
If size is [21], memory allocated is 33.
While it is possible to get the meta-data that the memory allocator places preceding the allocated block, this would only work if the pointer is truly a pointer to a dynamically allocated block. This would seriously affect the utility of function requiring that all passed arguments were pointers to such blocks rather than say a simple auto or static array.
The point is there is no portable way from inspection of the pointer to know what type of memory it points to. So while it is an interesting idea, it is not a particularly safe proposition.
A method that is safe and portable would be to reserve the first word of the allocation to hold the length. GCC (and perhaps some other compilers) supports a non-portable method of implementing this using a structure with a zero length array which simplifies the code somewhat compared to a portable solution:
typedef struct
{
size_t length ;
char alloc[0] ; // Compiler specific extension!!!
} tSizedAlloc ;
// Allocating a sized block
tSizedAlloc* blk = malloc( sizeof(tSizedAlloc) + length ) ;
blk->length = length ;
// Accessing the size and data information of the block
size_t blk_length = blk->length ;
char* data = blk->alloc ;
I know this thread is a little old, but still I have something to say. There is a function (or a macro, I haven't checked the library yet) malloc_usable_size() - obtains size of block of memory allocated from heap. The man page states that it's only for debugging, since it outputs not the number you've asked but the number it has allocated, which is a little bigger. Notice it's a GNU extention.
On the other hand, it may not even be needed, because I believe that to free memory chunk you don't have to know its size. Just remove the handle/descriptor/structure that is in charge for the chunk.
A non-standard way is to use _msize(). Using this function will make your code unportable. Also the documentation is not very clear on wheteher it will return the number passed into malloc() or the real block size (might be greater).
It's up to the malloc implementor how to store this data. Most often, the length is stored directly in front of the allocated memory (that is, if you want to allocate 7 bytes, 7+x bytes are allocated in reality where the x additional bytes are used to store the metadata). Sometimes, the metadata is both stored before and after the allocated memory to check for heap corruptions. But the implementor can as well choose to use an extra data structure to store the metadata.
You can allocate more memory to store size:
void my_malloc(size_t n,size_t size )
{
void *p = malloc( (n * size) + sizeof(size_t) );
if( p == NULL ) return NULL;
*( (size_t*)p) = n;
return (char*)p + sizeof(size_t);
}
void my_free(void *p)
{
free( (char*)p - sizeof(size_t) );
}
void my_realloc(void *oldp,size_t new_size)
{
// ...
}
int main(void)
{
char *p = my_malloc( 20, 1 );
printf("%lu\n",(long int) ((size_t*)p)[-1] );
return 0;
}
To answer the question about delete[], early versions of C++ actually required that you call delete[n] and tell the runtime the size, so it didn't have to store it. Sadly, this behaviour was removed as "too confusing".
(See D&E for details.)