allocate struct and memory for elements in one malloc - c

I am sure this is a basic question but I haven't been able to find whether or not this is a legitimate memory allocation strategy or not. I am reading in data from a file and I am filling in a struct. The size of the members are variable on each read so my struct elements are pointers like so
struct data_channel{
char *chan_name;
char *chan_type;
char *chan_units;
};
So before reading I figure out what the size of each string is so I can allocate memory for them my question is can I allocate the memory for the struct and the strings all in one malloc and then fill the pointer in?
Say the size of chan_name is 9, chan_type 10, and chan_units 5. So I would allocate the and do something like this.
struct data_channel *chan;
chan = malloc(sizeof(struct data_channel) + 9 + 10 + 5);
chan->chan_name = chan[1];
chan->chan_type = chan->chan_name + 9;
chan->chan_units = chan->chan_type + 10;
So I read a couple of articles on memory alignment but I don't know if doing the above is a problem or not or what kind of unintended consequences it could have. I have already implemented it in my code and it seems to work fine. I just don't want to have to keep track of all those pointers because in reality each of my structs has 7 elements and I could have upwards of 100 channels. That of course means 700 pointers plus the pointers for each struct so total 800. The I also have to devise a way to free them all. I also want to apply this strategy to arrays of strings of which I then need to have an array of pointers to. I don't have any structures right now that would mix data types could that be a problem but I might could that be a problem?

If chan_name is a 8 character string, chan_type is a 9 character string and chan_units is a 4 character string, then yes it will work fine when you fix the compilation error you have when assigning to chan_name.
If you allocate enough memory for the structure plus all the strings (including their string terminator) then it's okay to use such a method. Maybe not recommended by all, but it will work.

It depends in part on the element types. You will certainly be able to do it with character strings; with some other types, you have to worry about alignment and padding issues.
struct data_channel
{
char *chan_name;
char *chan_type;
char *chan_units;
};
struct data_channel *chan;
size_t name_size = 9;
size_t type_size = 10;
size_t unit_size = 5;
chan = malloc(sizeof(struct data_channel) + name_size + type_size + unit_size);
if (chan != 0)
{
chan->chan_name = (char *)chan + sizeof(*chan);
chan->chan_type = chan->chan_name + name_size;
chan->chan_units = chan->chan_type + type_size;
}
This will work OK in practice — it was being done for ages before the standard was standardized. I can't immediately see why the standard would disallow this.
What gets trickier is if you needed to allocate an array of int, say, as well as two strings. Then you have to worry about alignment issues.
struct data_info
{
char *info_name;
int *info_freq;
char *info_unit;
};
size_t name_size = 9;
size_t freq_size = 10;
size_t unit_size = 5;
size_t nbytes = sizeof(struct data_info) + name_size + freq_size * sizeof(int) + unit_size;
struct data_info *info = malloc(nbytes);
if (info != 0)
{
info->info_freq = (int *)((char *)info + sizeof(*info));
info->info_name = (char *)info->info_freq + freq_size * sizeof(int);
info->info_unit = info->info_name + name_size;
}
This has adopted the simple expedient of allocating the most stringently aligned type (the array of int) first, then allocating the strings afterwards. This part is, however, where you have to make judgement calls about portability. I'm confident that the code is portable in practice.
C11 has alignment facilities (_Alignof and _Alignas and <stdalign.h>, plus max_align_t in <stddef.h>) that could alter this answer (but I've not studied them sufficiently so I'm not sure how, yet), but the techniques outlined here will work in any version of C provided you are careful about the alignment of data.
Note that if you have a single array in the structure, then C99 provides an alternative to the older 'struct hack' called a flexible array member (FAM). This allows you to have an array explicitly as the last element of the structure.
struct data_info
{
char *info_name;
char *info_units;
int info_freq[];
};
size_t name_size = 9;
size_t freq_size = 10;
size_t unit_size = 5;
size_t nbytes = sizeof(struct data_info) + name_size + freq_size * sizeof(int) + unit_size;
struct data_info *info = malloc(nbytes);
if (info != 0)
{
info->info_name = ((char *)info + sizeof(*info) + freq_size * sizeof(int));
info->info_units = info->info_name + name_size;
}
Note that there was no step to initialize the FAM, info_freq in this example. You cannot have multiple arrays like this.
Note that the techniques outlined cannot readily be applied to arrays of structures (at least, arrays of the outer structure). If you go to considerable effort, you can make it work. Also, beware of realloc(); if you reallocate space, you have to fix up the pointers if the data has moved.
One other point: especially on 64-bit machines, if the sizes of the strings are uniform enough, you'd probably do better allocating the arrays in the structure, instead of using the pointers.
struct data_channel
{
char chan_name[16];
char chan_type[16];
char chan_units[8];
};
This occupies 40 bytes. On a 64-bit machine, the original data structure would occupy 24 bytes for the three pointers and another 24 bytes for the (9 + 10 + 5) bytes of data, for a total of 48 bytes allocated.

I know there is a sure way to do this when you have ONE array at the end of a structure, but since all your arrays have the same type, you may be in luck. The sure method is:
#include <stddef.h>
#include <stdlib.h>
struct StWithArray
{
int blahblah;
float arr[1];
};
struct StWithArray * AllocWithArray(size_t nb)
{
size_t size = nb*sizeof(float) + offsetof(structStWithArray, arr);
return malloc(size);
}
The use of an actual array in the structure guarantees alignment is respected.
Now to apply it to your case:
#include <stddef.h>
#include <stdlib.h>
struct data_channel
{
char *chan_name;
char *chan_type;
char *chan_units;
char actualCharArray[1];
};
struct data_channel * AllocDataChannel(size_t nb)
{
size_t size = nb*sizeof(char) + offsetof(data_channel, actualCharArray);
return malloc(size);
}
struct data_channel * CreateDataChannel(size_t length1, size_t length2, size_t length3)
{
struct data_channel * pt = AllocDataChannel(length1 + length2 + length3);
if(pt != NULL)
{
pt->chan_name = &pt->actualCharArray[0];
pt->chan_type = &pt->actualCharArray[length1];
pt->chan_name = &pt->actualCharArray[length1+length2];
}
return pt;
}

Joachim and Jonathan's answers are nice. Only addition I would like to mention is this.
Separate mallocs and frees buy you some basic protection like buffer overrun, access after
free, etc. I mean basic and not Valgrind like features. Allocating one single chunk and internally doling it out will lead to a loss of this feature.
In future, if the mallocs are for different sizes totally, then separate mallocs may buy you the efficiency of coming from different allocation buckets inside of the malloc implementation, especially if you are going to free them at different times.
The last thing you have to consider is how frequently you are calling mallocs. If it is frequent, then cost of multiple mallocs can be costly.

Related

Multiple structures in a single malloc invoking undefined behaviour?

From Use the correct syntax when declaring a flexible array member it says that when malloc is used for a header and flexible data when data[1] is hacked into the struct,
This example has undefined behavior when accessing any element other
than the first element of the data array. (See the C Standard, 6.5.6.)
Consequently, the compiler can generate code that does not return the
expected value when accessing the second element of data.
I looked up the C Standard 6.5.6, and could not see how this would produce undefined behaviour. I've used a pattern that I'm comfortable with, where the header is implicitly followed by data, using the same sort of malloc,
#include <stdlib.h> /* EXIT malloc free */
#include <stdio.h> /* printf */
#include <string.h> /* strlen memcpy */
struct Array {
size_t length;
char *array;
}; /* +(length + 1) char */
static struct Array *Array(const char *const str) {
struct Array *a;
size_t length;
length = strlen(str);
if(!(a = malloc(sizeof *a + length + 1))) return 0;
a->length = length;
a->array = (char *)(a + 1); /* UB? */
memcpy(a->array, str, length + 1);
return a;
}
/* Take a char off the end just so that it's useful. */
static void Array_to_string(const struct Array *const a, char (*const s)[12]) {
const int n = a->length ? a->length > 9 ? 9 : (int)a->length - 1 : 0;
sprintf(*s, "<%.*s>", n, a->array);
}
int main(void) {
struct Array *a = 0, *b = 0;
int is_done = 0;
do { /* Try. */
char s[12], t[12];
if(!(a = Array("Foo!")) || !(b = Array("To be or not to be."))) break;
Array_to_string(a, &s);
Array_to_string(b, &t);
printf("%s %s\n", s, t);
is_done = 1;
} while(0); if(!is_done) {
perror(":(");
} {
free(a);
free(b);
}
return is_done ? EXIT_SUCCESS : EXIT_FAILURE;
}
Prints,
<Foo> <To be or >
The compliant solution uses C99 flexible array members. The page also says,
Failing to use the correct syntax when declaring a flexible array
member can result in undefined behavior, although the incorrect syntax
will work on most implementations.
Technically, does this C90 code produce undefined behaviour, too? And if not, what is the difference? (Or the Carnegie Mellon Wiki is incorrect?) What is the factor on the implementations this will not work on?
This should be well defined:
a->array = (char *)(a + 1);
Because you create a pointer to one element past the end of an array of size 1 but do not dereference it. And because a->array now points to bytes that do not yet have an effective type, you can use them safely.
This only works however because you're using the bytes that follow as an array of char. If you instead tried to create an array of some other type whose size is greater than 1, you could have alignment issues.
For example, if you compiled a program for ARM with 32 bit pointers and you had this:
struct Array {
int size;
uint64_t *a;
};
...
Array a = malloc(sizeof *a + (length * sizeof(uint64_t)));
a->length = length;
a->a= (uint64_t *)(a + 1); // misaligned pointer
a->a[0] = 0x1111222233334444ULL; // misaligned write
Your program would crash due to a misaligned write. So in general you shouldn't depend on this. Best to stick with a flexible array member which the standard guarantees will work.
As an adjunct to #dbush good answer, a way to get around alignment woes is to use a union. This insures &p[1] is properly aligned for (uint64_t*)1. sizeof *p includes any needed padding vs. sizeof *a.
union {
struct Array header;
uint64_t dummy;
} *p;
p = malloc(sizeof *p + length*sizeof p->header->array);
struct Array *a = (struct Array *)&p[0]; // or = &(p->header);
a->length = length;
a->array = (uint64_t*) &p[1]; // or &p[1].dummy;
Or go with C99 and flexible array member.
1 As well as struct Array
Before the publication of C89, there were some implementations that would attempt to identify and trap upon out-of-bounds array accesses. Given something like:
struct foo {int a[4],b[4];} *p;
such implementations would squawk at an effort to access p->a[i] if i wasn't in the range 0 to 3. For programs that don't need to index the address of array-type lvalue p->a to access anything outside that array, being able to trap on such out-of-bounds accesses would be useful.
The authors of C89 were also almost certainly aware that it was common for programs to use the address of dummy-sized array at the end of a structure as a means of accessing storage beyond the structure. Using such techniques made it possible to do things that couldn't be done nearly as nicely otherwise, and part of the Spirit of C, according to the authors of the Standard, is "Don't prevent the programmer from doing what needs to be done".
Consequently, the authors of the Standard treated such accesses as something which implementations could support or not, at their leisure, presumably based upon what would be most useful for their customers. While it would often be helpful for implementations which would normally bounds-check accesses to structures in an array, to provide an option to omit such checks in cases where the last item of an indirectly-accessed structure is an array with one element (or, if they extend the language to waive a compile-time constraint, zero elements), people writing such implementations would presumably be capable of recognizing such things without the authors of the Standard having to tell them. The notion that "Undefined Behavior" was intended as some form of prohibition doesn't seem to have really taken hold until after the publication of C89's successor standard.
With regard to your example, having a pointer within a struct point to later storage in the same allocation should work, but with a couple of caveats:
If the allocation is passed to realloc, the pointer within it will become invalid.
The only real advantage of using a pointer versus a flexible array member is that it allows for the possibility of having it point somewhere else. That may be good if the only kind of "something else" will always be an constant object of static duration that never has to be freed, or perhaps if it is some other kind of object that won't have to be freed, but may be problematical if it could hold the only reference to something stored in a separate allocation.
Flexible array members have been available as an extension in some compilers before C89 was written, and were officially added in C99. Any decent compiler should support them.
You can define struct Array as:
struct Array
{
size_t length;
char array[1];
}; /* +(length + 1) char */
then malloc( sizeof *a + length ). The "+1" element is in array[1] member. Fill structure with:
a->length = length;
strcpy( a->array, str );

Allocate Pointer and pointee at once

If I want to reduce malloc()s (espacially if the data is small and allocated often) I would like to allocate the pointer and pointee at once.
If you assume something like the following:
struct entry {
size_t buf_len;
char *buf;
int something;
};
I would like to allocate memory in the following way (don't care about error checking here):
size_t buf_len = 4; // size of the buffer
struct entry *e = NULL;
e = malloc( sizeof(*e) + buf_len ); // allocate struct and buffer
e->buf_len = buf_len; // set buffer size
e->buf = e + 1; // the buffer lies behind the struct
This could even be extende, so that a whole array is allocated at once.
How would you assess such a technuique with regard to:
Portability
Maintainability / Extendability
Performance
Readability
Is this reasonable? If it is ok to use, are there any ideas on how to design a possible interface for that?
You could use a flexible array member instead of a pointer:
struct entry {
size_t buf_len;
int something;
char buf[];
};
// ...
struct entry *e = malloc(sizeof *e + buf_len);
e->buf_len = buf_len;
Portability and performance are fine. Readability: not perfect but good enough.
Extendability: you can't use this for more than one member at a time, you'd have to fall back to your explicit pointer version. Also, the explicit pointer version means that you have to muck around to ensure correct alignment if you use it with a type that doesn't have an alignment of 1.
If you are seriously thinking about this I'd consider revisiting your entire data structure's design to see if there is another way of doing it. (Maybe this way is actually the best way, but have a good think about it first).
As to portability, I am unaware of any issues, as long as the sizes are found via suitable calls to sizeof(), as in your code.
Regarding maintainability, extendability and readability, you should certainly wrap allocation and de-allocation in a well-commented function. Calls to...
entry *allocate_entry_with_buffer();
void deallocate_entry_with_buffer(entry **entry_with_buffer);
...do not need to know implementation details of how the memory actually gets handled. People use stranger things like custom allocators and memory pools quite frequently.
As for speed, this is certainly faster than making lots of small allocations. I used to allocate whole 2D matrices with a similar strategy...
It should work, but in fact you are using a pointer for a useless indirection. Windows API (for example) uses another method for variable size structs : the variable size buffer is last in struct and is declared to be char buf[1].
Your struct would become :
struct entry {
size_t buf_len;
int something;
char buf[1];
};
The allocation is (still no error checking) :
size_t buf_len = 4; // size of the buffer
struct entry *e;
e = malloc( sizeof(*e) + buf_len - 1); // struct already has room for 1 char
e->buf_len = buf_len; // set buffer size
That's all e.buf is guaranteed to be a char array of size buf_len.
That way ensures that even if the variable part was not a character array but a int, long, or anything array, the alignement would be given by the last element being a array of proper type and size 1.
For starters, the line:
e->buf = e + sizeof(*e); // the buffer lies behind the struct
Should be:
e->buf = e + 1; // the buffer lies behind the struct
This is because e + 1 will be equal to the address at the end of the structure. As you have it, it will only be the number of bytes into the structure equal to the number of bytes in a pointer.
And, yes, it's reasonable. However, I prefer this approach:
struct entry {
size_t buf_len;
int something;
char buf[1];
};
This way, you don't mess with the pointers. Just append as many bytes as needed, and they will grow the size of your buf array.
Note: I wrote a text editor using an approach similar to this but used a Microsoft c++ extension that allowed me to declare the last member as char buf[]. So it was an empty array that was exactly as long as the number of extra bytes I allocated.
seems fine to me - put comments in though
Or you could do this - which is quite common
struct entry {
size_t buf_len;
int something;
char buf;
};
ie make the struct itself variable length. and do
size_t buf_len = 4; // size of the buffer
struct entry *e = NULL;
// check that it packs right
e = malloc(sizeof(size_t) + sizeof(int) + buf_len ); // allocate struct and buffer
e->buf_len = buf_len; // set buffer size
...... later
printf(&e.buf);

Get the length of an array with a pointer? [duplicate]

I've allocated an "array" of mystruct of size n like this:
if (NULL == (p = calloc(sizeof(struct mystruct) * n,1))) {
/* handle error */
}
Later on, I only have access to p, and no longer have n. Is there a way to determine the length of the array given just the pointer p?
I figure it must be possible, since free(p) does just that. I know malloc() keeps track of how much memory it has allocated, and that's why it knows the length; perhaps there is a way to query for this information? Something like...
int length = askMallocLibraryHowMuchMemoryWasAlloced(p) / sizeof(mystruct)
I know I should just rework the code so that I know n, but I'd rather not if possible. Any ideas?
No, there is no way to get this information without depending strongly on the implementation details of malloc. In particular, malloc may allocate more bytes than you request (e.g. for efficiency in a particular memory architecture). It would be much better to redesign your code so that you keep track of n explicitly. The alternative is at least as much redesign and a much more dangerous approach (given that it's non-standard, abuses the semantics of pointers, and will be a maintenance nightmare for those that come after you): store the lengthn at the malloc'd address, followed by the array. Allocation would then be:
void *p = calloc(sizeof(struct mystruct) * n + sizeof(unsigned long int),1));
*((unsigned long int*)p) = n;
n is now stored at *((unsigned long int*)p) and the start of your array is now
void *arr = p+sizeof(unsigned long int);
Edit: Just to play devil's advocate... I know that these "solutions" all require redesigns, but let's play it out.
Of course, the solution presented above is just a hacky implementation of a (well-packed) struct. You might as well define:
typedef struct {
unsigned int n;
void *arr;
} arrInfo;
and pass around arrInfos rather than raw pointers.
Now we're cooking. But as long as you're redesigning, why stop here? What you really want is an abstract data type (ADT). Any introductory text for an algorithms and data structures class would do it. An ADT defines the public interface of a data type but hides the implementation of that data type. Thus, publicly an ADT for an array might look like
typedef void* arrayInfo;
(arrayInfo)newArrayInfo(unsignd int n, unsigned int itemSize);
(void)deleteArrayInfo(arrayInfo);
(unsigned int)arrayLength(arrayInfo);
(void*)arrayPtr(arrayInfo);
...
In other words, an ADT is a form of data and behavior encapsulation... in other words, it's about as close as you can get to Object-Oriented Programming using straight C. Unless you're stuck on a platform that doesn't have a C++ compiler, you might as well go whole hog and just use an STL std::vector.
There, we've taken a simple question about C and ended up at C++. God help us all.
keep track of the array size yourself; free uses the malloc chain to free the block that was allocated, which does not necessarily have the same size as the array you requested
Just to confirm the previous answers: There is no way to know, just by studying a pointer, how much memory was allocated by a malloc which returned this pointer.
What if it worked?
One example of why this is not possible. Let's imagine the code with an hypothetic function called get_size(void *) which returns the memory allocated for a pointer:
typedef struct MyStructTag
{ /* etc. */ } MyStruct ;
void doSomething(MyStruct * p)
{
/* well... extract the memory allocated? */
size_t i = get_size(p) ;
initializeMyStructArray(p, i) ;
}
void doSomethingElse()
{
MyStruct * s = malloc(sizeof(MyStruct) * 10) ; /* Allocate 10 items */
doSomething(s) ;
}
Why even if it worked, it would not work anyway?
But the problem of this approach is that, in C, you can play with pointer arithmetics. Let's rewrite doSomethingElse():
void doSomethingElse()
{
MyStruct * s = malloc(sizeof(MyStruct) * 10) ; /* Allocate 10 items */
MyStruct * s2 = s + 5 ; /* s2 points to the 5th item */
doSomething(s2) ; /* Oops */
}
How get_size is supposed to work, as you sent the function a valid pointer, but not the one returned by malloc. And even if get_size went through all the trouble to find the size (i.e. in an inefficient way), it would return, in this case, a value that would be wrong in your context.
Conclusion
There are always ways to avoid this problem, and in C, you can always write your own allocator, but again, it is perhaps too much trouble when all you need is to remember how much memory was allocated.
Some compilers provide msize() or similar functions (_msize() etc), that let you do exactly that
May I recommend a terrible way to do it?
Allocate all your arrays as follows:
void *blockOfMem = malloc(sizeof(mystruct)*n + sizeof(int));
((int *)blockofMem)[0] = n;
mystruct *structs = (mystruct *)(((int *)blockOfMem) + 1);
Then you can always cast your arrays to int * and access the -1st element.
Be sure to free that pointer, and not the array pointer itself!
Also, this will likely cause terrible bugs that will leave you tearing your hair out. Maybe you can wrap the alloc funcs in API calls or something.
malloc will return a block of memory at least as big as you requested, but possibly bigger. So even if you could query the block size, this would not reliably give you your array size. So you'll just have to modify your code to keep track of it yourself.
For an array of pointers you can use a NULL-terminated array. The length can then determinate like it is done with strings. In your example you can maybe use an structure attribute to mark then end. Of course that depends if there is a member that cannot be NULL. So lets say you have an attribute name, that needs to be set for every struct in your array you can then query the size by:
int size;
struct mystruct *cur;
for (cur = myarray; cur->name != NULL; cur++)
;
size = cur - myarray;
Btw it should be calloc(n, sizeof(struct mystruct)) in your example.
Other have discussed the limits of plain c pointers and the stdlib.h implementations of malloc(). Some implementations provide extensions which return the allocated block size which may be larger than the requested size.
If you must have this behavior you can use or write a specialized memory allocator. This simplest thing to do would be implementing a wrapper around the stdlib.h functions. Some thing like:
void* my_malloc(size_t s); /* Calls malloc(s), and if successful stores
(p,s) in a list of handled blocks */
void my_free(void* p); /* Removes list entry and calls free(p) */
size_t my_block_size(void* p); /* Looks up p, and returns the stored size */
...
really your question is - "can I find out the size of a malloc'd (or calloc'd) data block". And as others have said: no, not in a standard way.
However there are custom malloc implementations that do it - for example http://dmalloc.com/
I'm not aware of a way, but I would imagine it would deal with mucking around in malloc's internals which is generally a very, very bad idea.
Why is it that you can't store the size of memory you allocated?
EDIT: If you know that you should rework the code so you know n, well, do it. Yes it might be quick and easy to try to poll malloc but knowing n for sure would minimize confusion and strengthen the design.
One of the reasons that you can't ask the malloc library how big a block is, is that the allocator will usually round up the size of your request to meet some minimum granularity requirement (for example, 16 bytes). So if you ask for 5 bytes, you'll get a block of size 16 back. If you were to take 16 and divide by 5, you would get three elements when you really only allocated one. It would take extra space for the malloc library to keep track of how many bytes you asked for in the first place, so it's best for you to keep track of that yourself.
This is a test of my sort routine. It sets up 7 variables to hold float values, then assigns them to an array, which is used to find the max value.
The magic is in the call to myMax:
float mmax = myMax((float *)&arr,(int) sizeof(arr)/sizeof(arr[0]));
And that was magical, wasn't it?
myMax expects a float array pointer (float *) so I use &arr to get the address of the array, and cast it as a float pointer.
myMax also expects the number of elements in the array as an int. I get that value by using sizeof() to give me byte sizes of the array and the first element of the array, then divide the total bytes by the number of bytes in each element. (we should not guess or hard code the size of an int because it's 2 bytes on some system and 4 on some like my OS X Mac, and could be something else on others).
NOTE:All this is important when your data may have a varying number of samples.
Here's the test code:
#include <stdio.h>
float a, b, c, d, e, f, g;
float myMax(float *apa,int soa){
int i;
float max = apa[0];
for(i=0; i< soa; i++){
if (apa[i]>max){max=apa[i];}
printf("on i=%d val is %0.2f max is %0.2f, soa=%d\n",i,apa[i],max,soa);
}
return max;
}
int main(void)
{
a = 2.0;
b = 1.0;
c = 4.0;
d = 3.0;
e = 7.0;
f = 9.0;
g = 5.0;
float arr[] = {a,b,c,d,e,f,g};
float mmax = myMax((float *)&arr,(int) sizeof(arr)/sizeof(arr[0]));
printf("mmax = %0.2f\n",mmax);
return 0;
}
In uClibc, there is a MALLOC_SIZE macro in malloc.h:
/* The size of a malloc allocation is stored in a size_t word
MALLOC_HEADER_SIZE bytes prior to the start address of the allocation:
+--------+---------+-------------------+
| SIZE |(unused) | allocation ... |
+--------+---------+-------------------+
^ BASE ^ ADDR
^ ADDR - MALLOC_HEADER_SIZE
*/
/* The amount of extra space used by the malloc header. */
#define MALLOC_HEADER_SIZE \
(MALLOC_ALIGNMENT < sizeof (size_t) \
? sizeof (size_t) \
: MALLOC_ALIGNMENT)
/* Set up the malloc header, and return the user address of a malloc block. */
#define MALLOC_SETUP(base, size) \
(MALLOC_SET_SIZE (base, size), (void *)((char *)base + MALLOC_HEADER_SIZE))
/* Set the size of a malloc allocation, given the base address. */
#define MALLOC_SET_SIZE(base, size) (*(size_t *)(base) = (size))
/* Return base-address of a malloc allocation, given the user address. */
#define MALLOC_BASE(addr) ((void *)((char *)addr - MALLOC_HEADER_SIZE))
/* Return the size of a malloc allocation, given the user address. */
#define MALLOC_SIZE(addr) (*(size_t *)MALLOC_BASE(addr))
malloc() stores metadata regarding space allocation before 8 bytes from space actually allocated. This could be used to determine space of buffer. And on my x86-64 this always return multiple of 16. So if allocated space is multiple of 16 (which is in most cases) then this could be used:
Code
#include <stdio.h>
#include <malloc.h>
int size_of_buff(void *buff) {
return ( *( ( int * ) buff - 2 ) - 17 ); // 32 bit system: ( *( ( int * ) buff - 1 ) - 17 )
}
void main() {
char *buff = malloc(1024);
printf("Size of Buffer: %d\n", size_of_buff(buff));
}
Output
Size of Buffer: 1024
This is my approach:
#include <stdio.h>
#include <stdlib.h>
typedef struct _int_array
{
int *number;
int size;
} int_array;
int int_array_append(int_array *a, int n)
{
static char c = 0;
if(!c)
{
a->number = NULL;
a->size = 0;
c++;
}
int *more_numbers = NULL;
a->size++;
more_numbers = (int *)realloc(a->number, a->size * sizeof(int));
if(more_numbers != NULL)
{
a->number = more_numbers;
a->number[a->size - 1] = n;
}
else
{
free(a->number);
printf("Error (re)allocating memory.\n");
return 1;
}
return 0;
}
int main()
{
int_array a;
int_array_append(&a, 10);
int_array_append(&a, 20);
int_array_append(&a, 30);
int_array_append(&a, 40);
int i;
for(i = 0; i < a.size; i++)
printf("%d\n", a.number[i]);
printf("\nLen: %d\nSize: %d\n", a.size, a.size * sizeof(int));
free(a.number);
return 0;
}
Output:
10
20
30
40
Len: 4
Size: 16
If your compiler supports VLA (variable length array), you can embed the array length into the pointer type.
int n = 10;
int (*p)[n] = malloc(n * sizeof(int));
n = 3;
printf("%d\n", sizeof(*p)/sizeof(**p));
The output is 10.
You could also choose to embed the information into the allocated memory yourself with a structure including a flexible array member.
struct myarray {
int n;
struct mystruct a[];
};
struct myarray *ma =
malloc(sizeof(*ma) + n * sizeof(struct mystruct));
ma->n = n;
struct mystruct *p = ma->a;
Then to recover the size, you would subtract the offset of the flexible member.
int get_size (struct mystruct *p) {
struct myarray *ma;
char *x = (char *)p;
ma = (void *)(x - offsetof(struct myarray, a));
return ma->n;
}
The problem with trying to peek into heap structures is that the layout might change from platform to platform or from release to release, and so the information may not be reliably obtainable.
Even if you knew exactly how to peek into the meta information maintained by your allocator, the information stored there may have nothing to do with the size of the array. The allocator simply returned memory that could be used to fit the requested size, but the actual size of the memory may be larger (perhaps even much larger) than the requested amount.
The only reliable way to know the information is to find a way to track it yourself.

Getting Size of Struct That Contains Dynamically Allocated Memory

I would like to allocate memory for a buffer that will contain, via memcpy in the future, a struct that contains a pointer that has been previously dynamically allocated memory.
That is, I have a struct
struct test_struct {
int num;
char *values;
};
Where test_struct.values contains num amount of strings of length LENGTH. I know I can't get the size of memory a pointer has been allocated, so I just keep track of it via num. What is the easiest/cleanest way of getting the size of this struct?
The only solution I can come up with is something like
buf = malloc(sizeof(test_struct) + (num * LENGTH));
But I'm new to this low-level memory management stuff, so there might be something better.
If you would like to memcpy two structs then the memory in both of them must be continuous. But you would have to determine num beforehand.
struct test_struct {
int num;
char ** values;
} * TestStruct;
int _num = 0;
// find _num
TestStruct = malloc (sizeof (struct test_struct) + (sizeof(char*) * _num) + (LENGTH * _num));
TestStruct->num = _num;
TestStruct->values = &TestStruct + sizeof (struct test_struct);
for (int i = 0; i < _num; i++){
TestStruct->values[i] = &TestStruct + sizeof (struct test_struct) + (i * LENGTH);
}
The reason I changed char * to char ** is because using char * it becomes harder to access the strings after the first (I'm assuming they're null terminated). Also, after calling memcpy, you must update all the string pointers in the new struct.
To memcpy you would do this:
memcpy (buf, TestStruct->values[0], LENGTH * TestStruct->num);
But in buf, however, you would only see the first string (unless your strings are not null-terminated). You would have to increment the pointer after every null terminated character until you know, with num, that you've reached the end of the buffer.
Now that I understand more of the context of your request, consider the following.
If you're using UDP packets, you should send the data in one packet so that it arrives in the order you expect. When more than one packet is sent, it may arrive out of order. Because of this, you need to make sure the size of the data is <= 512 bytes - which is the maximum size of a UDP packet. Also, you need to make sure all the data is in contiguous memory. I'm going to assume you have your data already in the struct you've provided in this example:
// this function puts the struct in contiguous memory
int PrepareBuffer (struct test_struct TestStruct, char ** buffer){
char * cast = (char *) &TestStruct->num;
* buffer = malloc ((TestStruct->num * LENGTH) + sizeof (int));
for (int i = 0; i < sizeof (int); i++) *buffer[i] = cast[i];
for (int i = 0; i < (TestStruct->num * LENGTH); i++) *buffer[i + sizeof (int)] = TestStruct->values[i];
return 0;
}
You will have to implement another function on the receiving end that maps the buffer to struct test_struct. Also, I have omitted error checking for clarity. You should check for how big the packet is going to be before to allocate memory (it has to be <= 512). You should also check to make sure malloc returns a none-null pointer.
You should only need to allocate 4 bytes (for the integer on 32 bit linux) and 4 bytes for the char * (in 32 bit. 64 is 8).
What you're really asking though, is how do I know how much memory I need to allocate to the region pointed to by char *value. You figure this out in the wya you're doing. Then set value to the location of buf. There's a comment blow me that is the correct way if you have multiple string, and you don't want to just jam them all together in that region and have to figure out which is which yourself.
I'm assuming that you want to allocate memory for both the structure and the buffer that values points to. If so, this is correct. To point at the extra space, do buf->values = buf + 1; (this is assuming you declare buf as struct test_struct buf;

Declare array in struct

I am trying to make a struct for a circular buffer that contains an array of type "quote." However, the quote array must start out at a size of 10. I am trying to figure out if I declare the size of 10 in my .h file or in my .c file. My two files are as follows:
.h file:
typedef struct{
unsigned int time;
double rate;
}quote;
typedef struct{
unsigned int testNum;
quote quoteBuffer[];
}cbuf;
cbuf* cbuf_init();
.c file:
cbuf* cbuf_init(){
cbuf *buffer = (cbuf *)calloc(1,sizeof(cbuf));
buffer->testNum = 1;
quote newQuote = {1,1.00};
buffer->quoteBuffer[1] = newQuote;
return buffer;
}
These are obviously just test values, however if I wanted to specifically make the quote array in the cbuf struct start out at a size of 10, would I declare that in the .h file as:
typedef struct{
unsigned int testNum;
quote quoteBuffer[10];
}cbuf;
or in the .c file some other way?
There are two ways of having dynamic arrays in structures. The obvious is of course to have it as a pointer, and dynamically allocate (or reallocate) when needed.
The other is to have an array of size 1, and then allocate a larger size than the structure, to accommodate for the array:
typedef struct {
unsigned int testNum;
quote quoteBuffer[1];
} cbuf;
cbuf *cbuf_init(const size_t num_quotes) {
/* Allocate for the `cbuf` structure, plus a number of `quote`
* structures in the array
*/
cbuf *buffer = malloc(sizeof(cbuf) + (num_quotes - 1) * sizeof(quote));
/* Other initialization */
return buffer;
}
/* If more quotes are needed after initial allocation, use this function */
cbuf *cbuf_realloc(cbuf *buffer, const size_t new_num_quotes) {
buffer = realloc(buffer, sizeof(cbuf) + (new_num_quotes - 1) * sizeof(quote));
/* Other initialization */
return buffer;
}
Now you can use the array as a normal array:
cbuf *buffer = cbuf_init();
buffer->quoteBuffer[5].time = 123;
Note: I only allocate extra space for 9 quote structures, but state that I allocate ten. The reason is that the cbuf structure already contains one quote structure in its array. 1 + 9 = 10. :)
Note 2: I put the quote array in the cbuf structure with one entry already in it for backwards compatibility. Having an array without a size in the structure is quite new in the C world.
you can also do this if you want 10 quotes in a cbuf but a statically allocated like quote buffer[10] would work too:
cbuf* cbuf_init(int numQuotes)
{
cbuf *b = calloc(1, sizeof(cbuf) + numQuotes * sizeof(quote));
return b;
}
If you want a statically sized circular buffer then your could declare the size in the header file. Using a #define for the buffer size will make the code more readable and maintainable, as you'll reference the size elsewhere in your code.
If you want the circular buffer to be growable then define the size in your C file. You'll then have to take care of tracking the size and destroying the memory that you will have to allocate dynamically.
In your example, I think you need to allocate more room for your quote structs...
cbuf *buffer = (cbuf *)calloc(1,sizeof(cbuf) + NUM_QUOTES*sizeof(struct quote));
---------------------------------
The reason for this is that in your struct def...
quote quoteBuffer[];
... quoteBuffer doesn't add size to the struct. quoteBuffer will point to one byte past the end of the struct, hence the need to allocate memory for the struct + memory for the array.
EDIT: Daniel Fischer's comment (thanks Daniel) - quoteBuffer may, in some cases, add size to the struct if it introduces padding. The reason is that the compiler will probably strive to get the most optimal alignment for quoteBuffer. For example, ints normally aligned of 4-byte boundaries. E.g. a struct like:
struct {
char a;
int b;
}
is probably changed by compiler to
struct {
char a;
char pad[3]; // compiler adds padding behind the scenes
int b; // align b on a 4-byte boundary
}
This probs doesn't apply in your case as your struct leaves quoteBuffer[] on a 4 byte boundary.
The reason that the compiler does this is two fold.
1. On some architectures (not so common nowadays I think?), unaligned accesses aren't supported.
2. Aligned accesses are more efficient, even if architecture allows non-aligned accesses as it is one memory read as opposed to two memory reads plus a manipulation.

Resources