Buffer Array Usage in C - c

My mind tells me there is something wrong here, like maybe I won't be able to append a value to the last element in the array, or it will exceed the UINT8 when doing the cbuf_length++ and crash? My mind thinks something is wrong, but i just can't see what it is. What is the best way to use arrays that will allow me to append to all 256 elements in the array without crashing or confusing the next programmer?
#define CBUF_SIZE 256
static UINT8 cbuf[CBUF_SIZE];
static UINT8 cbuf_length;
void CBUF_AppendChar(char c)
{
if (cbuf_length < CBUF_SIZE) {
cbuf[cbuf_length] = c;
cbuf_length++;
}
}

First, there's some issues with your code.
UINT8 is not a standard type, I suspect it's a Microsoft type. Unless you have a good reason, use uint8_t from stdint.h.
Second, cbuf_length < CBUF_SIZE is always true because uint8_t can't store higher than 255.
test.c:10:21: warning: comparison of constant 256 with expression of type 'uint8_t'
(aka 'unsigned char') is always true [-Wtautological-constant-out-of-range-compare]
if (cbuf_length < CBUF_SIZE) {
~~~~~~~~~~~ ^ ~~~~~~~~~
More important, global variables and constants should be avoided. They create complexity, limit you to a single instance, cause problems with concurrency. It's not always clear which global variables must be updated together. It's also not immediately clear what functions affect what data.
Instead, consider using a struct.
For array sizes use size_t to avoid exactly the sort of bug you have where the size of the array is larger than the capacity of your type. size_t is the type for storing, you guessed it, sizes! It will always be large enough.
Finally, don't forget error handling. Like when your buffer can hold no more elements. In this case I have the append function return a boolean which can be checked to see if the append succeeded. (I've been sloppy in a few other places, like not checking malloc).
#include <stdio.h>
#include <stdint.h>
#include <stdbool.h>
#include <stdlib.h>
typedef struct {
uint8_t *buf;
size_t size;
size_t capacity;
} cbuf;
cbuf *cbuf_new( size_t capacity ) {
cbuf *self = malloc( sizeof(cbuf) );
self->size = 0;
self->capacity = capacity;
self->buf = malloc( capacity * sizeof(uint8_t) );
return self;
}
bool cbuf_append( cbuf *self, char c) {
// Get the error check out of the way early.
if( self->size >= self->capacity ) {
return false;
}
// Don't forget to cast signed char to uint8_t.
self->buf[self->size] = (uint8_t)c;
self->size++;
return true;
}
int main(void) {
cbuf *buf = cbuf_new(2);
cbuf_append(buf, 'f'); // true
cbuf_append(buf, 'o'); // true
cbuf_append(buf, 'o'); // false
// Prints 'fo'
for( size_t i = 0; i < buf->size; i++ ) {
printf("%c", buf->buf[i]);
}
puts("");
}
All the information about the buffer is collected together in a single struct. Now it's immediately clear that cbuf_append works on buf. You can have multiple buffers if you wish.
If you really want a statically allocated buffer, a slight modification gives you optional control of the memory allocation.
cbuf *cbuf_new_from_buf( uint8_t *buf, size_t capacity ) {
cbuf *self = malloc( sizeof(cbuf) );
self->size = 0;
self->capacity = capacity;
self->buf = buf;
bzero( self->buf, self->capacity );
return self;
}
cbuf *cbuf_new( size_t capacity ) {
return cbuf_new_from_buf(
malloc( capacity * sizeof(uint8_t) ),
capacity
);
}
Now you can still use cbuf_new() to let it take care of memory allocation for you. Or you can use cbuf_new_from_buf() to use an existing buffer you've already allocated.
uint8_t mybuf[2];
cbuf *buf = cbuf_new_from_buf(mybuf, sizeof(mybuf));
But you shouldn't be writing your own buffers!
This is all fine for a learning exercise. But in production if all you want is a sized array you can append to, don't write a bunch of code that you'll (or somebody else) will have to document and maintain and test and fix and add features to. Use an existing library. Like GLib's GArray type. It does what you want and more.
#include <glib.h>
#include <stdio.h>
#include <stdint.h>
int main(void) {
// Start with a size 2 buffer, it will grow as needed.
GArray *buf = g_array_sized_new(FALSE, FALSE, sizeof(uint8_t), 2);
uint8_t c = 'f';
g_array_append_val(buf, c);
c = 'o';
g_array_append_val(buf, c);
c = 'o';
g_array_append_val(buf, c); // It will grow the buffer.
for( guint i = 0; i < buf->len; i++ ) {
printf("%c", g_array_index(buf, uint8_t, i));
}
puts("");
}
While GArray has some quirks, like not being able to append literals, it's well documented, powerful, and fast. That means you can spend your time on the real code, not basic data structures.
Even better, because the array will grow if necessary there's no chance of a buffer overflow and no need to check if the append succeeded. Your original concern about the buffer size is eliminated, the buffer will just grow as necessary.

The only mistake is in the type of cbuf_length — after 255, it will overflow to 0. Defining it as a larger type (UINT16 or even just unsigned) will fix it.
Other than that, it's perfectly valid and understandable code. You might want to return something from that function, though — some boolean result telling the caller whether the value was appended successfully or not.

Related

Convert pointer to 2D char array in C

This probably has been asked already, but I'm unable to find anything on it.
I have a string array, where the numbers of strings in it is determined at runtime (the max string length is known, if that helps). Since I need global access to that array, I used a pointer and malloc'ed enough space to it when I actually know how much has to fit in there:
char *global_strings;
void some_func(int strings_nr, int strings_size)
{
global_strings = (char*) malloc(strings_nr* strings_size* sizeof(char));
}
What would be the correct way in C to use this pointer like a two-dimensional char array equivalent to
global_strings[strings_nr][strings_size] ?
As a global pointer to 2D data, whose N*M characteristics defined at run-time, I'd recommend a helper function to access the strings rather than directly use it. Make it inline or as a macro if desired.
char *global_strings = NULL;
size_t global_strings_nr = 0;
size_t global_strings_size = 0;
// Allocation -
// OK to call again, but prior data may not be organized well with a new string_size
// More code needed to handle that.
void some_func(int strings_nr, int strings_size) {
global_strings_nr = strings_nr; // save for later use
global_strings_size = strings_size; // save for later use
global_strings = realloc(global_strings,
sizeof *global_strings * strings_nr * strings_size);
if (global_strings == NULL) {
global_strings_nr = global_strings_size = 0;
}
}
// Access function
char *global_strings_get(size_t index) {
if (index >= global_strings_nr) {
return NULL;
}
return global_strings + index*global_strings_size;
}
#define GLOBAL_STRINGS_GET_WO_CHECK(index) \
(global_strings + (index)*global_strings_size)
Better to use size_t for array indexing and sizing than int.
Casts not needed.
Memory calculations should begin with a size_t rather than int * int * size_t.

How to concat byte arrays in C

My current concat function:
char* concat(char* a, int a_size,
char* b, int b_size) {
char* c = malloc(a_size + b_size);
memcpy(c, a, a_size);
memcpy(c + a_size, b, b_size);
free(a);
free(b);
return c;
}
But this used extra memory. Is it possible to append two byte arrays using realloc without making extra memory space?
Like:
void append(char* a, int a_size, char* b, int b_size)
...
char* a = malloc(2);
char* b = malloc(2);
void append(a, 2, b, 2);
//The size of a will be 4.
While Jean-François Fabre answered the stated question, I'd like to point out that you can manage such byte arrays better by using a structure:
typedef struct {
size_t max; /* Number of chars allocated for */
size_t len; /* Number of chars in use */
unsigned char *data;
} bytearray;
#define BYTEARRAY_INIT { 0, 0, NULL }
void bytearray_init(bytearray *barray)
{
barray->max = 0;
barray->len = 0;
barray->data = NULL;
}
void bytearray_free(bytearray *barray)
{
free(barray->data);
barray->max = 0;
barray->len = 0;
barray->data = NULL;
}
To declare an empty byte array, you can use either bytearray myba = BYTEARRAY_INIT; or bytearray myba; bytearray_init(&myba);. The two are equivalent.
When you no longer need the array, call bytearray_free(&myba);. Note that free(NULL) is safe and does nothing, so it is perfectly safe to free a bytearray that you have initialized, but not used.
To append to a bytearray:
int bytearray_append(bytearray *barray, const void *from, const size_t size)
{
if (barray->len + size > barray->max) {
const size_t len = barray->len + size;
size_t max;
void *data;
/* Example policy: */
if (len < 8)
max = 8; /* At least 8 chars, */
else
if (len < 4194304)
max = (3*len) / 2; /* grow by 50% up to 4,194,304 bytes, */
else
max = (len | 2097151) + 2097153 - 24; /* then pad to next multiple of 2,097,152 sans 24 bytes. */
data = realloc(barray->data, max);
if (!data) {
/* Not enough memory available. Old data is still valid. */
return -1;
}
barray->max = max;
barray->data = data;
}
/* Copy appended data; we know there is room now. */
memmove(barray->data + barray->len, from, size);
barray->len += size;
return 0;
}
Since this function can at least theoretically fail to reallocate memory, it will return 0 if successful, and nonzero if it cannot reallocate enough memory.
There is no need for a malloc() call, because realloc(NULL, size) is exactly equivalent to malloc(size).
The "growth policy" is a very debatable issue. You can just make max = barray->len + size, and be done with it. However, dynamic memory management functions are relatively slow, so in practice, we don't want to call realloc() for every small little addition.
The above policy tries to do something better, but not too aggressive: it always allocates at least 8 characters, even if less is needed. Up to 4,194,304 characters, it allocates 50% extra. Above that, it rounds the allocation size to the next multiple of 2,097,152 and substracts 24. The reasoning behid this is complex, but it is more for illustration and understanding than anything else; it is definitely NOT "this is best, and this is what you should do too". This policy ensures that each byte array allocates at most 4,194,304 = 222 unused characters. However, 2,097,152 = 221 is the size of a huge page on AMD64 (x86-64), and is a power-of-two multiple of a native page size on basically all architectures. It is also large enough to switch from so-called sbrk() allocation to memory mapping on basically all architectures that do that. It means that such huge allocations use a separate part of the heap for each, and the unused part is usually just virtual memory, not necessarily backed by any RAM, until accessed. As a result, this policy tends to work quite well for both very short byte arrays, and very long byte arrays, on most architectures.
Of course, if you know (or measure!) the typical size of the byte arrays in typical workloads, you can optimize the growth policy for that, and get even better results.
Finally, it uses memmove() instead of memcpy(), just in case someone wishes to repeat a part of the same byte array: memcpy() only works if the source and target areas do not overlap; memmove() works even in that case.
When using more advanced data structures, like hash tables, a variant of the above structure is often useful. (That is, this is much better in cases where you have lots of empty byte arrays.)
Instead of having a pointer to the data, the data is part of the structure itself, as a C99 flexible array member:
typedef struct {
size_t max;
size_t len;
unsigned char data[];
} bytearray;
You cannot declare a byte array itself (i.e. bytearray myba; will not work); you always declare a pointer to a such byte arrays: bytearray *myba = NULL;. The pointer being NULL is just treated the same as an empty byte array.
In particular, to see how many data items such an array has, you use an accessor function (also defined in the same header file as the data structure), rather than myba.len:
static inline size_t bytearray_len(bytearray *const barray)
{
return (barray) ? barray->len : 0;
}
static inline size_t bytearray_max(bytearray *const barray)
{
return (barray) ? barray->max : 0;
}
The (expression) ? (if-true) : (if-false) is a ternary operator. In this case, the first function is exactly equivalent to
static inline size_t bytearray_len(bytearray *const barray)
{
if (barray)
return barray->len;
else
return 0;
}
If you wonder about the bytearray *const barray, remember that pointer declarations are read from right to left, with * as "a pointer to". So, it just means that barray is constant, a pointer to a byte array. That is, we may change the data it points to, but we won't change the pointer itself. Compilers can usually detect such stuff themselves, but it may help; the main point is however to remind us human programmers that the pointer itself is not to be changed. (Such changes would only be visible within the function itself.)
Since such arrays often need to be resized, the resizing is often put into a separate helper function:
bytearray *bytearray_resize(bytearray *const barray, const size_t len)
{
bytearray *temp;
if (!len) {
free(barray);
errno = 0;
return NULL;
}
if (!barray) {
temp = malloc(sizeof (bytearray) + len * sizeof barray->data[0]);
if (!temp) {
errno = ENOMEM;
return NULL;
}
temp->max = len;
temp->len = 0;
return temp;
}
if (barray->len > len)
barray->len = len;
if (barray->max == len)
return barray;
temp = realloc(barray, sizeof (bytearray) + len * sizeof barray->data[0]);
if (!temp) {
free(barray);
errno = ENOMEM;
return NULL;
}
temp->max = len;
return temp;
}
What does that errno = 0 do in there? The idea is that because resizing/reallocating a byte array may change the pointer, we return the new one. If the allocation fails, we return NULL with errno == ENOMEM, just like malloc()/realloc() do. However, since the desired new length was zero, this saves memory by freeing the old byte array if any, and returns NULL. But since that is not an error, we set errno to zero, so that it is easier for callers to check if an error occurred or not. (If the function returns NULL, check errno. If errno is nonzero, an error occurred; you can use strerror(errno) to get a descriptive error message.)
You probably also noted the sizeof barray->data[0], used even when barray is NULL. This is okay, because sizeof is not a function, but an operator: it does not access the right side at all, it only evaluates to the size of the thing the right side refers to. (You only need to use parentheses when the right size is a type.) This form is nice, because it lets a programmer change the type of the data member, without changing any other code.
To append data to such a byte array, we probably want to be able to specify whether we anticipate further appends to the same array, or whether this is probably the final append, so that only the exact needed amount of memory is needed. For simplicity, I'll only implement the exact size version here. Note that this function returns a pointer to the (modified) byte array:
bytearray *bytearray_append(bytearray *barray,
const void *from, const size_t size,
int exact)
{
size_t len = bytearray_len(barray) + size;
if (exact) {
barray = bytearray_resize(barray, len);
if (!barray)
return NULL; /* errno already set by bytearray_resize(). */
} else
if (bytearray_max(barray) < len) {
if (!exact) {
/* Apply growth policy */
if (len < 8)
len = 8;
else
if (len < 4194304)
len = (3 * len) / 2;
else
len = (len | 2097151) + 2097153 - 24;
}
barray = bytearray_resize(barray, len);
if (!barray)
return NULL; /* errno already set by the bytearray_resize() call */
}
if (size) {
memmove(barray->data + barray->len, from, size);
barray->len += size;
}
return barray;
}
This time, we declared bytearray *barray, because we change where barray points to in the function. If the fourth parameter, final, is nonzero, then the resulting byte array is exactly the size needed; otherwise the growth policy is applied.
yes, since realloc will preserve the start of your buffer if the new size is bigger:
char* concat(char* a, size_t a_size,
char* b, size_t b_size) {
char* c = realloc(a, a_size + b_size);
memcpy(c + a_size, b, b_size); // dest is after "a" data, source is b with b_size
free(b);
return c;
}
c may be different from a (if the original memory block cannot be resized in-place contiguously to the new size by the system) but if that's the case, the location pointed by a will be freed (you must not free it), and the original data will be "moved".
My advice is to warn the users of your function that the input buffers must be allocated using malloc, else it will crash badly.

Get the length of an array with a pointer? [duplicate]

I've allocated an "array" of mystruct of size n like this:
if (NULL == (p = calloc(sizeof(struct mystruct) * n,1))) {
/* handle error */
}
Later on, I only have access to p, and no longer have n. Is there a way to determine the length of the array given just the pointer p?
I figure it must be possible, since free(p) does just that. I know malloc() keeps track of how much memory it has allocated, and that's why it knows the length; perhaps there is a way to query for this information? Something like...
int length = askMallocLibraryHowMuchMemoryWasAlloced(p) / sizeof(mystruct)
I know I should just rework the code so that I know n, but I'd rather not if possible. Any ideas?
No, there is no way to get this information without depending strongly on the implementation details of malloc. In particular, malloc may allocate more bytes than you request (e.g. for efficiency in a particular memory architecture). It would be much better to redesign your code so that you keep track of n explicitly. The alternative is at least as much redesign and a much more dangerous approach (given that it's non-standard, abuses the semantics of pointers, and will be a maintenance nightmare for those that come after you): store the lengthn at the malloc'd address, followed by the array. Allocation would then be:
void *p = calloc(sizeof(struct mystruct) * n + sizeof(unsigned long int),1));
*((unsigned long int*)p) = n;
n is now stored at *((unsigned long int*)p) and the start of your array is now
void *arr = p+sizeof(unsigned long int);
Edit: Just to play devil's advocate... I know that these "solutions" all require redesigns, but let's play it out.
Of course, the solution presented above is just a hacky implementation of a (well-packed) struct. You might as well define:
typedef struct {
unsigned int n;
void *arr;
} arrInfo;
and pass around arrInfos rather than raw pointers.
Now we're cooking. But as long as you're redesigning, why stop here? What you really want is an abstract data type (ADT). Any introductory text for an algorithms and data structures class would do it. An ADT defines the public interface of a data type but hides the implementation of that data type. Thus, publicly an ADT for an array might look like
typedef void* arrayInfo;
(arrayInfo)newArrayInfo(unsignd int n, unsigned int itemSize);
(void)deleteArrayInfo(arrayInfo);
(unsigned int)arrayLength(arrayInfo);
(void*)arrayPtr(arrayInfo);
...
In other words, an ADT is a form of data and behavior encapsulation... in other words, it's about as close as you can get to Object-Oriented Programming using straight C. Unless you're stuck on a platform that doesn't have a C++ compiler, you might as well go whole hog and just use an STL std::vector.
There, we've taken a simple question about C and ended up at C++. God help us all.
keep track of the array size yourself; free uses the malloc chain to free the block that was allocated, which does not necessarily have the same size as the array you requested
Just to confirm the previous answers: There is no way to know, just by studying a pointer, how much memory was allocated by a malloc which returned this pointer.
What if it worked?
One example of why this is not possible. Let's imagine the code with an hypothetic function called get_size(void *) which returns the memory allocated for a pointer:
typedef struct MyStructTag
{ /* etc. */ } MyStruct ;
void doSomething(MyStruct * p)
{
/* well... extract the memory allocated? */
size_t i = get_size(p) ;
initializeMyStructArray(p, i) ;
}
void doSomethingElse()
{
MyStruct * s = malloc(sizeof(MyStruct) * 10) ; /* Allocate 10 items */
doSomething(s) ;
}
Why even if it worked, it would not work anyway?
But the problem of this approach is that, in C, you can play with pointer arithmetics. Let's rewrite doSomethingElse():
void doSomethingElse()
{
MyStruct * s = malloc(sizeof(MyStruct) * 10) ; /* Allocate 10 items */
MyStruct * s2 = s + 5 ; /* s2 points to the 5th item */
doSomething(s2) ; /* Oops */
}
How get_size is supposed to work, as you sent the function a valid pointer, but not the one returned by malloc. And even if get_size went through all the trouble to find the size (i.e. in an inefficient way), it would return, in this case, a value that would be wrong in your context.
Conclusion
There are always ways to avoid this problem, and in C, you can always write your own allocator, but again, it is perhaps too much trouble when all you need is to remember how much memory was allocated.
Some compilers provide msize() or similar functions (_msize() etc), that let you do exactly that
May I recommend a terrible way to do it?
Allocate all your arrays as follows:
void *blockOfMem = malloc(sizeof(mystruct)*n + sizeof(int));
((int *)blockofMem)[0] = n;
mystruct *structs = (mystruct *)(((int *)blockOfMem) + 1);
Then you can always cast your arrays to int * and access the -1st element.
Be sure to free that pointer, and not the array pointer itself!
Also, this will likely cause terrible bugs that will leave you tearing your hair out. Maybe you can wrap the alloc funcs in API calls or something.
malloc will return a block of memory at least as big as you requested, but possibly bigger. So even if you could query the block size, this would not reliably give you your array size. So you'll just have to modify your code to keep track of it yourself.
For an array of pointers you can use a NULL-terminated array. The length can then determinate like it is done with strings. In your example you can maybe use an structure attribute to mark then end. Of course that depends if there is a member that cannot be NULL. So lets say you have an attribute name, that needs to be set for every struct in your array you can then query the size by:
int size;
struct mystruct *cur;
for (cur = myarray; cur->name != NULL; cur++)
;
size = cur - myarray;
Btw it should be calloc(n, sizeof(struct mystruct)) in your example.
Other have discussed the limits of plain c pointers and the stdlib.h implementations of malloc(). Some implementations provide extensions which return the allocated block size which may be larger than the requested size.
If you must have this behavior you can use or write a specialized memory allocator. This simplest thing to do would be implementing a wrapper around the stdlib.h functions. Some thing like:
void* my_malloc(size_t s); /* Calls malloc(s), and if successful stores
(p,s) in a list of handled blocks */
void my_free(void* p); /* Removes list entry and calls free(p) */
size_t my_block_size(void* p); /* Looks up p, and returns the stored size */
...
really your question is - "can I find out the size of a malloc'd (or calloc'd) data block". And as others have said: no, not in a standard way.
However there are custom malloc implementations that do it - for example http://dmalloc.com/
I'm not aware of a way, but I would imagine it would deal with mucking around in malloc's internals which is generally a very, very bad idea.
Why is it that you can't store the size of memory you allocated?
EDIT: If you know that you should rework the code so you know n, well, do it. Yes it might be quick and easy to try to poll malloc but knowing n for sure would minimize confusion and strengthen the design.
One of the reasons that you can't ask the malloc library how big a block is, is that the allocator will usually round up the size of your request to meet some minimum granularity requirement (for example, 16 bytes). So if you ask for 5 bytes, you'll get a block of size 16 back. If you were to take 16 and divide by 5, you would get three elements when you really only allocated one. It would take extra space for the malloc library to keep track of how many bytes you asked for in the first place, so it's best for you to keep track of that yourself.
This is a test of my sort routine. It sets up 7 variables to hold float values, then assigns them to an array, which is used to find the max value.
The magic is in the call to myMax:
float mmax = myMax((float *)&arr,(int) sizeof(arr)/sizeof(arr[0]));
And that was magical, wasn't it?
myMax expects a float array pointer (float *) so I use &arr to get the address of the array, and cast it as a float pointer.
myMax also expects the number of elements in the array as an int. I get that value by using sizeof() to give me byte sizes of the array and the first element of the array, then divide the total bytes by the number of bytes in each element. (we should not guess or hard code the size of an int because it's 2 bytes on some system and 4 on some like my OS X Mac, and could be something else on others).
NOTE:All this is important when your data may have a varying number of samples.
Here's the test code:
#include <stdio.h>
float a, b, c, d, e, f, g;
float myMax(float *apa,int soa){
int i;
float max = apa[0];
for(i=0; i< soa; i++){
if (apa[i]>max){max=apa[i];}
printf("on i=%d val is %0.2f max is %0.2f, soa=%d\n",i,apa[i],max,soa);
}
return max;
}
int main(void)
{
a = 2.0;
b = 1.0;
c = 4.0;
d = 3.0;
e = 7.0;
f = 9.0;
g = 5.0;
float arr[] = {a,b,c,d,e,f,g};
float mmax = myMax((float *)&arr,(int) sizeof(arr)/sizeof(arr[0]));
printf("mmax = %0.2f\n",mmax);
return 0;
}
In uClibc, there is a MALLOC_SIZE macro in malloc.h:
/* The size of a malloc allocation is stored in a size_t word
MALLOC_HEADER_SIZE bytes prior to the start address of the allocation:
+--------+---------+-------------------+
| SIZE |(unused) | allocation ... |
+--------+---------+-------------------+
^ BASE ^ ADDR
^ ADDR - MALLOC_HEADER_SIZE
*/
/* The amount of extra space used by the malloc header. */
#define MALLOC_HEADER_SIZE \
(MALLOC_ALIGNMENT < sizeof (size_t) \
? sizeof (size_t) \
: MALLOC_ALIGNMENT)
/* Set up the malloc header, and return the user address of a malloc block. */
#define MALLOC_SETUP(base, size) \
(MALLOC_SET_SIZE (base, size), (void *)((char *)base + MALLOC_HEADER_SIZE))
/* Set the size of a malloc allocation, given the base address. */
#define MALLOC_SET_SIZE(base, size) (*(size_t *)(base) = (size))
/* Return base-address of a malloc allocation, given the user address. */
#define MALLOC_BASE(addr) ((void *)((char *)addr - MALLOC_HEADER_SIZE))
/* Return the size of a malloc allocation, given the user address. */
#define MALLOC_SIZE(addr) (*(size_t *)MALLOC_BASE(addr))
malloc() stores metadata regarding space allocation before 8 bytes from space actually allocated. This could be used to determine space of buffer. And on my x86-64 this always return multiple of 16. So if allocated space is multiple of 16 (which is in most cases) then this could be used:
Code
#include <stdio.h>
#include <malloc.h>
int size_of_buff(void *buff) {
return ( *( ( int * ) buff - 2 ) - 17 ); // 32 bit system: ( *( ( int * ) buff - 1 ) - 17 )
}
void main() {
char *buff = malloc(1024);
printf("Size of Buffer: %d\n", size_of_buff(buff));
}
Output
Size of Buffer: 1024
This is my approach:
#include <stdio.h>
#include <stdlib.h>
typedef struct _int_array
{
int *number;
int size;
} int_array;
int int_array_append(int_array *a, int n)
{
static char c = 0;
if(!c)
{
a->number = NULL;
a->size = 0;
c++;
}
int *more_numbers = NULL;
a->size++;
more_numbers = (int *)realloc(a->number, a->size * sizeof(int));
if(more_numbers != NULL)
{
a->number = more_numbers;
a->number[a->size - 1] = n;
}
else
{
free(a->number);
printf("Error (re)allocating memory.\n");
return 1;
}
return 0;
}
int main()
{
int_array a;
int_array_append(&a, 10);
int_array_append(&a, 20);
int_array_append(&a, 30);
int_array_append(&a, 40);
int i;
for(i = 0; i < a.size; i++)
printf("%d\n", a.number[i]);
printf("\nLen: %d\nSize: %d\n", a.size, a.size * sizeof(int));
free(a.number);
return 0;
}
Output:
10
20
30
40
Len: 4
Size: 16
If your compiler supports VLA (variable length array), you can embed the array length into the pointer type.
int n = 10;
int (*p)[n] = malloc(n * sizeof(int));
n = 3;
printf("%d\n", sizeof(*p)/sizeof(**p));
The output is 10.
You could also choose to embed the information into the allocated memory yourself with a structure including a flexible array member.
struct myarray {
int n;
struct mystruct a[];
};
struct myarray *ma =
malloc(sizeof(*ma) + n * sizeof(struct mystruct));
ma->n = n;
struct mystruct *p = ma->a;
Then to recover the size, you would subtract the offset of the flexible member.
int get_size (struct mystruct *p) {
struct myarray *ma;
char *x = (char *)p;
ma = (void *)(x - offsetof(struct myarray, a));
return ma->n;
}
The problem with trying to peek into heap structures is that the layout might change from platform to platform or from release to release, and so the information may not be reliably obtainable.
Even if you knew exactly how to peek into the meta information maintained by your allocator, the information stored there may have nothing to do with the size of the array. The allocator simply returned memory that could be used to fit the requested size, but the actual size of the memory may be larger (perhaps even much larger) than the requested amount.
The only reliable way to know the information is to find a way to track it yourself.

Is it OK to malloc an array in a called function but free it in the calling function?

I'm not an expert in C, but here's what I'm trying to do:
int main(void) {
double *myArray;
...
myFunction(myArray);
...
/* save myArray contents to file */
...
free(myArray);
...
return 0;
}
int myFunction(double *myArray) {
int len=0;
...
/* compute len */
...
myArray = malloc( sizeof(double) * len );
if (myArray == NULL)
exit(1);
...
/* populate myArray */
...
return 0;
}
I'd like to save the contents of myArray inside main, but I don't know the size required until the program is inside myFunction.
Since I'm using CentOS 6.2 Linux, which I could only find a gcc build available up to 4.4.6 (which doesn't support C99 feature of declaring a variable-length array; see "broken" under "Variable-length arrays in http://gcc.gnu.org/gcc-4.4/c99status.html), I'm stuck using -std=c89 to compile.
Simple answer is no.
You are not passing back the pointer.
use
int main(void) {
double *myArray;
...
myFunction(&myArray);
...
/* save myArray contents to file */
...
free(myArray);
...
return 0;
}
int myFunction(double **myArray) {
int len=0;
...
/* compute len */
...
*myArray = malloc( sizeof(double) * len );
if (NULL == *myArray)
exit(1);
...
EDIT
poputateThis = *myArray;
/* populate poputateThis */
END OF EDIT
...
return 0;
EDIT
Should simplify thigs for your
}
What you are doing is not OK since myFunction doesn't change the value myArray holds in main; it merely changes its own copy.
Other than that, it's OK even if stylistically debatable.
As a question of good design and practice (apart from syntax issues pointed out in other answers) this is okay as long as it is consistent with your code base's best practices and transparent. Your function should be documented so that the caller knows it has to free and furthermore knows not to allocate its own memory. Furthermore consider making an abstract data type such as:
// myarray.h
struct myarray_t;
int myarray_init(myarray_t* array); //int for return code
int myarray_cleanup(myarray_t* array); // will clean up
myarray_t will hold a dynamic pointer that will be encapsulated from the calling function, although in the init and cleanup functions it will respectively allocate and deallocate.
What you want to do is fine, but your code doesn't do it -- main never gets to see the allocated memory. The parameter myArray of myFunction is initialized with the value passed in the function call, but modifying it thereafter doesn't modify the otherwise-unrelated variable of the same name in main.
It appears in your code snippet that myFunction always returns 0. If so then the most obvious way to fix your code is to return myArray instead (and take no parameter). Then the call in main would look like myArray = myFunction();.
If myFunction in fact already uses its return value then you can pass in a pointer to double*, and write the address to the referand of that pointer. This is what Ed Heal's answer does. The double ** parameter is often called an "out-param", since it's a pointer to a location that the function uses to store its output. In this case, the output is the address of the buffer.
An alternative would be to do something like this:
size_t myFunction(double *myArray, size_t buf_len) {
int len=0;
...
/* compute len */
...
if (buf_len < len) {
return len;
}
/* populate myArray */
...
return len;
}
Then the callers have the freedom to allocate memory any way they like. Typical calling code might look like this:
size_t len = myFunction(NULL, 0);
// warning -- watch the case where len == 0, if that is possible
double *myArray = malloc(len * sizeof(*myArray));
if (!myArray) exit(1);
myFunction(myArray, len);
...
free(myArray);
What you've gained is that the caller can allocate the memory from anywhere that's convenient. What you've lost is that the caller has to write more code.
For an example of how to use that freedom, a caller could write:
#define SMALLSIZE 10;
void one_of_several_jobs() {
// doesn't usually require much space, occasionally does
double smallbuf[SMALLSIZE];
double *buf = 0;
size_t len = myFunction(smallbuf, SMALLSIZE);
if (len > SMALLSIZE) {
double *buf = malloc(len * sizeof(*buf));
if (!buf) {
puts("this job is too big, skipping it and moving to the next one");
return;
}
} else {
buf = smallbuf;
}
// use buf and len for something
...
if (buf != smallbuf) free(buf);
}
It's usually an unnecessary optimization to avoid a malloc in the common case where only a small buffer is needed -- this is only one example of why the caller might want a say in how the memory is allocated. A more pressing reason might be that your function is compiled into a different dll from the caller's function, perhaps using a different compiler, and the two don't use compatible implementations of malloc/free.

how can i know the allocated memory size of pointer variable in c [duplicate]

This question already has answers here:
Determine size of dynamically allocated memory in C
(15 answers)
Closed 3 years ago.
I have faced some problem in this case can you please your ideas.
main()
{
char *p=NULL;
p=(char *)malloc(2000 * sizeof(char));
printf("size of p = %d\n",sizeof (p));
}
In this program Its print the 4 that (char *) value,but i need how many bytes allocated for
that.
You could also implement a wrapper for malloc and free to add tags (like allocated size and other meta information) before the pointer returned by malloc. This is in fact the method that a c++ compiler tags objects with references to virtual classes.
Here is one working example:
#include <stdlib.h>
#include <stdio.h>
void * my_malloc(size_t s)
{
size_t * ret = malloc(sizeof(size_t) + s);
*ret = s;
return &ret[1];
}
void my_free(void * ptr)
{
free( (size_t*)ptr - 1);
}
size_t allocated_size(void * ptr)
{
return ((size_t*)ptr)[-1];
}
int main(int argc, const char ** argv) {
int * array = my_malloc(sizeof(int) * 3);
printf("%u\n", allocated_size(array));
my_free(array);
return 0;
}
The advantage of this method over a structure with size and pointer
struct pointer
{
size_t size;
void *p;
};
is that you only need to replace the malloc and free calls. All other pointer operations require no refactoring.
There is no portable way but for windows:
#include <stdio.h>
#include <malloc.h>
#if defined( _MSC_VER ) || defined( __int64 ) /* for VisualC++ or MinGW/gcc */
#define howmanybytes(ptr) ((unsigned long)_msize(ptr))
#else
#error no known way
#endif
int main()
{
char *x=malloc(1234);
printf( "%lu", howmanybytes(x) );
return 0;
}
Although it may be possible that some libraries allows you to determine the size of an allocated buffer, it wouldn't be a standard C function and you should be looking at your library's own documentations for this.
However, if there are many places that you need to know the size of your allocated memory, the cleanest way you could do it is to keep the size next to the pointer. That is:
struct pointer
{
size_t size;
void *p;
};
Then every time you malloc the pointer, you write down the size in the size field also. The problem with this method however is that you have to cast the pointer every time you use it. If you were in C++, I would have suggested using template classes. However, in this case also it's not hard, just create as many structs as the types you have. So for example
struct charPtr
{
size_t size;
char *p;
};
struct intPtr
{
size_t size;
int *p;
};
struct objectPtr
{
size_t size;
struct object *p;
};
Given similar names, once you define the pointer, you don't need extra effort (such as casting) to access the array. An example of usage is:
struct intPtr array;
array.p = malloc(1000 * sizeof *array.p);
array.size = array.p?1000:0;
...
for (i = 0; i < array.size; ++i)
printf("%s%d", i?" ":"", array.p[i]);
printf("\n");
It is impossible to know how much memory was allocated by just the pointer. doing sizeof (p) will get the size of the pointer variable p which it takes at compile time, and which is the size of the pointer. That is, the memory the pointer variable takes to store the pointer variable p. Inside p the starting address of the memory block is stored.
Once you allocate some memory with malloc it will return the starting address of the memory block, but the end of the block cannot be found from it, as there is no terminator for a block. You define the end of the block therefore you need to identify it by any means, so store it somewhere. Therefore you need to preserve the block length somewhere to know where the block which is pointed to by p ends.
Note: Although the memory allocation structure keeps track of allocated and unallocated blocks, therefore we can know the allocated memory block length from these structures, but these structures are not available to be used by the users, unless any library function provides them. Therefore a code using such feature is not portable (pointed by #Rudy Velthuis) . Therefore it is the best to keep track of the structure yourself.
You need to keep track of it in a variable if you want to know it for later:
char *p = NULL;
int sizeofp = 2000*sizeof(char);
p = (char *)malloc(sizeofp);
printf("size of p = %d\n",sizeofp);
You cannot use the sizeof in this case, since p is a pointer, not an array, but since you allocate it, you already know:
main()
{
size_t arr_size = 2000;
char *p=NULL;
p=malloc(arr_size * sizeof(char));
printf("size of p = %d\n",arr_size);
}
Edit - If the malloc fails to allocate the size you wanted, it won't give you a pointer to a smaller buffer, but it will return NULL.

Resources