My question is related to this one :
c define arrays in struct with different sizes
However, I do NOT want to use dynamic allocation (embedded target).
Problem recap :
In C, I want to have two versions of the same structure, each one with a different size for its static arrays.
Both the structures will be used by the same functions through pointer parameter.
typedef struct {
short isLarge; //set 0 at initialization
short array[SIZE_A];
//more arrays
} doc_t;
typedef struct {
short isLarge; //set 1 at initialization
short array[SIZE_B];
//more arrays
} doc_large_t;
void function( doc_t* document ) {
if ( document->isLarge ) {
//change document into doc_large_t* [1]
}
//common code for both doc_t and doc_large_t
}
Questions :
(1) The above description needs a way to dynamically cast the pointer doc_t* pointer to doc_large_t* document [1]. Is that possible ? How ?
(2) An other solution i came with is to have a common header data part for both structure, including not only the isLarge flag, but also the pointers to the following static arrays. How ugly is that ?
(3) Also, do you have a good trick or workarround I could use ?
EDIT :
More context :
My application is a path finding on an embedded MCU.
I have geometrical objects, like polygons. Polygons can describe simple rectangular obstacles, as well as more complex shapes (such as the accessible area).
Complex polygons can have a huge amount of vertices, but are in small quantity. Simple polygons are very common.
Both will use the same algorithms.
I know in advance which polygon will need more vertices.
What I am trying to do is to optimize working memory to make it fit into the MCU. (i.e. small shapes get small arrays; complex ones get large arrays)
Idea similar to what you mentioned in your question already (pointers to arrays), but with only one single pointer:
typedef struct
{
short array[SIZE_B - SIZE_A];
// more arrays alike...
} Extension;
typedef struct
{
short array[SIZE_A];
//more arrays (all the small ones!)
Extension* extraData;
} doc_t;
If extraData is NULL, you have a small polygone, otherwise, you find the additional data in the struct referenced. Admitted, iterating over all values for large polygons gets a little nasty...
If you can use global arrays of predefined size for each object type (as Dominic Gibson proposed - a good proposition, by the way), you could spare the isLarge flag by replacing it with a function:
int isLarge(void* ptr)
{
return
(uintptr_t)globalLargeArray <= (uintptr_t)ptr
&&
(uintptr_t)ptr < (uintptr_t)globalLargeArray + sizeof(globalLargeArray);
}
Of course, all polygons (in above case: the large ones at least) would have to live in this array to make it work. If you create at least one dynamically or otherwise elsewhere (stack, another global variable) - we are out...
Create the arrays globally and use a pointer pointig to the big or small array.
You should try to keep a single structure and for the different array sizes put them in an union. I don't know whether the following structure would make sense to your case.
typedef struct {
short isLarge; //manually set to 0 or 1 after creating structure
//and accordingly initialize the arrays in below union
union my_varying_arrays {
short array_A[SIZE_A];
short array_B[SIZE_B];
};
//more arrays
} doc_t;
If isLarge is 0, set the value for array_A array and if 1 set the value for array array_B.
You can do this is the data is const by using a void * to the specific array.
Then you just cast the void * to what you need it to be depending on the attributes in the structure.
It becomes more complicated when you need the structures in runtime.
Especially on embedded targets.
typedef struct {
short size;
void *array;
} doc_t;
Where array points to a memory block allocated by the memory manager.
You now have to decide whether to use C standard malloc or use some pooled memory system based on the largest block size.
An example would be ChibiOS Memory pools.
If you are allocating and freeing variable sized memory blocks at random you risk memory fragmentation.
If you allocate incrementally you don't have to worry about much about memory. Just create one large block and keep track of where you are. A bit like a stack.
After the edit, I think the best thing you can do is to profile your needs defining max simple and complex polygons your target can manage and then declare a pool of simplex and common polygons, like:
#include <stdio.h>
#include <stdint.h>
#include <stdbool.h>
#define MAX_COMPLEX 16
#define MAX_SIMPLE 16
uint16_t g_Simple_Poly_set[MAX_COMPLEX][SIZE_A];
uint16_t g_Complex_Poly_set[MAX_COMPLEX][SIZE_B];
uint16_t g_Simple_Poly_used = 0;
uint16_t g_Complex_Poly_used = 0;
struct poly
{
bool isLarge;
uint16_t *vetexes;
};
bool create_poly_simple (struct poly *p)
{
bool retVal = false; // default: not more space for poly
if (g_Simple_Poly_used < MAX_SIMPLE)
{
p->isLarge = false;
p->vetexes = &g_Simple_Poly_set[g_Simple_Poly_used][0];
g_Simple_Poly_used++;
retVal = true;
}
return retVal;
}
bool create_poly_compleX (struct poly *p)
{
bool retVal = false; // default: not more space for poly
if (g_Complex_Poly_used < MAX_COMPLEX)
{
p->isLarge = true;
p->vetexes = &g_Complex_Poly_set[g_Complex_Poly_used][0];
g_Complex_Poly_used++;
retVal = true;
}
return retVal;
}
void your_stuff_with_poly ( struct poly *p)
{
uint32_t poly_size = (p->isLarge == false) ? SIZE_A : SIZE_B;
// your stuff with the correct size
}
This is a simple implementation designed for a static "instantiation" of structs. You can also enhance the code with a create/destroy function that trace which array into pool is free to be used.
Your number 2 solution is the right idea. It's unclear to me why you think that is ugly. Maybe this beautiful implementation will change your mind.
You can implement single inheritance is C by placing the base structure as the first member of the inheriting structure. Then inheriting objects can be referenced with a pointer to the base type.
typedef struct {
short doc_type;
short *array_ptr;
// more array pointers
} doc_base_t;
typedef struct {
doc_base_t base; // base.doc_type set 0 at initialization
short array[SIZE_A]; // base.array_ptr initialized to point here
//more arrays
} doc_small_t;
typedef struct {
doc_base_t base; // base.doc_type set 1 at initialization
short array[SIZE_B]; // base.array_ptr initialized to point here
//more arrays
} doc_large_t;
void function( doc_base_t* document ) {
if ( document->doc_type == 1) {
// array size is large
} else {
// array size is small
}
//common code referencing arrays through doc_base_t->array_ptr
}
The array_ptr member in doc_base_t isn't necessary for the inheritance mechanism. But I added that specifically for the "common code" portion of your function. If doc_base_t didn't include the array_ptr then you could cast the generic document to either adoc_small_t or doc_large_t type based upon the base_type value. But then you might need a different implementation for each inherited type. By adding the array_ptr member to doc_base_t I suspect you could write a common implementation for all inherited types.
So you will statically declare all your instances of doc_small_t and doc_large_t. And you'll initialize both the base.doc_type and base.array_ptr members when initializing each object. Then you will cast both types of objects to doc_base_t before calling function. (Or pass the address of the base member, which results in the same pointer value.)
Updated example:
static doc_small_t doc_small_instances[NUM_SMALL_INSTANCES];
static doc_large_t doc_large_instances[NUM_LARGE_INSTANCES];
// DocInit must be called once at startup to initialize all the instances.
void DocInit()
{
int index;
for (index = 0; index < NUM_SMALL_INSTANCES; index++)
{
doc_small_instances[index].base.doc_type = SMALL;
doc_small_instances[index].base.array_ptr = doc_small_instances[index].array;
}
for (index = 0; index < NUM_LARGE_INSTANCES; index++)
{
doc_large_instances[index].base.doc_type = LARGE;
doc_large_instances[index].base.array_ptr = doc_large_instances[index].array;
}
}
// DocProcess processes one doc, large or small.
void DocProcess(doc_base_t *document)
{
int index;
short *array_member_ptr = document->array_ptr;
int array_size = SMALL;
if (document->doc_type == LARGE)
{
array_size = LARGE;
}
for (index = 0; index < array_size; index++)
{
// Application specific processing of *array_member_ptr goes here.
array_member_ptr++;
}
}
// ProcessAllDocs processes all large and small docs.
void ProcessAllDocs(void)
{
int index;
for (index = 0; index < NUM_SMALL_INSTANCES; index++)
{
DocProcess(&doc_small_instances[index].base);
}
for (index = 0; index < NUM_LARGE_INSTANCES; index++)
{
DocProcess(&doc_large_instances[index].base);
}
}
It's easy with malloc() or similar dynamic allocation methods. Just use a flexible array member:
typedef struct {
short isLarge; //set 0 at initialization
.
.
.
short array[SIZE_A];
short largeArray[];
} doc_t;
To allocate a "small structure":
doc_t *small = malloc( sizeof( *small ) );
small->isLarge = 0;
To allocate a "large structure":
doc_t *large = malloc( sizeof( *large ) + ( SIZE_B - SIZE_A ) * sizeof( large->largeArray[ 0 ] );
large->isLarge = 1;
Note that you must keep the largeArray element last, which means that the array element must be next-to-last for this to work.
Depending on how you do your own allocation, this may or may not be applicable.
(It's also a bit of a hack, since it depends on being able to access data in largeArray by using an index of SIZE_A or greater on array. That's accessing an object outside its bounds...)
Related
I'm struggling for a few days now to find a solution to wrap a C struct containing multiple variable-sized int arrays (stored as pointers) in swig.
Suppose the following minimal example:
typedef struct {
size_t length;
int *a;
int *b;
} mystruct;
where both a and b are pointers to int arrays allocated somewhere in C. The size of both arrays is stored in the length member.
Now, what I would really like to have is two-fold:
access to a and b members in objects of type mystruct should be safe, i.e. exceptions should be thrown if index is out-of-bounds.
the data in a and b must not be copied-over into a python list or tuple but I want to provide __getitem__ methods instead. The reason for that is that the actual struct consists of many such arrays and they get really huge and I don't want to waste any memory by duplicating them.
I've seen examples how to accomplish this with fixed-sized arrays by writing wrapper classes and templates for each member that internally store the size/length of each array individually, e.g.: SWIG interfacing C library to Python (Creating 'iterable' Python data type from C 'sequence' struct) and SWIG/python array inside structure.
However, I assume once I would wrap a and b into a class to enable them to be extended with __getitem__ methods, I won't have access to the length member of mystruct, i.e. the 'container' of a and b.
One thing I tried without success was to write explicit _get and _set methods
typedef struct {
size_t length;
} mystruct;
%extend mystruct {
int *a;
};
%{
int *mystruct_a_get(mystruct *s) {
return mx->a;
}
int *mystruct_b_get(mystruct *s) {
return mx->b;
}
...
%}
But here, the entire arrays a and b would be returned without any control of the maximum index...
My target languages are Python and Perl 5, so I guess one could start writing complicated typemaps for each language. I've done that before for other wrappers and hope there is a more generic solution to my situation that involves only C++ wrapper classes and such.
Any help or idea is appreciated!
Edit for possible solution
So, I couldn't let it go and came up with the following (simplified) solution that more or less combines the solutions I already saw elsewhere. The idea was to redundantly store the array lengths for each of the wrapped arrays:
%{
/* wrapper for variable sized arrays */
typedef struct {
size_t length;
int *data;
} var_array_int;
/* convenience constructor for variable sized array wrapper */
var_array_int *
var_array_int_new(size_t length,
int *data)
{
var_array_int *a = (var_array_int *)malloc(sizeof(var_array_int));
a->length = length;
a->data = data;
return a;
}
/* actual structure I want to wrap */
typedef struct {
size_t length;
int *a;
int *b;
} mystruct;
%}
/* hide all struct members in scripting language */
typedef struct {} var_array_int;
typedef struct {} mystruct;
/* extend variable sized arrays with __len__ and __getitem__ */
%extend var_array_int {
size_t __len__() const {
return $self->length;
}
const int __getitem__(int i) const throw(std::out_of_range) {
if ((i < 0) ||
(i >= $self->length))
throw std::out_of_range("Index out of bounds");
return $self->data[i];
}
};
/* add read-only variable sized array members to container struct */
%extend mystruct {
var_array_int *const a;
var_array_int *const b;
};
/* implement explict _get() methods for the variable sized array members */
%{
var_array_int *
mystruct_a_get(mystruct *s)
{
return var_array_int_new(s->length, s->a);
}
var_array_int *
mystruct_b_get(mystruct *s)
{
return var_array_int_new(s->length, s->b);
}
%}
The above solution only provides read access to the variable sized arrays and does not include any NULL checks for the wrapped int * pointers. My actual solution of course does that and also makes use of templates to wrap variable sized arrays of different types. But I refrained from showing that here for the sake of clarity.
I wonder if there is an easier way to do the above. Also the solution only seems to work in Python so far. Implementing something similar for Perl 5 already gives me a headache.
Currently in my testframework, to keep track of testresults , I am maintaining 4D array : TestResult[domain][client][testno][resourceno]
The testcase basically has 3 loops
For each domain(0-2)
For each client(0-8)
For each resource(0-8)
Execute test1
Store TestResult
Execute test2
Store TestResult
Is there a better datastructure which can be used in C, for this purpose other than 4D array?
Well, let's get to the opposite extrem:
typedef struct
{
// whatever you need...
} Resource;
typedef struct
{
Resource a;
Resource b;
// ...
Resource g;
} Client;
typedef struct
{
Client a;
Client b;
// ...
Client g;
} Domain;
OK, I skipped the Test struct and the one containing the all the domains, the idea should still be clear...
void processResource(Resource* resource)
{ /* ... */ }
void processClient(Client* c)
{
processResource(&c->a);
// ...
processResource(&c->g);
}
void processDomain(Domain* d)
{
processClient(&d->a);
// ...
processClient(&d->g);
}
As already mentioned: it is the other extreme. You might maintain arrays within your structs, such that you might iterate over as:
void processDomain(Domain* d)
{
for(size_t i = 0; i < sizeof(d->clients)/sizeof(*d->clients); ++i)
{
processResource(d->clients + i);
}
}
Or have some lower n-dimensional arrays within some of the structs. Any combination would do the trick, all up to you, whatever you consider most appropriate...
You could even extend to dynamic arrays:
typedef struct
{
Resource* resources;
size_t count;
} Client;
if you consider managing variable numbers of sub-elements useful.
If all this is 'better' than the original 4D-array? Mainly a matter of taste, I'd say, unless perhaps a better chance of reusability especially of the lower nested structs, but current use-case appears too specific for this really applying here.
I am trying to define a C structure where an element is present conditionally. Here is the specific example of a header that I want to define:
typedef struct flowHeader {
int magicNum ;
int trafficType ;
// few other int parameters
int flowDirection; // Present ONLY if trafficType = TT_V6
// few other int parameters
} t_flowHeader ;
I want to know what's the best way to define this data type. I want to apply this header to a buffer that is received on wire. Because of one element missing - the size of structure varies by 4 bytes and I am struggling how to manage that?
# define TT_V6 31
# define FD_NA 0
int flowDir ;
unsigned char buf[ MAXSZ ] ;
t_flowHeader * hdr ;
hdr = (t_flowHeader *) buf ;
if (hdr->trafficType == TT_V6) {
flowDir = hdr->flowDirection ;
} else {
flowDir = FD_NA ;
}
..
One way is to use two separate types. You can reduce duplication by defining nested types for all of the other shared components:
struct flowHeader_start {
int magicNum ;
int trafficType ;
// few other int parameters
};
struct flowHeader_end {
// few other int parameters
};
typedef struct flowHeaderA {
struct flowHeader_start s;
int flowDirection; // Present ONLY if trafficType = TT_V6
struct flowHeader_end e;
} t_flowHeaderA ;
typedef struct flowHeaderB {
struct flowHeader_start s; struct flowHeader_end e;
} t_flowHeaderB ;
...then rearrange the casts a bit:
if (((struct flowHeader_start *)buf)->trafficType == TT_V6) {
hdr = (t_flowHeaderA *) buf ;
flowDir = hdr->flowDirection ;
} else {
hdr = (t_flowHeaderB *) buf ;
flowDir = FD_NA ;
}
The type itself isn't dependent on a condition, but where each of the two possible types is used is dependent on it.
The structure of both types up to the end of the struct flowHeader_start is also guaranteed to be exactly the same (because they have the same first element, and a struct must begin at the first element without padding), so you can cast between either type and struct flowHeader_start safely, meaning that you can test for conditions placed within the first block without knowing the form of the rest of the buffer.
As an extension, some compilers might also let you make the nested structs anonymous, which would be more convenient to look at, not having to access elements of the last struct through the intermediate e field (can't remember if the standard allows this for named types, I think not).
No, you can't do this.
C is a static language, which is why you must declare things in the first place. The compiler must be able to generate code before the program runs, the code can't change based on runtime requirements like that.
You're going to have to declare two different structures and then select the proper one based on runtime data using if tests and so on.
I Don't think you can vary the size of the structure on the basic of a condition.
You can define multiple structures and use them accordingly>
or define those elements which are supposed to be conditionally present at the end of the structure.
I want to define a new data type consisting of an array with a size inputted by the user. For example if the user inputs 128, then my program should make a new type which is basically an array of 16 bytes. This structure's definition needs to be global since I am going to use that type thereafter in my program. It is necessary to have a dynamic size for this structure because I will have a HUGE database populated by that type of variables in the end.
The code I have right now is:
struct user_defined_integer;
.
.
.
void def_type(int num_bits)
{
extern struct user_defined_integer
{
int val[num_bits/sizeof(int)];
};
return;
}
(which is not working)
The closest thing to my question, I have found, is in here:
I need to make a global array in C with a size inputted by the user
(Which is not helpful)
Is there a way to do this, so that my structure is recognized in the whole file?
When doing:
extern struct user_defined_integer
{
int val[num_bits/sizeof(int)];
};
You should get the warning:
warning: useless storage class specifier in empty declaration
because you have an empty declaration. extern does not apply to user_defined_integer, but rather the variable that comes after it. Secondly, this won't work anyway because a struct that contains a variable length array can't have any linkage.
error: object with variably modified type must have no linkage
Even so, variable length arrays allocate storage at the point of declaration. You should instead opt for dynamic memory.
#include <stdlib.h>
typedef struct
{
int num_bits;
int* val;
} user_defined_integer;
void set_val(user_defined_integer* udi, int num_bits)
{
udi->num_bits = num_bits;
udi->val = malloc(num_bits/sizeof(int));
}
What you need is a VLA member, as asked about here. Basically, you declare a struct with a size field and one element's worth of storage as last member, and over-allocate it.
Imported from that question :
typedef struct Bitmapset {
int nwords;
uint32 words[1];
} Bitmapset;
Bitmapset *allocate(int n) {
Bitmapset *p = malloc(offsetof(Bitmapset, words) + n * sizeof *p->words);
p->nwords = n;
return p;
}
I want to define a new data type consisting of an array with a size inputted by the user. For example if the user inputs 128, then my program should make a new type which is basically an array of 16 bytes.
This is not possible in C, because C types are a compile-time thing and don't exist at all at run-time.
However, with a C99 conforming compiler, you might use flexible array member. You'll need a struct containing some members and ending with an array without any given dimension, e.g.
struct my_flex_st {
unsigned size;
int arr[]; // of size elements
};
Here is a way to allocate it:
struct my_flex_st *make_flex(unsigned siz) {
struct my_flex_st* ptr
= malloc(sizeof(struct my_flex_st) + siz * sizeof(int));
if (!ptr) { perror("malloc my_flex_st"); exit(EXIT_FAILURE); };
ptr->size = siz;
memset (ptr->arr, 0, siz*sizeof(int));
return ptr;
}
Don't forget to free it once you don't use it anymore.
Of course, you'll need to use pointers in your code. If you really want to have a global variable, declare it as e.g.
extern struct my_flex_st* my_glob_ptr;
Try this method-
#include<stdio.h>
#include<stdlib.h>
#include<limits.h>
struct user_defined_integer
{
int *val;
}user_int;
void memory_allocate(int num_bit)
{
int result;
result = (num_bit+CHAR_BIT-1)/CHAR_BIT; // since 8 bit =1 byte
user_int.val=malloc(result*sizeof(int));
if(user_int.val == NULL){
printf("Failed to allocate memory\n");
return ;
}
else
printf("Allocated %d bytes for val\n",result);
}
int main()
{
int num_bit;
printf("Enter the number of bits\n");
scanf("%d",&num_bit);
memory_allocate(num_bit);
// do your stuff here
free(user_int.val); // free the memory at the end;
return 0;
}
I create drvm *drv structure in my function. This structure itself contains fields which contains malloc()-ed fields (uint32_t *buffer). The code which do that is similar to that:
...
size_t elm_size = sizeof(model*);
uint32_t length = *(uint32_t*)len;
GArray *models = g_array_sized_new(FALSE, FALSE, elm_size, length);
model *mod;
for (int i = 0; i < length; ++i) {
mod = create_model(...);
g_array_append_val(models, mod);
}
This piece of code doesn't contain errors and is highly tested.
At the start of program I register function free_all() (by atexit()) which should clean all resources (especially memory) when exit() is performed.
Inside this function I'm trying freeing memory of elements of GArray* (model * structure) and memory for GArray * itself:
GArray *models;
g_array_set_clear_func(models, clean_model);
if(!g_array_free(models, FALSE)) { //OK }
The problem is that when clean_model(void *data) is called inside glib library I suggest it contains pointer to one model * element. But the address is wrong, it doesn't seem point to any correct value. Neither GArray*, nor model*.
Furthermore GArray *models in free_all() function is correct (the same as when I created it) and when I iterate through all GArray * elements in free_all() by
for (int i = 0; i < len; ++i) {
mod = g_array_index(models, model*, i); // Here I get correct pointer to model*
clean_model(mod);
}
I get expected values.
Question: What's wrong? How should I free memory of elements of GArray * if these elements contain malloc()-ed memory?
Part of header:
struct _info {
uint32_t *buffer;
uint32_t len;
};
typedef struct _info info;
struct _prod {
uint32_t *buffer;
uint32_t len;
};
typedef struct _prod prod;
struct _model {
uint32_t name;
prod product;
info inform;
};
typedef struct _model model;
struct _drvm {
GArray *models;
GArray *strings;
};
typedef struct _drvm drvm;
Basically the problem is that your clean_model function is passed model** instead of model* you were expecting.
Remember that GArray is meant to store complete structs, not just pointers to structs. In order to do that it needs to copy the whole contents of the struct into the internal data array and therefore any subsequent pointers to the structs (as passed to clean_model) are going to be pointing somewhere inside data (i.e. clean_model((elt_type*)&models->data[index * sizeof(elt_type)]) - where in your case elt_type is model*)
To fix the situation couple options come to mind, in order of (subjective) preference:
use GPtrArray instead; given that your elements are dynamically allocated already the memory management / pointer handling / typecasts (or lack thereof) would be less confusing
change clean_model argument to model**
use GArray to store model structs rather than pointers, but only makes sense if you can separate the allocation from populating the model contents, e.g. g_array_new(FALSE, FALSE, sizeof(model)) and fill_model(&g_array_index(models, model, i))
In all cases you should also probably pass TRUE to g_array_free since you don't seem to be using the GArray.data for anything afterwards (not that it would make any sense given that you're freeing all the useful data in it anyway.)