c array wrapper struct no malloc allowed - c

(btw I'm not allowed to malloc in this, I'm writing in c for c99)
I'm trying to create a wrapper struct in c for arrays to keep things tidy so I don't need to keep passing the array pointer and the length around, and can just use a struct:
#define MAX_LEN 64
typedef struct {
uint8_t data[MAX_LEN];
size_t len;
} byteArray_t;
Which is fine if MAX_LEN is known, but is there a way to make the length variable although known at compile time, so that for example I could have something like:
typedef struct {
byteArray_t tag;
byteArray_t length;
byteArray_t value;
} tlv_t;
In which the array corresponding to tag would have size MAX_TAG_LEN and so on for the others - so I'd need to tell the compiler somehow that that was the case...
Anyone got any ideas? Thanks!
Edit
Alright, sorry for the confusion. Here's what I'm trying to do.
I basically have the following structures at present:
// tag object
typedef struct {
uint8_t data[MAX_TAG_LENGTH_IN_BYTES];
uint32_t len;
} tlvTag_t;
// length object
typedef struct {
uint8_t data[MAX_LENGTH_OF_LENGTH_IN_BYTES];
uint32_t len;
} tlvLength_t;
typedef struct tlv tlv_t;
// value object definition
typedef struct {
// can consist of a byte array, or a list of sub TLVs
union {
uint8_t data[MAX_VALUE_LENGTH_IN_BYTES];
// data can be parsed into subTLVs
tlv_t* subTLVs;
};
// need to store the number of subTLVs
uint32_t numberOfSubTLVs;
// len is the total length of the data represented:
// the total length of the subTLVs placed end to end
// or the length of the data array.
uint32_t len;
} tlvValue_t;
// tlv object definition
struct tlv {
tlvTag_t tag;
tlvLength_t len;
tlvValue_t value;
// total length of entire tlv block (not value length)
// (if there are sub TLVs, place them end to end)
uint32_t totalLen;
};
I thought the design would be better if I could wrap the arrays in another struct to avoid all the code duplication and be able to pass fewer arguments around, but I can't because I don't know how to tell the compiler to create different sized byte arrays - maybe it's possible using macros? Hope that makes sense.

It seems you are trying to somehow declare a struct whose contents depend on a parameter. In c++ this could be implemented by a template:
template <size_t MAX_LEN>
struct byteArray_t
{
uint8_t data[MAX_LEN];
size_t len;
};
...
byteArray_t<MAX_TAG_LENGTH_IN_BYTES> tag;
byteArray_t<MAX_LENGTH_OF_LENGTH_IN_BYTES> len;
...
This is as straightforward as it can get.
To accomplish the same in C, you can use macros:
#define DECLARE_BYTE_ARRAY_T(your_type_name, MAX_LEN) \
typedef struct { \
uint8_t data[MAX_LEN]; \
size_t len; \
} your_type_name
DECLARE_BYTE_ARRAY_T(tlvTag_t, MAX_TAG_LENGTH_IN_BYTES);
DECLARE_BYTE_ARRAY_T(tlvLenght_t, MAX_LENGTH_OF_LENGTH_IN_BYTES);
...
tlvTag_t tag;
tlvLength_t len;
Or (the same) without declaring types (good if you don't need names for your structs):
#define BYTE_ARRAY_T(MAX_LEN) \
struct { \
uint8_t data[MAX_LEN]; \
size_t len; \
}
BYTE_ARRAY_T(MAX_TAG_LENGTH_IN_BYTES) tag;
BYTE_ARRAY_T(MAX_LENGTH_OF_LENGTH_IN_BYTES) len;
This may be marginally better than the code you already have. However, in my opinion, this is not worth the effort, because any non-trivial macro decreases readability of code.

If you make a struct like this and then pass it by value to a function, then the whole array gets passed by value. You do not want that.
Actually you don't need an array inside the struct, just declare it elsewhere.
typedef struct {
uint8_t* data;
size_t len;
} byteArray_t;
int main()
{
uint8_t some_array[X];
...
byteArray_t wrapper = {some_array, X};
some_function (&wrapper);
}

Related

How to expose variable sized arrays inside C struct in swig?

I'm struggling for a few days now to find a solution to wrap a C struct containing multiple variable-sized int arrays (stored as pointers) in swig.
Suppose the following minimal example:
typedef struct {
size_t length;
int *a;
int *b;
} mystruct;
where both a and b are pointers to int arrays allocated somewhere in C. The size of both arrays is stored in the length member.
Now, what I would really like to have is two-fold:
access to a and b members in objects of type mystruct should be safe, i.e. exceptions should be thrown if index is out-of-bounds.
the data in a and b must not be copied-over into a python list or tuple but I want to provide __getitem__ methods instead. The reason for that is that the actual struct consists of many such arrays and they get really huge and I don't want to waste any memory by duplicating them.
I've seen examples how to accomplish this with fixed-sized arrays by writing wrapper classes and templates for each member that internally store the size/length of each array individually, e.g.: SWIG interfacing C library to Python (Creating 'iterable' Python data type from C 'sequence' struct) and SWIG/python array inside structure.
However, I assume once I would wrap a and b into a class to enable them to be extended with __getitem__ methods, I won't have access to the length member of mystruct, i.e. the 'container' of a and b.
One thing I tried without success was to write explicit _get and _set methods
typedef struct {
size_t length;
} mystruct;
%extend mystruct {
int *a;
};
%{
int *mystruct_a_get(mystruct *s) {
return mx->a;
}
int *mystruct_b_get(mystruct *s) {
return mx->b;
}
...
%}
But here, the entire arrays a and b would be returned without any control of the maximum index...
My target languages are Python and Perl 5, so I guess one could start writing complicated typemaps for each language. I've done that before for other wrappers and hope there is a more generic solution to my situation that involves only C++ wrapper classes and such.
Any help or idea is appreciated!
Edit for possible solution
So, I couldn't let it go and came up with the following (simplified) solution that more or less combines the solutions I already saw elsewhere. The idea was to redundantly store the array lengths for each of the wrapped arrays:
%{
/* wrapper for variable sized arrays */
typedef struct {
size_t length;
int *data;
} var_array_int;
/* convenience constructor for variable sized array wrapper */
var_array_int *
var_array_int_new(size_t length,
int *data)
{
var_array_int *a = (var_array_int *)malloc(sizeof(var_array_int));
a->length = length;
a->data = data;
return a;
}
/* actual structure I want to wrap */
typedef struct {
size_t length;
int *a;
int *b;
} mystruct;
%}
/* hide all struct members in scripting language */
typedef struct {} var_array_int;
typedef struct {} mystruct;
/* extend variable sized arrays with __len__ and __getitem__ */
%extend var_array_int {
size_t __len__() const {
return $self->length;
}
const int __getitem__(int i) const throw(std::out_of_range) {
if ((i < 0) ||
(i >= $self->length))
throw std::out_of_range("Index out of bounds");
return $self->data[i];
}
};
/* add read-only variable sized array members to container struct */
%extend mystruct {
var_array_int *const a;
var_array_int *const b;
};
/* implement explict _get() methods for the variable sized array members */
%{
var_array_int *
mystruct_a_get(mystruct *s)
{
return var_array_int_new(s->length, s->a);
}
var_array_int *
mystruct_b_get(mystruct *s)
{
return var_array_int_new(s->length, s->b);
}
%}
The above solution only provides read access to the variable sized arrays and does not include any NULL checks for the wrapped int * pointers. My actual solution of course does that and also makes use of templates to wrap variable sized arrays of different types. But I refrained from showing that here for the sake of clarity.
I wonder if there is an easier way to do the above. Also the solution only seems to work in Python so far. Implementing something similar for Perl 5 already gives me a headache.

C: Elegant way to decode array of char (deserialize) to struct?

So I'm currently designing a simple transmission protocol for a ring network implemented in UART.
To transmit, I am converting the data from a struct to a char stream, started by < and terminated by > - I know of the possibility of any 0x3c or 0x3e value being mistaken for a < or > respectively, i am working on a solution to escape that. That's not part of my question.
So, the structure of it is something like <UINT32UINT32UINT8UINT8UINT16char[0...n]>, those types representing: <destinationId senderId TimeToLive Cmdtype CmdId Payloadlength Payload>. This always stays the same, so I can assume it without any delimiters between the values. This works, and I can theoretically also decode this. To easily access the bytes, I implemented the struct with unions:
typedef struct{
union{
uint32_t val;
char bytes[sizeof(uint32_t)];
} recipientId;
union{
uint32_t val;
char bytes[sizeof(uint32_t)];
} senderId;
union{
uint8_t val;
char bytes[sizeof(uint8_t)];
} timeToLive;
union{
uint8_t val;
char bytes[sizeof(uint8_t)];
} cmdType;
union{
uint8_t val;
char bytes[sizeof(uint8_t)];
} cmdId;
union{
uint16_t val;
char bytes[sizeof(uint16_t)];
} payloadLength;
char *payload;
char *commandRaw;
} aurPacket_t;
Once a packet exists, I decode it with something akin to this:
void decode_command(aurPacket_t packet){
if((packet.commandRaw[0] != '<' ) || (packet.commandRaw[strlen(packet.commandRaw) - 1] != '>') ){
printf("\n\nINVALID COMMAND\n\n");
}
else{
aurPacket_t newpacket;
// EITHER:
// for (int i = 0; i < strlen(newpacket.recipientId.bytes); i++){
// newpacket.recipientId.bytes[i] = (char)*(packet.commandRaw + 1 + i);
// }
// OR:
strncpy(newpacket.recipientId.bytes, (packet.commandRaw + 1), sizeof(newpacket.recipientId.bytes));
}
}
commandRaw contains the char stream that would be received in a message.
Using something like this I'd be able to do it, but I'd need to iterate it one by one since not all values are the same datatype - copying the string out to my payload in the end. Is there a way to make this more elegant than iterating through every single variable, to somehow iterate by using my struct as a guide for the iterator?
I was made aware of memcpy, but since I want to keep the protocol platform-independent, I'd prefer not to use it unless there is an elegant way to. Or is there a way to use it elegantly? Is my method of adding the variables even different from just using memcpy? Thinking about it, it doesn't seem like it would be, considering the ordering of the vars inside my struct. If I made a string containing <, memcpy-appended everything up to the payload, then memcpy-appended the payload, then appended a >, would there be any actual difference in the data? If not, I could just use that process in reverse to decode a message.
I'm currently encoding the message by using this function:
#define RAW_PACKET_LEN 1024
void parse_command(aurPacket_t packet){
snprintf(packet.commandRaw, RAW_PACKET_LEN, "<%.4s%.4s%.1s%.1s%.1s%.02s%.*s>",
packet.recipientId.bytes,
packet.senderId.bytes,
packet.timeToLive.bytes,
packet.cmdType.bytes,
packet.cmdId.bytes,
packet.payloadLength.bytes,
packet.payloadLength.val, packet.payload
);
} // memory for commandRaw allocated outside of function
which has the problem of not actually writing 0x00-bytes to the stream, but that's not part of the question - I'm looking for an answer to that at the moment (of course, if you know an easy fix, feel free to let me know :) )
The system is an ESP32 programmed using ESP-IDF.
Some tips here:
Your packet should not contain a pointer, it would point to an address that only the sender would know about. You want to actually copy the array into the command (see 4 below).
“char bytes[sizeof(uint8_t)]” is literally preprocessed at compile time into “char bytes[1]”, so you might as well just have “char byte”, which is singular.
If you are just using a union for a uint8_t and a char, don’t bother just cast it before you use it: (eg; printf(“%c”,(char)val);
You will save yourself a ton of headache if you can just agree on fixed packet size. It looks like the only thing that will be dynamic is the payload, and the commandraw. Pick your worst case and go with that. I know that you are using serial, but unless you are doing something other than 8N1 it won’t have checks anyway, so you don’t want anything too long. I suggest you pick a total length under 1472 incase you move to UDP/TCP one day.
Assuming you can do these things, copying the data out is a piece of cake. You make your command as a struct. Then you make a union which contains the command as the first member and an array the size of the command as the second member. For example I would used uint8s (you could use chars).
union CommandMsg{
//you could also define this ahead of time..inlining b/c lazy.
struct Command{
uint32_t recipientId;
uint32_t senderId;
uint8_t val;
char timeToLive; //why char?
uint8_t cmdType;
uint8_t cmdId;
uint16_t val;
uint16_t payloadLength; //maybe just strlen later if term'd?
char payload[256];
char commandRaw[256];
} asCommand;
uint8_t asByteArray[sizeof(Command)];
} commandMsg;
Safety first, before you stuff your command struct memcpy zeros into the whole thing. The zeros will act like terminators for any strings you memcpy later.
Make sure to __pack your struct
When you copy it back out cast your arrays to void pointer memcpy((void *)localCopy, (const void *)incoming, sizeof(CommandMsg));
Not really sure what counts as "elegant", but here are some diverse solutions.
The old-fashioned C way to deal with something like this would be to create a look-up table. Given that you only have a static "singleton" buffer static aurPacket_t aur_packet;, then you can create a pointer look-up table at compile-time:
#define AUR_PACKET_MEMBERS_N 6
static char* const AUR_PACKET_LOOKUP[AUR_PACKET_MEMBERS_N] =
{
aur_packet.recipientId.bytes,
aur_packet.senderId.bytes,
aur_packet.timeToLive.bytes,
aur_packet.cmdType.bytes,
aur_packet.cmdId.bytes,
aur_packet.payloadLength.bytes,
};
And now you can iterate over each part of the struct by accessing AUR_PACKET_LOOKUP[i] and get a char* to its bytes member.
A more radical (and not necessarily readable) approach would be "X macros", creating something like
#define AUR_PACKET_LIST \
X(UINT32, recipientId) \
X(UINT32, senderId) \
X(UINT8, timeToLive) \
X(UINT8, cmdType) \
X(UINT8, cmdId) \
X(UINT16, payloadLength) \
You can now generate the string at compile-time, without involving sprintf functions:
strcpy(packet.commandRaw,
#define X(type, member) #type
AUR_PACKET_LIST
#undef X
);
Which expands to strcpy(packet.commandRaw, "UINT32" "UINT32" ... and the pre-processor concatenates the string literals from there.
And finally, it's perfectly possible to go completely macro ape and use X macros to define the type itself too, which I don't really recommend unless you have severe requirements to avoid code repetition:
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef uint8_t UINT8;
typedef uint16_t UINT16;
typedef uint32_t UINT32;
#define AUR_PACKET_LIST(X) \
/* type name */ \
X(UINT32, recipientId) \
X(UINT32, senderId) \
X(UINT8, timeToLive) \
X(UINT8, cmdType) \
X(UINT8, cmdId) \
X(UINT16, payloadLength) \
#define AUR_PACKET_DECL_MEMBER(type, name) \
union { \
type val; \
char bytes[sizeof(type)]; \
} name;
typedef struct{
AUR_PACKET_LIST(AUR_PACKET_DECL_MEMBER)
char *payload;
char *commandRaw;
} aurPacket_t;
int main(void)
{
aurPacket_t packet = {.commandRaw=malloc(256)};
packet.commandRaw[0]='\0';
#define AUR_PACKET_COMMANDRAW(type, name) #type
strcpy(packet.commandRaw, AUR_PACKET_LIST(AUR_PACKET_COMMANDRAW));
puts(packet.commandRaw);
return 0;
}
Your struct looks redundant. It's better to use something like this:
typedef struct {
uint8_t head;
uint32_t recipientId;
uint32_t senderId;
uint8_t timeToLive;
uint8_t cmdType;
uint8_t cmdId;
uint16_t payloadLength;
uint8_t *payload;
uint8_t tail;
} aurPacket_t;
Anyway you should deal with buffer to have memory for payload. It can be global array with fixed size, it's better for embedded software. memcpy is good solution when you controls all the sizes you operates with, because it's usually optimized for target hardware. You should use only fixed size types, like uint8_t and uint32_ for your data members in your packet to make it cross-platform and remember about big- or little-endian on your machines, when you storing data to packet.
Also it's good practice to make check sum a part of your packet to validate data.

Kind of polymorphism in C

I'm writing a C program in which I define two types:
typedef struct {
uint8_t array[32];
/* struct A's members */
...
} A;
typedef struct {
uint8_t array[32];
/* struct B's members, different from A's */
...
} B;
Now I would like to build a data structure which is capable of managing both types without having to write one for type A and one for type B, assuming that both have a uint8_t [32] as their first member.
I read how to implement a sort of polymorphism in C here and I also read here that the order of struct members is guaranteed to be kept by the compiler as written by the programmer.
I came up with the following idea, what if I define the following structure:
typedef struct {
uint8_t array[32];
} Element;
and define a data structure which only deals with data that have type Element? Would it be safe to do something like:
void f(Element * e){
int i;
for(i = 0; i < 32; i++) do_something(e->array[i]);
}
...
A a;
B b;
...
f(((Element *)&a));
...
f(((Element *)&b));
At a first glance it looks unclean, but I was wondering whether there are any guarantees that it will not break?
If array is always the first in your struct, you can simply access it by casting pointers. There is no need for a struct Element. You data structure can store void pointers.
typedef struct {
char array[32];
} A;
typedef struct {
void* elements;
size_t elementSize;
size_t num;
} Vector;
char* getArrayPtr(Vector* v, int i) {
return (char*)(v->elements) + v->elementSize*i;
}
int main()
{
A* pa = malloc(10*sizeof(A));
pa[3].array[0] = 's';
Vector v;
v.elements = pa;
v.num = 10;
v.elementSize = sizeof(A);
printf("%s\n", getArrayPtr(&v, 3));
}
but why not have a function that works with the array directly
void f(uint8_t array[32]){
int i;
for(i = 0; i < 32; i++) do_something(array[i]);
}
and call it like this
f(a.array)
f(b.array)
polymorphism makes sense when you want to kepp
a and b in a container of some sorts
and you want to iterate over them but you dont want to care that they are different types.
This should work fine if you, you know, don't make any mistakes. A pointer to the A struct can be cast to a pointer to the element struct, and so long as they have a common prefix, access to the common members will work just fine.
A pointer to the A struct, which is then cast to a pointer to the element struct can also be cast back to a pointer to the A struct without any problems. If element struct was not originally an A struct, then casting the pointer back to A will be undefined behavior. And this you will need to manage manually.
One gotcha (that I've run into) is, gcc will also allow you to cast the struct back and forth (not just pointer to struct) and this is not supported by the C standard. It will appear to work fine until your (my) friend tries to port the code to a different compiler (suncc) at which point it will break. Or rather, it won't even compile.

Table in struct initialisation in C

How to initialise the table DetectionSensors in this structure:
typedef struct
{
DetectionSensor *DetectionSensors[];
unsigned char nbSensors;
} SENSOR_STRUCT;
SENSOR_STRUCT my_var = { } ?
This table contains just some DetectionSensor pointers;
You can't; the structure definition shown shouldn't compile.
typedef struct
{
DetectionSensor *DetectionSensors[]; // Not C
unsigned char nbSensors;
} SENSOR_STRUCT;
If you're trying for a flexible array member (FAM), that has to be the last field in the structure and you can't write initializers for structure containing a FAM.
Otherwise, you need to use an explicit size for the dimension of the array, or lose the array notation and use DetectionSensor *DetectionsSensors; (or conceivably, but it seems implausible) DetectionSensor **DetectionSensors;.
typedef struct
{
DetectionSensor *DetectionSensors[10]; // Use an enum or #define
unsigned char nbSensors;
} SENSOR_STRUCT;
With this, you need some DetectionSensors around:
DetectionSensor ds[10];
SENSOR_STRUCT my_var = { { &ds[0], &ds[1], &ds[2], &ds[3] }, 4 };
In general, reserve ALL_CAPS for macros (FILE and DIR notwithstanding).
If your intention is to initialise the structure with a predefined set of Detectionsensor, you can do like this.
DetectionSensor sample [] = {sensor1, sensor2];
my_var.DetectionSensors = sample;
There's no automatic constructor for C structs, first you need to build the DetectionSensors array and then assign the value of that array to the DetectionSensor variable in your SENSOR_STRUCT.
typedef struct
{
DetectionSensor * DetectionSensors;
unsigned char nbSensors;
} SENSOR_STRUCT;
DetectionSensor * sensors = ...; //get your detection sensors.
SENSOR_STRUCT my_var = {sensors, ... };
You should reserve some memory for the DetectionSensors, e.g. this way:
#define MAX_SENSORS 10
typedef struct
{
DetectionSensor *DetectionSensors[MAX_SENSORES];
unsigned char nbSensors;
} Sensor_Struct;
Sensor_Struct my_var;
myvar.DetectionSensors[0] = somePointer;
my_var.nbSensors = 1;
btw: CAPS_LOCKED_NAMES are by convention for preprocessor variables (#define SOMETHING abc)
You could provide functions to add a new sensor and even make the memory use dynamic, if you need that.

fixed length structure with variable length reserved space

In the embedded world we often have data structures that are passed around via fixed-length buffers. These are relatively easy to handle using something like this:
#define TOTAL_BUFFER_LENGTH 4096
struct overlay {
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
};
static_assert(sizeof(struct overlay) <= TOTAL_BUFFER_LENGTH);
struct overlay* overlay = malloc(TOTAL_BUFFER_LENGTH);
That is, we use a data structure as an overlay to allow easy access to the part of the buffer that is currently being used.
We have a number of buffer formats, however, that also use the last few bytes of the buffer to store things like checksums. We currently use constructions like this:
struct overlay {
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
char reserved[TOTAL_BUFFER_LENGTH -
sizeof(uint16_t) - sizeof(uint16_t) -
(sizeof(uint8_t) * ARY1_LEN) -
sizeof(uint32_t)];
uint32_t crc;
};
As ugly as this looks for this simple data structure, it's an absolute monstrosity when the structure grows to have dozens of fields. It's also a maintainability nightmare, as adding or removing a structure field means that the size calculation for reserved must be updated at the same time.
When the end of the structure only contains one item (like a checksum), then we sometimes use a helper function for reading/writing the value. That keeps the data structure clean and maintainable, but it doesn't scale well when the end of the buffer has multiple fields.
It would help greatly if we could do something like this instead:
struct overlay {
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
char reserved[TOTAL_BUFFER_LENGTH -
offsetof(struct overlay, reserved) -
sizeof(uint32_t)];
uint32_t crc;
};
Unfortunately, offsetof only works on complete object types and since this is in the middle of the definition of struct overlay, that type isn't yet complete.
Is there a cleaner, more maintainable way to do this sort of thing? I essentially need a fixed-length structure with fields at the beginning and at the end, with the remaining space in the middle reserved/unused.
In your situation, I think I'd probably do things this way:
typedef struct overlay_head
{
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
} overlay_head;
typedef struct overlay_tail
{
uint32_t crc;
} overlay_tail;
enum { OVERLAY_RSVD = TOTAL_BUFFER_LENGTH - sizeof(overlay_head)
- sizeof(overlay_tail) };
typedef struct overlay
{
overlay_head h;
uint8_t reserved[OVERLAY_RSVD];
overlay_tail t;
} overlay;
You can then work almost as before, except that where you used to write p->field1
you now write p->h.field1, and where you used to write p->crc you now write p->t.crc.
Note that this handles arbitrarily large tail structures quite effectively, as long as the head and tail both fit inside the overall size.
You could define a structure that simply has the buffer with a CRC field at the end:
struct checked_buffer {
char data[TOTAL_BUFFER_LENGTH - sizeof(uint32_t)];
uint32_t crc;
};
and then place your "overlays" on its data field. You're presumably already casting pointers to "convert" a raw buffer's char* into an overlay*, so it shouldn't be a big deal to cast from overlay* to checked_buffer* when you want to access the CRC field.
But if you want to have a field in a consistent position across a bunch of structures, it'd be easier to put it at the beginning of each structure. That way you can declare it directly in each structure without needing to do anything strange, and you don't need any pointer casts to access it.
How about that?
union a256
{
struct
{
int field_a;
int field_b;
char name[16];
//
int crcshadow;
};
struct
{
char buff[256-sizeof(int)];
int crc;
};
} ;
static_assert(offsetof(a256, crcshadow) < offsetof(a256, crc), "data too big");
The first struct contains data, the second define fixed size for this union.

Resources