fixed length structure with variable length reserved space - c

Question

In the embedded world we often have data structures that are passed around via fixed-length buffers. These are relatively easy to handle using something like this:
#define TOTAL_BUFFER_LENGTH 4096
struct overlay {
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
};
static_assert(sizeof(struct overlay) <= TOTAL_BUFFER_LENGTH);
struct overlay* overlay = malloc(TOTAL_BUFFER_LENGTH);
That is, we use a data structure as an overlay to allow easy access to the part of the buffer that is currently being used.
We have a number of buffer formats, however, that also use the last few bytes of the buffer to store things like checksums. We currently use constructions like this:
struct overlay {
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
char reserved[TOTAL_BUFFER_LENGTH -
sizeof(uint16_t) - sizeof(uint16_t) -
(sizeof(uint8_t) * ARY1_LEN) -
sizeof(uint32_t)];
uint32_t crc;
};
As ugly as this looks for this simple data structure, it's an absolute monstrosity when the structure grows to have dozens of fields. It's also a maintainability nightmare, as adding or removing a structure field means that the size calculation for reserved must be updated at the same time.
When the end of the structure only contains one item (like a checksum), then we sometimes use a helper function for reading/writing the value. That keeps the data structure clean and maintainable, but it doesn't scale well when the end of the buffer has multiple fields.
It would help greatly if we could do something like this instead:
struct overlay {
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
char reserved[TOTAL_BUFFER_LENGTH -
offsetof(struct overlay, reserved) -
sizeof(uint32_t)];
uint32_t crc;
};
Unfortunately, offsetof only works on complete object types and since this is in the middle of the definition of struct overlay, that type isn't yet complete.
Is there a cleaner, more maintainable way to do this sort of thing? I essentially need a fixed-length structure with fields at the beginning and at the end, with the remaining space in the middle reserved/unused.

Answer 1

In your situation, I think I'd probably do things this way:
typedef struct overlay_head
{
uint16_t field1;
uint16_t field2;
uint8_t array1[ARY1_LEN];
} overlay_head;
typedef struct overlay_tail
{
uint32_t crc;
} overlay_tail;
enum { OVERLAY_RSVD = TOTAL_BUFFER_LENGTH - sizeof(overlay_head)
- sizeof(overlay_tail) };
typedef struct overlay
{
overlay_head h;
uint8_t reserved[OVERLAY_RSVD];
overlay_tail t;
} overlay;
You can then work almost as before, except that where you used to write p->field1
you now write p->h.field1, and where you used to write p->crc you now write p->t.crc.
Note that this handles arbitrarily large tail structures quite effectively, as long as the head and tail both fit inside the overall size.

Answer 2

You could define a structure that simply has the buffer with a CRC field at the end:
struct checked_buffer {
char data[TOTAL_BUFFER_LENGTH - sizeof(uint32_t)];
uint32_t crc;
};
and then place your "overlays" on its data field. You're presumably already casting pointers to "convert" a raw buffer's char* into an overlay*, so it shouldn't be a big deal to cast from overlay* to checked_buffer* when you want to access the CRC field.
But if you want to have a field in a consistent position across a bunch of structures, it'd be easier to put it at the beginning of each structure. That way you can declare it directly in each structure without needing to do anything strange, and you don't need any pointer casts to access it.

Answer 3

How about that?
union a256
{
struct
{
int field_a;
int field_b;
char name[16];
//
int crcshadow;
};
struct
{
char buff[256-sizeof(int)];
int crc;
};
} ;
static_assert(offsetof(a256, crcshadow) < offsetof(a256, crc), "data too big");
The first struct contains data, the second define fixed size for this union.