I have a struct defined that is used for messages sent across two different interfaces. One of them requires 32-bit alignment, but I need to minimize the space they take. Essentially I'm trying to byte-pack the structs, i.e. #pragma pack(1) but ensure that the resulting struct is a multiple of 32-bits long. I'm using a gcc arm cross-compiler for a 32-bit M3 processor. What I think I want to do is something like this:
#pragma pack(1)
typedef struct my_type_t
{
uint32_t someVal;
uint8_t anotherVal;
uint8_t reserved[<??>];
}
#pragma pack()
where <??> ensures that the size of my_type_t is divisible by 4 bytes, but without hard-coding the padding size. I can do something like this:
#pragma pack(1)
typedef struct wrapper_t
{
my_type_t m;
uint8_t reserved[sizeof(my_type_t) + 4 - (sizeof(my_type_t) % 4)]
}
#pragma pack()
but I'd like to avoid that.
Ultimately what I need to do is copy this to a buffer that is 32-bit addressable, like:
static my_type_t t; //If it makes a difference, this will be declared statically in my C source file
...
memcpy(bufferPtr, (uint32_t*)&t, sizeof(t)) //or however I should do this
I've looked at the __attribute__((align(N))) attribute, which gives me the 32-bit aligned memory address for the struct, but it does not byte-pack it. I am confused about how (or if) this can be combined with pack(1).
My question is this:
What is the right way to declare these structs so that I can minimize their footprint in memory but that allows me to copy/set it in 4-byte increments with a unsigned 32-bit pointer? (There are a bunch of these types of arbitrary size and content). If my approach above of combining pack and padding is going about this totally wrong, I'll happily take alternatives.
Edit:
Some constraints: I do not have control over one of the interfaces. It is expecting byte-packed frames. The other side is 32-bit addressable memory mapped registers. I have 64k of memory for the entire executable, and I'm limited on the libraries etc. I can bring in. There is already a great deal of space optimization I've had to do.
The struct in this question was just to explain my question. I have numerous messages of varying content that this applies to.
I can't speak for the specific compiler and architecture you are using, but I would expect the following to be sufficient:
typedef struct {
uint32_t x;
uint8_t y;
} my_type_t;
The structure normally has the same alignment as its largest field, and that includes adding the necessary padding at the end.
my_type_t
+---------------+
| x |
+---+-----------+
| y | [padding] |
+---+-----------+
|<-- 32 bits -->|
Demo
This is done so the fields are properly aligned when you have an array of them.
my_type_t my_array[2];
my_array[1].x = 123; // Needs to be properly aligned.
The above assumes you have control over the order of the fields to get the best space efficiency, because it relies on the compiler aligning the individual fields. But those assumptions can be removed using GCC attributes.
typedef struct {
uint8_t x;
uint32_t y;
uint8_t z;
}
__attribute__((packed)) // Remove interfield padding.
__attribute__((aligned(4))) // Set alignment and add tail padding.
my_type_t;
This produces this:
my_type_t
+---+-----------+
| x | y
+---+---+-------+
| z | [pad] |
+---+---+-------+
|<-- 32 bits -->|
Demo
The packed attribute prevents padding from being added between fields, but aligning the structure to a 32-bit boundary forces the alignment you desire. This has the side effect of adding trailing padding so you can safely have an array of these structures.
As you use gcc you need to use one of the attributes.
Example + demo.
#define PACKED __attribute__((packed))
#define ALIGN(n) __attribute__((aligned(n)))
typedef struct
{
uint8_t anotherVal;
uint32_t someVal;
}PACKED my_type_t;
my_type_t t = {1, 5};
ALIGN(64) my_type_t t1 = {1, 5};
ALIGN(512) my_type_t t2 = {2, 6};
int main()
{
printf("%p, %p, %p", (void *)&t, (void *)&t1, (void *)&t2);
}
Result:
0x404400, 0x404440, 0x404600
https://godbolt.org/z/j9YjqzEYW
I suggest combining #pragma pack with alignas:
#include <stdalign.h>
#include <stdint.h>
typedef struct {
#pragma pack(1)
alignas(4) struct { // requires 2+1+2 bytes but is aligned to even 4:s
uint16_t someVal; // +0
uint8_t anotherVal; // +2
uint16_t foo; // +3 (would be 4 without packing)
};
#pragma pack()
} my_type_t;
The anonymous inside struct makes access easy as before:
int main() {
my_type_t y;
y.someVal = 10;
y.anotherVal = 'a';
y.foo = 20;
printf("%zu\n", (char*)&y.someVal - (char*)&y.someVal); // 0
printf("%zu\n", (char*)&y.anotherVal - (char*)&y.someVal); // 2
printf("%zu\n", (char*)&y.foo - (char*)&y.someVal); // 3
my_type_t x[2];
printf("%zu\n", (char*)&x[1] - (char*)&x[0]); // 8 bytes diff
}
If you'd like to be able to take the sizeof the actual data carrying part of my_type_t (to send it), you could name the inner struct (which makes accessing the fields a little more cumbersome):
#pragma pack(1)
typedef struct {
uint16_t someVal;
uint8_t anotherVal;
uint16_t foo;
} inner;
#pragma pack()
typedef struct {
alignas(4) inner i;
} my_type_t;
You'd now have to mention i to access the fields, but it has the benefit that you can take sizeof and get 5 (in this example):
int main() {
my_type_t y;
printf("%zu %zu\n", sizeof y, alignof(y)); // 8 4
printf("%zu\n", sizeof y.i); // 5 (the actual data)
}
To form a structure type that is aligned one must put the alignment attribute to the first member of the struct. It can be combined with the packed attribute.
typedef struct {
_Alignas(4) uint8_t anotherVal;
uint32_t someVal;
} __attribute__((packed)) my_type_t;
Exemplary usage with alignment exaggerated to 64 bytes.
#include <stdint.h>
#include <stdio.h>
typedef struct {
_Alignas(64) uint8_t anotherVal;
uint32_t someVal;
} __attribute__((packed)) my_type_t;
int main() {
my_type_t a, b;
printf("%zu %p\n", sizeof a, (void*)&a);
printf("%zu %p\n", sizeof b, (void*)&b);
}
prints:
64 0x7ffff26caf80
64 0x7ffff26cafc0
Related
Let's say I have a struct and variables in C like this:
typedef struct {
uint8_t x;
uint8_t y;
uint8_t z;
}my_type;
my_type a;
my_type a10[10];
With C99 is it certain that
sizeof(a) == 3? (and not 4 or 8)
sizeof(a10) == 30? (and not 40 or 80)
With C99 is it certain that sizeof(a) == 3?
This type will be used as color for an RGB888 framebuffer. So it's important to NOT have space between the adjacent RGB values.
typedef struct {
uint8_t x;
uint8_t y;
uint8_t z;
}my_type;
No. The size may be 3 or 4 (or theoretically others, but not likely)
2 solutions:
Give up portability and use compiler specif code to pack the structure.
Change struct to an array of 3 uint8_t and adjust code.
I recommend the last.
As mentioned in the comments, padding could be added by default.
To avoid that you could use __attribute__((packed)) for your struct.
This will tell the compiler to pack your struct as tight as possible.
Would look like this:
typedef struct {
uint8_t x;
uint8_t y;
uint8_t z;
} __attribute__((packed)) my_type;
I have a struct
typedef struct
{
int A ;
int B ;
…
} SomeStruct ;
I have an instance of SomeStruct that I want to persist to Flash memory which has a sector size of 512 bytes. What GCC attribute can I apply to that instance such that the allocation size is a multiple of 512 bytes?
The only options I can think of are :-
1) Pad the struct to make it exactly 512 bytes. This requires recalculation every time a field is added to the struct. No compiler warnings when I get it wrong. Also struct is larger than needed with normal initialisation, copying etc.
2) Place the variable in a separate Linker section. This provides full protection and warnings, but gets a bit tedious if multiple variables are used.
3) Make a union of the struct and a 512 byte array. Copes with adding extra fields until the struct is greater than 512 bytes, then fails without any warnings.
Referring 1:
#include <assert.h>
#define FLASH_BYTES (512)
#pragma pack(1)
struct flash
{
struct data
{
int i;
char c;
...
};
char pads[FLASH_BYTES - sizeof (struct data)];
};
#pragma pack()
int main(void)
{
assert(sizeof (struct flash) == FLASH_BYTES);
...
The assert might even not be necessary because if the result
FLASH_BYTES - sizeof (struct data)
is negative any GCC should issue an error. To make sure it will be negative cast the result of the sizeof operation to any signed integer, like for example so:
FLASH_BYTES - (int) sizeof (struct data)
So trying to compile this
#pragma pack(1)
struct flash
{
struct data
{
int i;
char c[FLASH_BYTES];
};
char pads[FLASH_BYTES - (int) (sizeof (struct data))];
};
#pragma pack()
int main(void)
{
}
You should be giving you something like:
main.c:14:12: error: size of array ‘pads’ is negative
char pads[FLASH_BYTES - (int) sizeof (struct data)];
A portable solution is to define a union of SomeStruct, with a char array whose size is calculated to meet the necessary alignment.
typedef struct
{
int A;
int B;
char c[512];
} SomeStruct;
#define STORAGE_ALIGNMENT 512
typedef union
{
SomeStruct data;
char paddedData[((sizeof(SomeStruct) + STORAGE_ALIGNMENT - 1) / STORAGE_ALIGNMENT) * STORAGE_ALIGNMENT];
} SomeStructAligned;
Online running version (Coliru) here
The sizing formula is well known and works for any integer. Since this is a power-of-2 you could also simplify it down to the form (sizeof(SomeStruct) + (STORAGE_ALIGNMENT - 1)) & ~(STORAGE_ALIGNMENT - 1)) == (sizeof(SomeStruct) + 0x1ff) & ~0x1ff). In practice you may need ~size_t(0x1ff) on the rightmost term to ensure portability to 64-bit machines; since 0x1ff is an int (32-bit), ~0x1ff results in a 64-bit 0x00000000fffffe00 value instead of the desired 0xFFFFFFFFfffffe00 mask.
Sub-Optimal Approach
An alternative approach could have been to define a wrapper struct containing your original data plus some automatically calculated padding.
typedef struct
{
int A;
int B;
} SomeStruct;
#define STORAGE_ALIGNMENT 512
typedef struct
{
SomeStruct data;
char padding[(STORAGE_ALIGNMENT) - (sizeof(SomeStruct) % STORAGE_ALIGNMENT)];
} SomeStructAligned;
Online running version (Coliru) here.
However, the above is not perfect: if sizeof(SomeStruct) is a multiple of 512, then sizeof(padding) will be 512, wasting a quantum of storage. Whereas the union never wastes space.
You can try something like this (though it is a dirty bit trick)
#define ROUND_UP_512(x) ((x) + 511 & ~511)
struct myStruct {
// put whatever
};
union myUnion{
myStruct s;
char ensureSize[ROUND_UP_512(sizeof(myStruct))];
};
in this case the size of "myUnion" is guaranteed to be a multiple of 512 that is greater than or equal to the size of "myStruct"
FYI: This is my first question using stackoverflow!
The code is as follows:
uint8_t TestVar1;
uint8_t TestVar2;
typedef struct
{
uint8_t member1;
uint32_t member2;
}Test1;
typedef struct
{
uint8_t member1;
uint8_t member2[4];
}Test2;
Test1 TestStruct1;
Test2 TestStruct2;
TestVar1 = sizeof(TestStruct1); /*size is 8*/
TestVar2 = sizeof(TestStruct2); /*size is 5*/
I thought I understood padding but I can't explain TestVar2. I can explain TestVar1 being 8 bytes because there is 3 padding bytes as part of uint8_t member1.
However, with struct test2 are there no padding bytes? (Apparently not) Could someone provide some insight as to what is happening in struct test2 case?
As a side note, I am aiming for 5 bytes but I don't know why the second case works. Is the array breaking down to a pointer or something? Is this safe(standard practice) to do?
Thanks!
uint8_t has no specific alignment requirements and thus does not require any padding to align it while uint32_t, being a multi-byte type, will want to be aligned on a 4-byte boundary. If you absolutely must have a struct with a uint8_t and uint32_t that takes only 5 bytes, you can use __attribute__((packed)) to tell the compiler to forego the padding (which I would carefully consider whether the space saved is worth misalignment):
typedef struct __attribute__((packed)) {
uint8_t member1;
uint32_t member2;
}test;
Another thing to consider is the ordering of the struct members. To keep a reduced struct size, put the largest members at the beginning of the struct as they will have the strictest alignment needs. Consider the following:
typedef struct {
uint8_t w;
uint32_t x;
uint8_t y;
uint32_t z;
}test2;
typedef struct {
uint32_t x;
uint32_t z;
uint8_t w;
uint8_t y;
}test3;
In order to keep the uint32_t members aligned, test2 will put padding after w and y, while test3 will already have the uint32_t members aligned and only put enough padding at the end of the struct so that the next free memory segment aligns with a 4-bytes boundary (as that is the strictest boundary of any of the struct's members). Therefore test2 will have a size of 16 bytes while test3 will have a size of 12 bytes.
Note: I have not seen where, if at all, this is defined in the C specs, but the results hold true for both gcc and clang in my tests.
I'm using packed structure for communication using direct DMA access, and here is my test code:
// structure for communication buf 1
typedef __packed struct _test1
{
uint8_t a;
uint32_t b;
uint16_t c;
uint16_t d;
uint32_t e;
} test1;
// structure for communication buf 2
.
.
.
// structure for communication buf 3
.
.
.
// structure for communication buf set
typedef __packed struct _test2
{
uint8_t dump[3];
test1 t;
// may have many other packed structure for communication buf
} test2;
#pragma anon_unions
typedef struct _test3
{
union
{
uint32_t buf[4];
__packed struct
{
__packed uint8_t dump[3];
test1 t;
};
};
} test3;
test1 t1;
test2 t2;
test3 t3;
size of these structures are
sizeof(t1) = 13
sizeof(t2) = 16
sizeof(t3) = 16
if I want to access variable b, for not effecting performance, read/write memory content with aligned access is needed, with calculated offset by hand
t3.buf[1]
but I cannot read/write variables in structure without using unaligned accesses
t2.t.b
t3.t.b
so I defined structures like the following code, packed only variable a
typedef struct _test4
{
__packed uint8_t a;
uint32_t b;
uint16_t c;
uint16_t d;
uint32_t e;
} test4;
typedef struct _test5
{
__packed uint8_t dump[3];
test4 t;
} test5;
test4 t4;
test5 t5;
although access of all element in structure is aligned, but padding is inserted either
sizeof(t4) = 16
sizeof(t5) = 20
so how can I define packed structures, and access single variable in it without using unaligned access(except a)?
thanks a lot for helping
Your question introduces two problems under the umbrella of one:
Communication between components and/or devices; this may or may not have the same underlying representation of structures and integers, hence your use of the non-portable __packed attribute.
Performance of access, biased by alignment and/or data size; on one hand the compiler aligns data to fall in line with the bus, yet on the other hand that data might occupy too much space in your cache.
One of these is the actual problem you want to solve, X, and the other the Y in your XY problem. Please avoid asking XY problems in the future.
Have you considered how to guarantee that uint16_t and uint32_t will be big endian or little endian, based on your requirements? You need to specify that, if you care about portability. I care about portability, so that's what my answer will focus on. You may also notice how optimal efficiency will be obtained, too. Nonetheless, to make this portable:
You should be serialising your data using serialisation functions to convert each member of your structure into sequences of bytes by division and modulo (or left shift and binary and) operations.
Similarly, you should be deserialising your data by inverse operations multiplication and addition (or right shift and binary or).
As an example, here's some code showing both little endian and big endian for serialising and deserialising test1:
typedef /*__packed*/ struct test1
{
uint32_t b;
uint32_t e;
uint16_t c;
uint16_t d;
uint8_t a;
} test1;
void serialise_test1(test1 *destination, void *source) {
uint8_t *s = source;
destination->a = s[0];
destination->b = s[1] * 0x01000000UL
+ s[2] * 0x00010000UL
+ s[3] * 0x00000100UL
+ s[4]; /* big endian */
destination->c = s[5] * 0x0100U
+ s[6]; /* big endian */
destination->d = s[7]
+ s[8] * 0x0100U; /* little endian */
destination->e = s[9]
+ s[10] * 0x00000100UL
+ s[11] * 0x00010000UL
+ s[12] * 0x01000000UL; /* little endian */
}
void deserialise_test1(void *destination, test1 *source) {
uint8_t temp[] = { source->a
, source->b >> 24, source->b >> 16
, source->b >> 8, source->b
, source->c >> 8, source->c
, source->d, source->d >> 8
, source->d >> 16, source->b >> 24 };
memcpy(destination, temp, sizeof temp);
}
You may notice that I removed the __packed attribute and rearranged the members, so that the larger members precede (i.e. come before) the smaller; this is likely to reduce padding significantly. The functions allow you to convert between an array of uint8_t (which you send to/receive from the wire, or DMA, or whatnot) and your test1 structure, so this code is much more portable. You benefit from the guarantees this code provides regarding the structure of your protocol, where-as before it was at the whim of the implementation, and two devices using two different implementations might have disagreed about the internal representation of integers for example.
You could hard code all the indexes like
typedef __packed struct _test1
{
uint8_t a;
uint32_t b;
uint16_t c;
uint16_t d;
uint32_t e;
} test1;
enum
{
a = 0,
b = 1,
c = 5,
d = 7,
e = 9,
};
test1 t1 = {1,2,3,4};//not sure if init lists work for packed values
printf("%u", *(uint32_t*)((uint8_t*)&t1 + b));
Or offsetof can be used like this
printf("%u", *(uint32_t*)((uint8_t*)&t1 + offsetof(test1, b)));
I am using ARM. I got alignment fault due to read/write in odd offset(we knew ARM is 4 byte aligned). All the structs defined in my program is single - byte aligned like
#pragma pack(push, 1)
typedef struct
{
char a1;
int a2;
char a3;
}a;
#pragma pack(pop)
I am trying to do
#pragma pack(push, 1)
typedef struct
{
char a1 __attribute__ ((aligned (4)));
int a2;
char a3;
}a;
#pragma pack(pop)
the gcc attribute _attribute_ ((aligned (4))) makes no effect.
Note :: The above code is not my actual code. sample scenario.
so I re-arranged structure member to solve the alignment issue. I want to ensure whether the re-arranging is the possible solution or we can make _attribute_ to work on this scenario. Any other solutions are welcome. Thanks in advance.
You can safely read/write char/int in byte-aligned structs on ARM, compiler take care about alignment. Single problem can occur with alignment issue is with the casting to 32 bit int like this:
char buf[5];
char *p = buf + 1;
*((int *)p) = 1;
Note: if you for some reasons like align some member (from begin of the struct) you can use following trick:
typedef struct {
struct {} __attribute__ ((aligned (4)));
char a1; // offset is 4
int a2; // offset is 5
char a3; // offset is 9
} a;