Let's say I have a struct and variables in C like this:
typedef struct {
uint8_t x;
uint8_t y;
uint8_t z;
}my_type;
my_type a;
my_type a10[10];
With C99 is it certain that
sizeof(a) == 3? (and not 4 or 8)
sizeof(a10) == 30? (and not 40 or 80)
With C99 is it certain that sizeof(a) == 3?
This type will be used as color for an RGB888 framebuffer. So it's important to NOT have space between the adjacent RGB values.
typedef struct {
uint8_t x;
uint8_t y;
uint8_t z;
}my_type;
No. The size may be 3 or 4 (or theoretically others, but not likely)
2 solutions:
Give up portability and use compiler specif code to pack the structure.
Change struct to an array of 3 uint8_t and adjust code.
I recommend the last.
As mentioned in the comments, padding could be added by default.
To avoid that you could use __attribute__((packed)) for your struct.
This will tell the compiler to pack your struct as tight as possible.
Would look like this:
typedef struct {
uint8_t x;
uint8_t y;
uint8_t z;
} __attribute__((packed)) my_type;
Related
I have a struct defined that is used for messages sent across two different interfaces. One of them requires 32-bit alignment, but I need to minimize the space they take. Essentially I'm trying to byte-pack the structs, i.e. #pragma pack(1) but ensure that the resulting struct is a multiple of 32-bits long. I'm using a gcc arm cross-compiler for a 32-bit M3 processor. What I think I want to do is something like this:
#pragma pack(1)
typedef struct my_type_t
{
uint32_t someVal;
uint8_t anotherVal;
uint8_t reserved[<??>];
}
#pragma pack()
where <??> ensures that the size of my_type_t is divisible by 4 bytes, but without hard-coding the padding size. I can do something like this:
#pragma pack(1)
typedef struct wrapper_t
{
my_type_t m;
uint8_t reserved[sizeof(my_type_t) + 4 - (sizeof(my_type_t) % 4)]
}
#pragma pack()
but I'd like to avoid that.
Ultimately what I need to do is copy this to a buffer that is 32-bit addressable, like:
static my_type_t t; //If it makes a difference, this will be declared statically in my C source file
...
memcpy(bufferPtr, (uint32_t*)&t, sizeof(t)) //or however I should do this
I've looked at the __attribute__((align(N))) attribute, which gives me the 32-bit aligned memory address for the struct, but it does not byte-pack it. I am confused about how (or if) this can be combined with pack(1).
My question is this:
What is the right way to declare these structs so that I can minimize their footprint in memory but that allows me to copy/set it in 4-byte increments with a unsigned 32-bit pointer? (There are a bunch of these types of arbitrary size and content). If my approach above of combining pack and padding is going about this totally wrong, I'll happily take alternatives.
Edit:
Some constraints: I do not have control over one of the interfaces. It is expecting byte-packed frames. The other side is 32-bit addressable memory mapped registers. I have 64k of memory for the entire executable, and I'm limited on the libraries etc. I can bring in. There is already a great deal of space optimization I've had to do.
The struct in this question was just to explain my question. I have numerous messages of varying content that this applies to.
I can't speak for the specific compiler and architecture you are using, but I would expect the following to be sufficient:
typedef struct {
uint32_t x;
uint8_t y;
} my_type_t;
The structure normally has the same alignment as its largest field, and that includes adding the necessary padding at the end.
my_type_t
+---------------+
| x |
+---+-----------+
| y | [padding] |
+---+-----------+
|<-- 32 bits -->|
Demo
This is done so the fields are properly aligned when you have an array of them.
my_type_t my_array[2];
my_array[1].x = 123; // Needs to be properly aligned.
The above assumes you have control over the order of the fields to get the best space efficiency, because it relies on the compiler aligning the individual fields. But those assumptions can be removed using GCC attributes.
typedef struct {
uint8_t x;
uint32_t y;
uint8_t z;
}
__attribute__((packed)) // Remove interfield padding.
__attribute__((aligned(4))) // Set alignment and add tail padding.
my_type_t;
This produces this:
my_type_t
+---+-----------+
| x | y
+---+---+-------+
| z | [pad] |
+---+---+-------+
|<-- 32 bits -->|
Demo
The packed attribute prevents padding from being added between fields, but aligning the structure to a 32-bit boundary forces the alignment you desire. This has the side effect of adding trailing padding so you can safely have an array of these structures.
As you use gcc you need to use one of the attributes.
Example + demo.
#define PACKED __attribute__((packed))
#define ALIGN(n) __attribute__((aligned(n)))
typedef struct
{
uint8_t anotherVal;
uint32_t someVal;
}PACKED my_type_t;
my_type_t t = {1, 5};
ALIGN(64) my_type_t t1 = {1, 5};
ALIGN(512) my_type_t t2 = {2, 6};
int main()
{
printf("%p, %p, %p", (void *)&t, (void *)&t1, (void *)&t2);
}
Result:
0x404400, 0x404440, 0x404600
https://godbolt.org/z/j9YjqzEYW
I suggest combining #pragma pack with alignas:
#include <stdalign.h>
#include <stdint.h>
typedef struct {
#pragma pack(1)
alignas(4) struct { // requires 2+1+2 bytes but is aligned to even 4:s
uint16_t someVal; // +0
uint8_t anotherVal; // +2
uint16_t foo; // +3 (would be 4 without packing)
};
#pragma pack()
} my_type_t;
The anonymous inside struct makes access easy as before:
int main() {
my_type_t y;
y.someVal = 10;
y.anotherVal = 'a';
y.foo = 20;
printf("%zu\n", (char*)&y.someVal - (char*)&y.someVal); // 0
printf("%zu\n", (char*)&y.anotherVal - (char*)&y.someVal); // 2
printf("%zu\n", (char*)&y.foo - (char*)&y.someVal); // 3
my_type_t x[2];
printf("%zu\n", (char*)&x[1] - (char*)&x[0]); // 8 bytes diff
}
If you'd like to be able to take the sizeof the actual data carrying part of my_type_t (to send it), you could name the inner struct (which makes accessing the fields a little more cumbersome):
#pragma pack(1)
typedef struct {
uint16_t someVal;
uint8_t anotherVal;
uint16_t foo;
} inner;
#pragma pack()
typedef struct {
alignas(4) inner i;
} my_type_t;
You'd now have to mention i to access the fields, but it has the benefit that you can take sizeof and get 5 (in this example):
int main() {
my_type_t y;
printf("%zu %zu\n", sizeof y, alignof(y)); // 8 4
printf("%zu\n", sizeof y.i); // 5 (the actual data)
}
To form a structure type that is aligned one must put the alignment attribute to the first member of the struct. It can be combined with the packed attribute.
typedef struct {
_Alignas(4) uint8_t anotherVal;
uint32_t someVal;
} __attribute__((packed)) my_type_t;
Exemplary usage with alignment exaggerated to 64 bytes.
#include <stdint.h>
#include <stdio.h>
typedef struct {
_Alignas(64) uint8_t anotherVal;
uint32_t someVal;
} __attribute__((packed)) my_type_t;
int main() {
my_type_t a, b;
printf("%zu %p\n", sizeof a, (void*)&a);
printf("%zu %p\n", sizeof b, (void*)&b);
}
prints:
64 0x7ffff26caf80
64 0x7ffff26cafc0
Lets take a look at the following structs:
struct child {
int a:1;
int b:2;
int c:2;
} __attribute__((packed));
struct parent1 {
int x:3;
struct child y;
} __attribute__((packed));
struct parent2 {
int p:1;
int q:5;
int r:5;
struct child s;
} __attribute__((packed));
These are the sizes I am getting:
sizeof(int) 4
sizeof(struct child) 1
sizeof(struct parent1) 2
sizeof(struct parent2) 3
I've heard that padding is added before structs for performance reasons.
But forgetting about performance for a moment,
is there a way so that I can get the following sizes?
sizeof(struct parent1) 1
sizeof(struct parent2) 2
As only that much of memory is actually required...
EDIT
Is there any way of doing it with gcc on linux?
No it is not possible to pack structures tighter than what your compiler does.
Every struct must start on a byte boundary, so the members s and y cannot use the available bits from the previous members of their enclosing struct definition.
Also note that __attribute__((packed)) is an extension that might not be supported on many compilers.
If all else fails, you can always write functions and macros to bitshift & extract what you need.
Using unions gets close to what you want. It may be possible to use C++ to clean up the syntax, but with C, this is the closest (non-bitwise) solution I could come up with:
(note parent1 is near perfect.)
#pragma pack(1)
typedef struct {
char x:3;
char a:3;
char b:3;
char c:3;
} child;
typedef union {
char x:3;
child y;
} parent1;
typedef struct {
short p:1;
short q:5;
short r:5;
short s:5;
} par2;
typedef struct {
char pad;
child s;
} padchild;
typedef union {
par2 parent2;
padchild s;
} parent2;
#pragma pop
Technically unions are for either-or use and compilers can pad however they want, but by forcing the bit counts to be the same, easiest way for the compiler to implement happens to be what you want.
FYI: This is my first question using stackoverflow!
The code is as follows:
uint8_t TestVar1;
uint8_t TestVar2;
typedef struct
{
uint8_t member1;
uint32_t member2;
}Test1;
typedef struct
{
uint8_t member1;
uint8_t member2[4];
}Test2;
Test1 TestStruct1;
Test2 TestStruct2;
TestVar1 = sizeof(TestStruct1); /*size is 8*/
TestVar2 = sizeof(TestStruct2); /*size is 5*/
I thought I understood padding but I can't explain TestVar2. I can explain TestVar1 being 8 bytes because there is 3 padding bytes as part of uint8_t member1.
However, with struct test2 are there no padding bytes? (Apparently not) Could someone provide some insight as to what is happening in struct test2 case?
As a side note, I am aiming for 5 bytes but I don't know why the second case works. Is the array breaking down to a pointer or something? Is this safe(standard practice) to do?
Thanks!
uint8_t has no specific alignment requirements and thus does not require any padding to align it while uint32_t, being a multi-byte type, will want to be aligned on a 4-byte boundary. If you absolutely must have a struct with a uint8_t and uint32_t that takes only 5 bytes, you can use __attribute__((packed)) to tell the compiler to forego the padding (which I would carefully consider whether the space saved is worth misalignment):
typedef struct __attribute__((packed)) {
uint8_t member1;
uint32_t member2;
}test;
Another thing to consider is the ordering of the struct members. To keep a reduced struct size, put the largest members at the beginning of the struct as they will have the strictest alignment needs. Consider the following:
typedef struct {
uint8_t w;
uint32_t x;
uint8_t y;
uint32_t z;
}test2;
typedef struct {
uint32_t x;
uint32_t z;
uint8_t w;
uint8_t y;
}test3;
In order to keep the uint32_t members aligned, test2 will put padding after w and y, while test3 will already have the uint32_t members aligned and only put enough padding at the end of the struct so that the next free memory segment aligns with a 4-bytes boundary (as that is the strictest boundary of any of the struct's members). Therefore test2 will have a size of 16 bytes while test3 will have a size of 12 bytes.
Note: I have not seen where, if at all, this is defined in the C specs, but the results hold true for both gcc and clang in my tests.
I am using ARM. I got alignment fault due to read/write in odd offset(we knew ARM is 4 byte aligned). All the structs defined in my program is single - byte aligned like
#pragma pack(push, 1)
typedef struct
{
char a1;
int a2;
char a3;
}a;
#pragma pack(pop)
I am trying to do
#pragma pack(push, 1)
typedef struct
{
char a1 __attribute__ ((aligned (4)));
int a2;
char a3;
}a;
#pragma pack(pop)
the gcc attribute _attribute_ ((aligned (4))) makes no effect.
Note :: The above code is not my actual code. sample scenario.
so I re-arranged structure member to solve the alignment issue. I want to ensure whether the re-arranging is the possible solution or we can make _attribute_ to work on this scenario. Any other solutions are welcome. Thanks in advance.
You can safely read/write char/int in byte-aligned structs on ARM, compiler take care about alignment. Single problem can occur with alignment issue is with the casting to 32 bit int like this:
char buf[5];
char *p = buf + 1;
*((int *)p) = 1;
Note: if you for some reasons like align some member (from begin of the struct) you can use following trick:
typedef struct {
struct {} __attribute__ ((aligned (4)));
char a1; // offset is 4
int a2; // offset is 5
char a3; // offset is 9
} a;
I realize that what I am trying to do isn't safe. But I am just doing some testing and image processing so my focus here is on speed.
Right now this code gives me the corresponding bytes for a 32-bit pixel value type.
struct Pixel {
unsigned char b,g,r,a;
};
I wanted to check if I have a pixel that is under a certain value (e.g. r, g, b <= 0x10). I figured I wanted to just conditional-test the bit-and of the bits of the pixel with 0x00E0E0E0 (I could have wrong endianness here) to get the dark pixels.
Rather than using this ugly mess (*((uint32_t*)&pixel)) to get the 32-bit unsigned int value, i figured there should be a way for me to set it up so I can just use pixel.i, while keeping the ability to reference the green byte using pixel.g.
Can I do this? This won't work:
struct Pixel {
unsigned char b,g,r,a;
};
union Pixel_u {
Pixel p;
uint32_t bits;
};
I would need to edit my existing code to say pixel.p.g to get the green color byte. Same happens if I do this:
union Pixel {
unsigned char c[4];
uint32_t bits;
};
This would work too but I still need to change everything to index into c, which is a bit ugly but I can make it work with a macro if i really needed to.
(Edited) Both gcc and MSVC allow 'anonymous' structs/unions, which might solve your problem. For example:
union Pixel {
struct {unsigned char b,g,r,a;};
uint32_t bits; // use 'unsigned' for MSVC
}
foo.b = 1;
foo.g = 2;
foo.r = 3;
foo.a = 4;
printf ("%08x\n", foo.bits);
gives (on Intel):
04030201
This requires changing all your declarations of struct Pixel to union Pixel in your original code. But this defect can be fixed via:
struct Pixel {
union {
struct {unsigned char b,g,r,a;};
uint32_t bits;
};
} foo;
foo.b = 1;
foo.g = 2;
foo.r = 3;
foo.a = 4;
printf ("%08x\n", foo.bits);
This also works with VC9, with 'warning C4201: nonstandard extension used : nameless struct/union'. Microsoft uses this trick, for example, in:
typedef union {
struct {
DWORD LowPart;
LONG HighPart;
}; // <-- nameless member!
struct {
DWORD LowPart;
LONG HighPart;
} u;
LONGLONG QuadPart;
} LARGE_INTEGER;
but they 'cheat' by suppressing the unwanted warning.
While the above examples are ok, if you use this technique too often, you'll quickly end up with unmaintainable code. Five suggestions to make things clearer:
(1) Change the name bits to something uglier like union_bits, to clearly indicate something out-of-the-ordinary.
(2) Go back to the ugly cast the OP rejected, but hide its ugliness in a macro or in an inline function, as in:
#define BITS(x) (*(uint32_t*)&(x))
But this would break the strict aliasing rules. (See, for example, AndreyT's answer: C99 strict aliasing rules in C++ (GCC).)
(3) Keep the original definiton of Pixel, but do a better cast:
struct Pixel {unsigned char b,g,r,a;} foo;
// ...
printf("%08x\n", ((union {struct Pixel dummy; uint32_t bits;})foo).bits);
(4) But that is even uglier. You can fix this by a typedef:
struct Pixel {unsigned char b,g,r,a;} foo;
typedef union {struct Pixel dummy; uint32_t bits;} CastPixelToBits;
// ...
printf("%08x\n", ((CastPixelToBits)foo).bits); // not VC9
With VC9, or with gcc using -pedantic, you'll need (don't use this with gcc--see note at end):
printf("%08x\n", ((CastPixelToBits*)&foo)->bits); // VC9 (not gcc)
(5) A macro may perhaps be preferred. In gcc, you can define a union cast to any given type very neatly:
#define CAST(type, x) (((union {typeof(x) src; type dst;})(x)).dst) // gcc
// ...
printf("%08x\n", CAST(uint32_t, foo));
With VC9 and other compilers, there is no typeof, and pointers may be needed (don't use this with gcc--see note at end):
#define CAST(typeof_x, type, x) (((union {typeof_x src; type dst;}*)&(x))->dst)
Self-documenting, and safer. And not too ugly. All these suggestions are likely to compile to identical code, so efficiency is not an issue. See also my related answer: How to format a function pointer?.
Warning about gcc: The GCC Manual version 4.3.4 (but not version 4.3.0) states that this last example, with &(x), is undefined behaviour. See http://davmac.wordpress.com/2010/01/08/gcc-strict-aliasing-c99/ and http://gcc.gnu.org/ml/gcc/2010-01/msg00013.html.
The problem with a structure inside a union, is that the compiler is allowed to add padding bytes between members of a structure (or class), except bit fields.
Given:
struct Pixel
{
unsigned char red;
unsigned char green;
unsigned char blue;
unsigned char alpha;
};
This could be laid out as:
Offset Field
------ -----
0x00 red
0x04 green
0x08 blue
0x0C alpha
So the size of the structure would be 16 bytes.
When put in a union, the compiler would take the larger capacity of the two to determine space. Also, as you can see, a 32 bit integer would not align correctly.
I suggest creating functions to combine and extract pixels from a 32-bit quantity. You can declare it inline too:
void Int_To_Pixel(const unsigned int word,
Pixel& p)
{
p.red = (word & 0xff000000) >> 24;
p.blue = (word & 0x00ff0000) >> 16;
p.green = (word & 0x0000ff00) >> 8;
p.alpha = (word & 0x000000ff);
return;
}
This is a lot more reliable than a struct inside a union, including one with bit fields:
struct Pixel_Bit_Fields
{
unsigned int red::8;
unsigned int green::8;
unsigned int blue::8;
unsigned int alpha::8;
};
There is still some mystery when reading this whether red is the MSB or alpha is the MSB. By using bit manipulation, there is no question when reading the code.
Just my suggestions, YMMV.
Why not make the ugly mess into an inline routine? Something like:
inline uint32_t pixel32(const Pixel& p)
{
return *reinterpret_cast<uint32_t*>(&p);
}
You could also provide this routine as a member function for Pixel, called i(), which would allow you to access the value via pixel.i() if you preferred to do it that way. (I'd lean on separating the functionality from the data structure when invariants need not be enforced.)