I've got some code provided by a vendor that I'm using and its typedef'ing an enum with __attribute__((aligned(1), packed)) and GCC is complaining about the multiple attributes:
error: ignoring attribute 'packed' because it conflicts with attribute 'aligned' [-Werror=attributes]
Not sure what the best approach is here. I feel like both of these attributes are not necessary. Would aligned(1) not also make it packed? And is this even necessary for an enum? Wouldn't it be best to have the struct that this enum goes into be packed?
Thanks!
I've removed the packed attribute and that works to make GCC happy but I want to make sure that it will still behave the same. This is going into an embedded system that relies on memory mapped registers so I need the mappings to be correct or else things won't work.
Here's an example from the code supplied by the vendor:
#define DMESCC_PACKED __attribute__ ((__packed__))
#define DMESCC_ENUM8 __attribute__ ((aligned (1), packed))
typedef enum DMESCC_ENUM8 {DMESCC_OFF, DMESCC_ON} dmescc_bittype_t;
typedef volatile struct {
dmescc_bittype_t rx_char_avail : 1;
dmescc_bittype_t zero_count : 1;
dmescc_bittype_t tx_buf_empty : 1;
dmescc_bittype_t dcd : 1;
dmescc_bittype_t sync_hunt : 1;
dmescc_bittype_t cts : 1;
dmescc_bittype_t txunderrun_eom : 1;
dmescc_bittype_t break_abort : 1;
} DMESCC_PACKED dmescc_rr0_t;
When I build the above code I get the GCC error I mentioned above.
Documentation here: https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#Common-Variable-Attributes emphasis mine:
When used on a struct, or struct member, the aligned attribute can only increase the alignment; in order to decrease it, the packed attribute must be specified as well. When used as part of a typedef, the aligned attribute can both increase and decrease alignment, and specifying the packed attribute generates a warning.
Note that the effectiveness of aligned attributes for static variables may be limited by inherent limitations in the system linker and/or object file format. On some systems, the linker is only able to arrange for variables to be aligned up to a certain maximum alignment. (For some linkers, the maximum supported alignment may be very very small.) If your linker is only able to align variables up to a maximum of 8-byte alignment, then specifying aligned(16) in an __attribute__ still only provides you with 8-byte alignment. See your linker documentation for further information.
Older GNU documentation said something else. Also, I don't know what the documentation is trying to say here: "specifying the packed attribute generates a warning", because there is no warning in case I do this (gcc x86_64 12.2.0 -Wall -Wextra):
typedef struct
{
char ch;
int i;
} __attribute__((aligned(1), packed)) aligned_packed_t;
However, this effectively places the struct on a 5 byte offset, where the first address appears to be 8 byte aligned (which might be a thing of the linker as suggested in the above docs). We'd have to place it in an array to learn more.
Since I don't really don't trust the GNU documentation, I did some trial & error to reveal how these work in practice. I created 4 structs:
one with aligned(1)
one with such a struct as its member and also aligned(1) in itself
one with packed
one with both aligned(1) and packed (again this compiles cleanly no warnings)
For each struct I created an array, then printed the address of the first 2 array items. Example:
#include <stdio.h>
typedef struct
{
char ch;
int i;
} __attribute__((aligned(1))) aligned_t;
typedef struct
{
char ch;
aligned_t aligned_member;
} __attribute__((aligned(1))) struct_aligned_t;
typedef struct
{
char ch;
int i;
} __attribute__((packed)) packed_t;
typedef struct
{
char ch;
int i;
} __attribute__((aligned(1),packed)) aligned_packed_t;
#define TEST(type,arr) \
printf("%-16s Address: %p Size: %zu\n", #type, (void*)&arr[0], sizeof(type)); \
printf("%-16s Address: %p Size: %zu\n", #type, (void*)&arr[1], sizeof(type));
int main (void)
{
aligned_t arr1 [3];
struct_aligned_t arr2 [3];
packed_t arr3 [3];
aligned_packed_t arr4 [3];
TEST(aligned_t, arr1);
TEST(struct_aligned_t, arr2);
printf(" Address of member: %p\n", arr2[0].aligned_member);
TEST(packed_t, arr3);
TEST(aligned_packed_t, arr4);
}
Output on x86 Linux:
aligned_t Address: 0x7ffc6f3efb90 Size: 8
aligned_t Address: 0x7ffc6f3efb98 Size: 8
struct_aligned_t Address: 0x7ffc6f3efbb0 Size: 12
struct_aligned_t Address: 0x7ffc6f3efbbc Size: 12
Address of member: 0x40123000007fd8
packed_t Address: 0x7ffc6f3efb72 Size: 5
packed_t Address: 0x7ffc6f3efb77 Size: 5
aligned_packed_t Address: 0x7ffc6f3efb81 Size: 5
aligned_packed_t Address: 0x7ffc6f3efb86 Size: 5
The first struct with just aligned(1) didn't make any difference against a normal struct.
The second struct where the first struct was included as a member, to see if it would be misaligned internally, did not pack it any tighter either, nor did the member get allocated at a misaligned (1 byte) address.
The third struct with only packed did get allocated at a potentially misaligned address and packed into 5 bytes.
The fourth struct with both aligned(1) and packed works just as the one that had packed only.
So my conclusion is that "the aligned attribute can only increase the alignment" is correct and as expected aligned(1) is therefore nonsense. However, you can use it to increase the alignment. ((aligned(16), packed) did give 16 bit size, which effectively cancels packed.
Also I can't make sense of this part of the manual:
When used as part of a typedef, the aligned attribute can both increase and decrease alignment, and specifying the packed attribute generates a warning.
Either I'm missing something or the docs are wrong (again)...
Not sure what the best approach is here. I feel like both of these
attributes are not necessary. Would aligned(1) not also make it
packed?
No, it wouldn't. From the docs:
The aligned attribute specifies a minimum alignment (in bytes) for
variables of the specified type.
and
When attached to an enum definition, the packed attribute indicates that the smallest integral type should be used.
These properties are related but neither is redundant with the other (which makes GCC's diagnostic surprising).
And is this even necessary for an enum? Wouldn't it be best to
have the struct that this enum goes into be packed?
It is meaningful for an enum to be packed regardless of how it is used to compose other types. In particular, having packed on an enum is (only) about the storage size of objects of the enum type. It does not imply packing of structure types that have members of the enum type, but you might want that, too.
On the other hand, the alignment requirement of the enum type is irrelevant to the layout of structure types that have the packed attribute. That's pretty much the point of structure packing.
I've removed the packed attribute and that works to make GCC happy but
I want to make sure that it will still behave the same. This is going
into an embedded system that relies on memory mapped registers so I
need the mappings to be correct or else things won't work.
If only one of the two attributes can be retained, then packed should be that one. Removing it very likely does cause meaningful changes, especially if the enum is used as a structure member or as the type of a memory-mapped register. I can't guarantee that removing the aligned attribute won't also cause behavioral changes, but that's less likely.
It might be worth your while to ask the vendor what version of GCC they use for development and testing, and what range of versions they claim their code is compatible with.
Overall, however, the whole thing has bad code smell. Where it is essential to control storage size exactly, explicit-size integer types such as uint8_t should be used.
Addendum
With regard to the example code added to the question: that the enum type in question is used as the type of a bitfield changes the complexity of the question. Portable code steers well clear of bitfields.
The C language specification does not guarantee that an enum type such as that one can be used as the type of a bitfield member, so you're straight away into implementation-defined territory. Not that using one of the types the specification designates are supported would delay that very long, because many of the properties of bitfields and the structures containing them are implementation defined anyway, in particular,
which data types other than qualified and unqualified versions of _Bool, signed int, and unsigned int are allowed as the type of a bitfield member;
the size and alignment requirement of the addressible storage units in which bitfields are stored (the spec does not connect these in any way with bitfields' declared types);
whether bitfields assigned to the same addressible storage unit are arranged from least-significant position to most, or the opposite;
whether bitfields can be split across adjacent addressible storage units;
whether bitfield members may have atomic type.
GCC's definitions for these behaviors are here: https://gcc.gnu.org/onlinedocs/gcc/Structures-unions-enumerations-and-bit-fields-implementation.html#Structures-unions-enumerations-and-bit-fields-implementation. Note well that many of them depend on the target ABI.
To use the vendor code safely, you really need to know which compiler it was developed for and tested against, and if that's a version of GCC, what target ABI. If you learn or are willing to assume GCC targeting the same ABI that you are targeting, then keep packed, dump aligned(1), and test thoroughly. Otherwise, you'll probably want to do more research.
Related
So, I'm writing a struct that's going to be used for de-serializing a binary stream of data. To get the point across, here is a cut-down version:
typedef struct
{
bool flag1 : 1;
bool flag2 : 1;
bool flag3 : 1;
bool flag4 : 1;
uint32_t reserved : 28;
} frame_flags_t;
typedef struct
{
/* Every frame starts with a magic value. */
uint32_t magic;
frame_flags_t flags;
uint8_t reserved_1;
/* A bunch of other things */
uint32_t crc;
} frame_t;
My question is, if do the following:
frame_t f;
memcpy(&f, raw_data_p, sizeof(frame_t));
Am I guaranteed that f.flags.flag1 is really the first bit (after the magic member, assuming a neatly packed struct (which it is))? And that .flags2 will be the one following that, and etc?
From what I understand the C and C++ standards don't guarantee this. Does GCC?
Am I guaranteed that f.flags.flag1 is really the first bit (after the magic member, assuming a neatly packed struct (which it is))?
The C language does not guarantee that, no.
And that .flags2 will be the one following that, and etc?
The C language does require that consecutive bitfields assigned to the same addressable storage unit be laid out without gaps between them. That is likely to mean that the flags end up occupying adjacent bits in the same byte, but it does not have to mean that.
From what I understand the C and C++ standards don't guarantee this. Does GCC?
No. Structure layout rules are a characteristic of an application binary interface (ABI), which is a property of operating system + hardware combinations. For example, there is an ABI for Linux running on x86_64, a different one for Linux running on 32-bit x86, and still different ones for Windows running on those platforms. GCC supports a wide variety of ABIs, and it lays out structures according to the rules of the target ABI. It cannot make any blanket guarantees about details of structure layout.
For example, the relevant ABI for Linux / x86_64 is https://www.intel.com/content/dam/develop/external/us/en/documents/mpx-linux64-abi.pdf. With respect to bitfield layout, it says:
Bit-fields obey the same size and alignment rules as other structure
and union members.
Also:
bit-fields are allocated from right to left
bit-fields must be contained in a storage unit appropriate for its declared type
bit-fields may share a storage unit with other struct / union members
That's actually not altogether consistent, but the way it's interpreted for your frame_flags_t is that:
the structure has size 4, consisting of a single "addressible storage unit" of that size into which all the bitfields are packed
flag1 uses the least-significant bit
flag2 uses the second-least-significant bit
flag3 uses the third-least-significant bit
flag4 uses the fourth-least-significant bit
Furthermore, the overall frame_t structure has a 4-byte alignment requirement on Linux / x86_64, and it will be laid out with the minimum padding required to align all members. On such a machine, therefore, there will be no padding between the magic member and the flags member. x86_64 is little-endian, so that will indeed put the flag bits in the first byte following magic on Linux / x86_64.
Also, this is for ARM v7,
On this target you can safely use the bitfields and their behaviour is clearly specified by ABI.
For code that is compiled on various/unknown architectures/compilers (8/16/32/64-bit) a global mempool array has to be defined:
uint8_t mempool[SIZE];
This mempool is used to store objects of different structure types e.g.:
typedef struct Meta_t {
uint16_t size;
struct Meta_t *next;
//and more
}
Since structure objects always have to be aligned to the largest possible boundary e.g. 64-byte it has to be ensured that padding bytes are added between those structure objects inside the mempool:
struct Meta_t* obj = (struct Meta_t*) mempool[123] + padding;
Meaning if a structure object would start on a not aligned address, the access to this would cause an alignment trap.
This works already well in my code. But I'm still searching for a portable way for aligning the mempool start address as well. Because without that, padding bytes have to be inserted already between the array start address and the address of the first structure inside the mempool.
The only way I have discovered so far is by defining the mempool inside a union together with another variable that will be aligned by the compiler anyways, but this is supposed be not portable.
Unfortunately for embedded platforms my code is also compiled with ANSI C90 compilers. In fact I cannot make any guess what compilers are exactly used. Because of this I'm searching for an absolutely portable solution and I guess any kind of preprocessor directives or compiler specific attributes or language features that were added after C90 cannot be used
You can use _Alignas, which is part of the C11 standard, to force a particular alignment.
_Alignas(uint64_t) uint8_t mempool[SIZE];
(struct Meta_t*) mempool will lead to undefined behavior for more reasons than alignment - it's also a strict aliasing violation.
The best solution might be to create a union such as this:
typedef union
{
struct Meta_t my_struct;
uint8_t bytes[sizeof(struct Meta_t)];
} thing;
This solves both the alignment and the pointer aliasing problems and works in C90.
Now if we do *(thing)mempool then this is well-defined since this (lvalue) access is done through a union type that includes an uint8_t array among its members. And type punning between union members is also well-defined in C. (No solution exists in C++.)
Unfortunately, this ...
This mempool is used to store objects of different structure types
... combined with this ...
I'm searching for an absolute portable solution and I guess any kind of preprocessor directives or compiler specific attributes or language features that were added after C90 cannot be used
... puts you absolutely up a creek, unless you know in advance all the structure types with which your memory pool may be used. Even the approach you are taking now does not conform strictly to C90, because there is no strictly-conforming way to determine the alignment of an address, so as to compute how much padding is needed.* (You have probably assumed that you can convert it to an integer and look at the least-significant bits, but C does not guarantee that you can determine anything about alignment that way.)
In practice, there is a variety of things that will work in a very wide range of target environments, despite not strictly conforming to the C language specification. Interpreting the result of converting a pointer to an integer as a machine address, so that it is sensible to use it for alignment computations, is one of those. For appropriately aligning a declared array, so would this be:
#define MAX(x,y) (((x) < (y)) ? (y) : (x))
union max_align {
struct d { long l; } l;
struct l { double d; } d;
struct p { void *p; } p;
unsigned char bytes[MAX(MAX(sizeof(struct d), sizeof(struct l)), sizeof(struct p))];
};
#undef MAX
#define MEMPOOL_BLOCK_SIZE sizeof(union max_align)
union maxalign mempool[(size + MEMPOOL_BLOCK_SIZE - 1) / MEMPOOL_BLOCK_SIZE];
For a very large set of C implementations, that not only ensures that the pool itself is properly aligned for any use by strictly-conforming C90 clients, but it also conveniently divides the pool into aligned blocks on which your allocator can draw. Refer also to #Lundin's answer for how pointers into that pool would need to be used to avoid strict aliasing violations.
(If you do happen to know all the types for which you must ensure alignment, then put one of each of those into union max_align instead of d, l, and p, and also make your life easier by having the allocator hand out pointers to union max_align instead of pointers to void or unsigned char.)
Overall, you need to choose a different objective than absolute portability. There is no such thing. Avoiding compiler extensions and language features added in C99 and later is a great start. Minimizing the assumptions you make about implementation behavior is important. And where that's not feasible, choose the most portable option you can come up with, and document it.
*Not to mention that you are relying on uint8_t, which is not in C90, and which is not necessarily provided even by all C99 and later implementations.
We recently found some code submitted to our codebase, along the following lines:
#pragma pack(push,1)
struct xyzzy {
BITMAPINFOHEADER header;
char plugh;
long twisty;
} myVar;
My question is: does the packing apply only to the immediate structure or might it affect the packing of the BITMAPINFOHEADER as well. I can't see the latter case being very useful since it would make the structure different to what you would get from the Windows API calls, for example. Case in point, let's assume the structure is:
typedef struct {
char aChar;
DWORD biSize;
} BITMAPINFOHEADER;
That structure would be vastly different with a packing of one rather than the default eight for Windows (32-bit anyway, may be sixteen for 64-bit).
Is the BITMAPINFOHEADER "protected" from the packing by virtue of the fact it's almost certainly be declared earlier? If it was declared as part of the outer declaration, would it be subject to the packing then?
From the relevant documentation:
pack takes effect at the first struct, union, or class declaration after the pragma is seen. pack has no effect on definitions.
header is a member definition, so it isn't affected.
it was declared as part of the outer declaration, would it be subject to the packing then?
Yes, as it would be a struct declaration.
Also, as Lightness Races in Orbit remarks in a comment, a more convincing wording can be found immediately before:
To pack a class is to place its members directly after each other in memory.
i.e. it says nothing about what those members themselves contain, which may be data and/or padding. The fact that (as explored above) packedness is attached to a type would seem to reinforce that
Still, the documentation is vague enough, so it's best to test that this interpretation is correct; both gcc and VC++ behave as expected. Not that I'm particularly surprised - anything different would break havoc in the type system (taking a pointer to a member of a packed structure would actually provide a pointer to something different than its type says1).
The general idea is: once you finish defining a struct, its binary layout is fixed, and any of its instantiations will conform to it, including subobjects of packed structures. The current #pragma pack value is considered only when defining new structures, and when doing so the binary layout of the members is a fixed black box.
Notes
To be honest, this is a bit of an x86-centric view; machines with stronger alignment requirements would object that even pointers to layout-correct but misaligned structures aren't kosher: although the offsets of fields relative to the given pointer are correct, they aren't really pointers that can be used as they are.
OTOH, given a pointer to an unaligned object you can always detect that it's unaligned and memcpy it to a correctly-aligned location, so it's not as bad as a hypothetical pointer to a packed object, whose layout is effectively unknown unless you happen to know the packing of its parent.
From what I have seen, it only applies to immediate structure.
Take a look at the snippet below:
#include <stdio.h>
struct /*__attribute__((__packed__))*/ struct_Inner {
char a;
int b;
char c;
};
struct __attribute__((__packed__)) struct_Outer {
char a;
int b;
char c;
struct struct_Inner stInner;
};
int main()
{
struct struct_Inner oInner;
struct struct_Outer oOuter;
printf("\n%zu Bytes", sizeof(oInner));
printf("\n%zu Bytes", sizeof(oOuter));
}
Prints:
12 Bytes
18 Bytes
When I pack the struct_Inner it prints:
6 Bytes
12 Bytes
This is when using GCC 7.2.0.
Again, this is not specific to C standard by any means (I just had to read), its more to do with what compilers do.
Is the BITMAPINFOHEADER "protected" from the packing by virtue of the fact it's almost certainly be declared earlier?
I guess yes. It would entirely depend upon the way BITMAPINFOHEADER is declared.
Assuming GCC (or Clang emulating GCC), you can find some relevant information at Structure Layout Pragmas, where it says that the presence of the push preserves the current packing state on a stack of states:
For compatibility with Microsoft Windows compilers, GCC supports a set of #pragma directives that change the maximum alignment of members of structures (other than zero-width bit-fields), unions, and classes subsequently defined. The n value below always is required to be a small power of two and specifies the new alignment in bytes.
#pragma pack(n) simply sets the new alignment.
#pragma pack() sets the alignment to the one that was in effect when compilation started (see also command-line option -fpack-struct[=n] see Code Gen Options).
#pragma pack(push[,n]) pushes the current alignment setting on an internal stack and then optionally sets the new alignment.
#pragma pack(pop) restores the alignment setting to the one saved at the top of the internal stack (and removes that stack entry). Note that #pragma pack([n]) does not influence this internal stack; thus it is possible to have #pragma pack(push) followed by multiple #pragma pack(n) instances and finalized by a single #pragma pack(pop).
Thus, the #pragma in the code added affects all subsequent structure definitions too, until countermanded by #pragma pack(pop). I'd be worried about that.
The documentation doesn't say what happens if you do #pragma pack(pop) when there is no state on the internal stack. Most likely, it falls back to the setting when compilation started.
According to GCC reference:
In the following example struct my_packed_struct's members are
packed closely together, but the internal layout of its s member is
not packed - to do that, struct my_unpacked_struct would need to
be packed too.
struct my_unpacked_struct
{
char c;
int i;
};
struct my_packed_struct __attribute__ ((__packed__))
{
char c;
int i;
struct my_unpacked_struct s;
};
You may only specify this attribute on the definition of a enum, struct or union, not on a typedef which does not also define the
enumerated type, structure or union.
This does not directly answer the question, but might offer an idea why existing compilers decided not to pack sub-structs, and why future compilers are unlikely to change that.
A packing that transitively affected sub-structures would break the type system in subtle ways.
Consider:
//Header A.h
typedef struct {
char aChar;
DWORD biSize;
} BITMAPINFOHEADER;
// File A.c
#include <A.h>
void doStuffToHeader(BITMAPINFOHEADER* h)
{
// compute stuff based on data stored in h
// ...
}
// File B.c
#include <A.h>
#pragma pack(push,1)
struct xyzzy {
BITMAPINFOHEADER header;
char plugh;
long twisty;
} myVar;
void foo()
{
doStuffToHeader(&myVar.header);
}
I pass a pointer to a packed struct to a function that is not aware of the packing. Any attempts by the function to read or write data from the struct will easily break in horrible ways. If a compiler deems this unacceptable, it has two possibilities for fixing the problem:
Transparently unpack the sub-struct into a temporary for the function call and re-pack the result later.
Internally change the type of the header field in xyzzy to something that indicates that it's now a packed type and incompatible with normal BITMAPINFOHEADER.
Both of these are obviously problematic. With this reasoning, even if I wanted to write a compiler that supported packing of sub-structs, I would run into numerous follow-up problems. I'd expect my users to start questioning my design decisions in this regard very soon.
I have the following code:
#pragma pack(push, 1)
typedef struct __attribute__((packed)){
uint64_t msg: 48;
uint16_t crc: 12;
int : 0;
} data_s;
#pragma pack(pop)
typedef union {
uint64_t tot;
data_s split;
} data_t;
int main() {
data_t data;
printf(
"Sizes are: union:%d,struct:%d,uint64_t:%d\n",
sizeof(data),
sizeof(data.split),
sizeof(data.tot)
);
return 0;
}
The output I get is Sizes are: union:16,struct:10,uint64_t:8.
Here I have two issues,
Even though I'm using bit fields and trying to pack it, I am getting 10 bytes even though the number of bits is less than 64(48+12=60) and can be packed into 8 bytes.
Even though the maximum size of the two members of the union is 10, why is its size 16?
Also how do I pack the bits into 8 bytes?
You are allocating an integral type and then tell how many bits to use.
Then you allocate another integral type and tell how many bits to use.
The compiler places these in their respective integrals. To have them in a single integral field, use comma's to separate them, e.g.:
uint64_t msg: 48, crc: 12;
(But note the implementation defined aspect user694733 mentions)
This is implementation defined; how bits are laid out depends on your compiler.
Many compilers split bitfields if they are different types. You could try changing type of crc to uint64_t to see if it makes a difference.
If you want to write portable code and layout is important, then don't use bitfields at all.
These are bit-fields. They are very poorly covered by standardization. If you use them - you are on your own.
#pragma pack(push, 1)
typedef struct __attribute__((packed)){
These are non-standard compiler extensions of the gcc compiler. What happens when you add them is not covered by any standard. The only thing the standard says is that if a compiler doesn't recognize the #pragma, it must ignore that line.
The C standard only guarantees that the types _Bool, unsigned int and signed int are valid for bit-fields. You use uint64_t and uint16_t. What happens when you do is not covered by the C standard - this is implementation-defined behavior. The standard speaks of "units", but it is not specified how large a "unit" is.
msg: 48; The C standard does not specify if this is the least significant 48 bits or the most significant ones. It does not specify order of allocation, it does not specify alignment. Add endianess on top of that, and you can't really know what this code does.
All the C standard guarantees is that msg resides on a lower address than trailing struct members. Unless they are merged into the same bit-field - then the standard guarantees nothing. Completely implementation-defined.
int : 0; is useless to add at the end of a bit-field, the only purpose of this code is to the compiler not to merge any trailing bit-field into the previous one.
#pragma pack and similar doesn't, as far as I know, guarantee that there is no trailing padding in the end of the struct/union.
gcc is known to behave strange together with bit-fields. It has this in common with every single C compiler ever written.
The answer to your questions can thus be summarized as: because bit-fields.
An alternative approach which will be 100% deterministic, well-defined portable and safe is something like this:
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
typedef uint64_t data_t;
static inline uint64_t msg (data_t* data)
{
return *data >> 12;
}
static inline uint64_t crc (data_t* data)
{
return *data & 0xFFFu;
}
int main() {
data_t data = 0xFFFFFAAAu;
printf("msg: %"PRIX64" crc:%"PRIX64, msg(&data), crc(&data));
return 0;
}
This is even portable across CPU:s of different endianess.
For your question 2. : A union always takes as much space as its largest member. Here it considers the struct split to be of size 10 and then you probably have optimization flag when you compile to align memory (which is recommended), making it a power of 2 (from 10 to 16).
Even though I'm using bit fields and trying to pack it, I am getting 10 bytes even though the number of bits is less than
64(48+12=60) and can be packed into 8 bytes.
Note in the first place that #pragma pack, like most #pragmas, is an extension with implementation-defined behavior. The C language does define its behavior.
In the second place, C affords implementations considerable freedom with respect to how they lay out the contents of a structure, especially with respect to bitfields. In fact, supposing that uint64_t is a different type from unsigned int in your implementation, whether you can even have a bitfield of the former type in the first place is implementation-defined.
C does not leave it completely open, however. Here's the key part of the specification for bitfield layout within a structure:
An implementation may allocate any addressable storage unit large
enough to hold a bit- field. If enough space remains, a bit-field that
immediately follows another bit-field in a structure shall be packed
into adjacent bits of the same unit. If insufficient space remains,
whether a bit-field that does not fit is put into the next unit or
overlaps adjacent units is implementation-defined. The order of
allocation of bit-fields within a unit (high-order to low-order or
low-order to high-order) is implementation-defined. The alignment of
the addressable storage unit is unspecified.
(C2011, 6.7.2.1/11; emphasis added)
Note well that C does not say that the declared type of a bitfield member has anything to do with the size of the addressable storage unit in which its bits are stored (neither there nor anywhere else), though in fact some compilers do implement such behavior. On the other hand, what it does say certainly leads me to expect that if C accepts a 48-bit bitfield in the first place then an immediately-following 12-bit bitfield should be stored in the same unit. Implementation-defined packing specifications don't even enter the picture. Thus, your implementation seems to be non-conforming in this regard.
Even though the maximum size of the two members of the union is 10, why is its size 16?
Unions can have trailing padding, just like structures can. Padding will have been introduced into the union's layout to support the compiler's idea of optimal alignment for objects of that type and their members. In particular, it is likely that your structure has at least an 8-byte alignment requirement, so the union is padded to a size that is a multiple of that alignment requirement. This is again implementation-defined, and as long as we're there, it's possible that you could avoid the padding by instructing the compiler to pack the union, too.
Also how do I pack the bits into 8 bytes?
It may be that you can't, but you should check your compiler's documentation. Since the observed behavior appears to be non-conforming, it's anyone's guess what you can or must do.
If it were me, though, the first thing I would try is to remove the pragma and attribute; remove the zero-length bitfield, and change the declared type of the crc bitfield to match that of the preceding bitfield (uint64_t). The point of all of these is to clear away details that conceivably might confuse the compiler, so as to try to get it to render the behavior that the standard demands in the first place. If you can get the struct to come out as 8 bytes, then you probably don't need to do anything more to get the union to come out the same size (on account of 8 being a somewhat magic number).
I'm writing C cross-platform library but eventually I've got error in my unittests, but only on Windows machines. I've tracked the problem and found it's related to alignment of structures (I'm using arrays of structures to hold data for multiple similar objects). The problem is: memset(sizeof(struct)) and setting structures members one by one produce different byte-to-byte result and therefore memcmp() returns "not equal" result.
Here the code for illustration:
#include <stdio.h>
#include <string.h>
typedef struct {
long long a;
int b;
} S1;
typedef struct {
long a;
int b;
} S2;
S1 s1, s2;
int main()
{
printf("%d %d\n", sizeof(S1), sizeof(S2));
memset(&s1, 0xFF, sizeof(S1));
memset(&s2, 0x00, sizeof(S1));
s1.a = 0LL; s1.b = 0;
if (0 == memcmp(&s1, &s2, sizeof(S1)))
printf("Equal\n");
else
printf("Not equal\n");
return 0;
}
This code with MSVC 2003 # Windows produce following output:
16 8
Not equal
But the same code with GCC 3.3.6 # Linux works as expected:
12 8
Equal
This makes my unit-testing very hard.
Am I understand correctly that MSVC uses size of biggest native type (long long) to determine alignment to structure?
Can somebody give me advice how can I change my code to make it more robust against this strange alignment problem? In my real code I'm working with arrays of structures via generic pointers to execute memset/memcmp and I'm usually don't know exact type, I have only sizeof(struct) value.
Your unit test's expectation is wrong. It (or the code it tests) should not scan the structure's buffer byte-by-byte. For byte-precise data the code should create a byte buffer explicitly on stack or on heap and fill it with the extracts from each member. The extracts can be obtained in CPU-endianness-independent way by using the right shift operation against the integer values and casting the result by the byte type such as (unsigned char).
BTW, your snippet writes past s2. You could fix that by changing this
memset(&s2, 0x00, sizeof(S1));
s1.a = 0LL; s1.b = 0;
if (0 == memcmp(&s1, &s2, sizeof(S1)))
to this,
memset(&s2, 0x00, sizeof(S2));
s1.a = 0LL; s1.b = 0;
if (0 == memcmp(&s1, &s2, sizeof(S2)))
but the result is technically "undefined" because the alignment of members in the structures is compiler-specific.
GCC Manual:
Note that the alignment of any given struct or union type is required by the ISO C standard to be at least a perfect multiple of the lowest common multiple of the alignments of all of the members of the struct or union in question.
Also, this typically introduces an element of padding (i.e. filler bytes to have the structure aligned). You can use the #pragma with an argument of packed. Note, #pragmas are NOT a portable way of working. Unfortunately, this is also about the only way of working in your case.
References:
Here GCC on structure alignment.
MSDN structure alignment.
What we have done is used the #pragma pack to specify how big the objects should be:
#pragma pack(push, 2)
typedef struct {
long long a;
int b;
} S1;
typedef struct {
long a;
int b;
} S2;
#pragma pack(pop)
If you do this, the structures will be the same size on both platforms.
Note that this is not a 'strange' alignment problem. MSVC has chosen to ensure that the struct is aligned on a 64-bit boundary since it has a 64-bit member so it adds some padding at the end of the struct to ensure that arrays of those objects will have each element properly aligned. I'm actually surprised that GCC doesn't do the same.
I'm curious what you're unit testing does that hits a snag with this - most of the time alignment of structure members isn't necessary unless you need to match a binary file format or a wire protocol or you really need to reduce the memory used by a structure (especially used in embedded systems). Without knowing what you're trying to do in your tests I don't think a good suggestion can be given. Packing the structure might be a solution, but it comes at some cost - performance (especially on non-Intel platforms) and portability (how struct packing is set up is can be different from compiler to compiler). These may not matter to you, but there might be a better way to solve the problem in your case.
You can either do something like
#ifdef _MSC_VER
#pragma pack(push, 16)
#endif
/* your struct defs */
#ifdef _MSC_VER
#pragma pack(pop)
#endif
to give a compiler directive forcing alignment
Or go into the project options and change the default struct alignment [under Code Generation]
Structure padding for 64-bit values is different on different compilers. I've seen differences between even between gcc targets.
Note that explicitly padding to 64-bit alignment will only hide the problem. It will come back if you begin naively nesting structures, because the compilers will still disagree on the natural alignment of the inner structures.