Troubles while reading the struct size - c

If I debug the following code then I see the size value is 12 (as expected).
#include <cstdint>
int main(int argc, char *argv[])
{
typedef struct __attribute__((__packed__)) { int8_t value; } time;
typedef struct __attribute__((__packed__)) {
uint8_t msg[8];
// time t1;
uint32_t count;
} theStruct;
theStruct s;
int size = sizeof(s);
return 0;
}
Interestingly, removing the comment at "time t1;", the value of size goes to 16. I was expecting 13.
I know (more or less) that this is explained by the data structure padding story...
But, is there some way to avoid this issue?
What to do in order to read size = 13?

There are some issues with MinGW's emulation of MSVC struct packing.
The workaround is to pass -mno-ms-bitfields flag to the compiler; this will cause it to use its own layout algorithm rather than attempt to emulate MSVC.
See also Struct packing and alignment with mingw (ARM but may be the same issue).

It is clearly an alignement problem, meaning it has nothing to do with the language itself but all with the underlying platform.
If the platform knows (or thinks) that int32_t need an alignement of 4, it should add 3 bytes of padding after time.
Anyway __attribute__((__packed__)) is non standard C and will only be interpreted by gcc (*). Worse it leads to non portable programs : according to this answer on another question, it would cause a bus error on a Sparc architecture because of a misaligned int32_t.
I do know that x86 (and derivatives) architecture are now the most common, but other architecture may still exist ...
(*) according to Visual C++ equivalent of GCC's __attribute__ ((__packed__)), the MSVC equivalent is #pragma pack(push, 1)

Related

ARM GCC Address of packed struct warning

In my code, I have something like this:
#include <stdint.h>
typedef struct __attribute__((packed)) {
uint8_t test1;
uint16_t test2;
} test_struct_t;
test_struct_t test_struct;
int main(void)
{
uint32_t *ptr = (uint32_t*) &test_struct;
return 0;
}
When I compile this using arm-none-eabi-gcc, I get the warning
.\test.c:11:2: warning: converting a packed 'test_struct_t' pointer
(alignment 1) to a 'uint32_t' {aka 'long unsigned int'} pointer
(alignment 4) may result in an unaligned pointer value
[-Waddress-of-packed-member]
Can anyone tell me why this is happening? Taking the address of a packed struct member is of course dangerous. But the whole struct itself should always be aligned, shouldn't it?
There is an answer in the comments, but since it's author didn't post it, I take the liberty to post it myself. All the credit is due to #Clifford.
By default, when the struct is packed, compilers also change alignment of the struct to 1 byte. However, for your case you need the struct to be both packed and aligned as 32-bit unsigned integer. This can be done by changing the packing attribute as following:
#include <stdint.h>
struct __attribute__((packed, aligned(sizeof(uint32_t)))) TestStruct {
uint8_t test1;
uint16_t test2;
};
struct TestStruct test_struct;
int32_t* p = (int32_t*)(&test_struct);
This compiles for ARM platform without any warnings.
In my experience, "packed" structs are almost always a bad idea. They don't always do what people think they do, and they might do other things as well. Depending on compilers, target processors, options, etc., you might find the compiler generating code that uses multiple byte accesses to things that you expect to be 16-bit or 32-bit accesses.
Hardware registers are always going to be properly aligned on a microcontroller. (Bitfields may be a different matter, but you are not using bitfields here.) But there might be gaps or padding.
And the whole idea of trying to access this using a pointer to a uint32_t is wrong. Don't access data via pointer casts like this - in fact, if you see a pointer cast at all, be highly suspicious.
So how do you get a structure that matches a hardware structure exactly? You write it out explicitly, and you use compile-time checks to be sure:
#pragma GCC diagnostic error "-Wpadded"
struct TestStruct {
uint8_t test1;
uint8_t padding;
uint16_t test2;
};
_Static_assert(sizeof(struct TestStruct) == 4, "Size check");
The padding is explicit. Any mistakes will be caught by the compiler.
What if you really, really want an unaligned 16-bit field in the middle here, and you haven't made a mistake in reading the datasheets? Use bitfields:
#pragma GCC diagnostic error "-Wpadded"
struct TestStruct2 {
uint32_t test1 : 8;
uint32_t test2 : 16;
uint32_t padding : 8;
};
_Static_assert(sizeof(struct TestStruct2) == 4, "Size check");
Put the padding in explicitly. Tell the compiler to complain about missing padding, and also check the size. The compiler doesn't charge for the extra microsecond of work.
And what if you really, really, really need to access this as a uint32_t ? You use a union for type-punning (though not with C++) :
union TestUnion {
uint32_t raw;
struct TestStruct2 s;
};
Your packed structure has a size of 3 bytes, and there can be no padding in it. Thus, if we were to create an array of such structures, with the first element having a 4-byte aligned address then, by the definition of arrays (contiguous memory), the second element would be three bytes (sizeof(struct test_struct_t)) from that. Thus, the second element would have only single byte alignment – so, the alignment requirement of your structure is, by deduction, one byte.
On your ARM platform, a unit32_t requires 4 byte alignment, hence the warning.

Byte allocation in struct containing struct within C

I have the following structs in a C program.
typedef struct
{
unsigned char msg_id : 8;
unsigned char msg_num : 8;
} Message_Header_Type;
typedef struct
{
Message_Header_Type header;
int msg[32];
} Raw_Message_Type;
What I'm seeing is that the header in the Raw_Message_Type is taking up 4 bytes but I only want it to take only 2. How would I go about doing this?
What you are looking for is struct packing, which is platform and compiler dependent, the details are not specified in the C standard.
With GCC, on a 32-bit platform, you could do the following to get a struct of size 2:
typedef struct __attribute__((__packed__))
{
unsigned char msg_id;
unsigned char msg_num;
} Message_Header_Type;
From the GCC documentation:
packed
This attribute, attached to an enum, struct, or union type definition, specified that the minimum required memory be used to represent the type.
Message_Header_type inside of Raw_Message_Type is still taking 2 bytes as you would expect, but the compiler is padding the structure for alignment reasons, so your Raw_Message_Type is 132 bytes long instead of the 130 bytes you expected. This is probably because 32-bit aligned accesses are more efficient on your processor.
If you are using gcc, you can tell the compiler to align on 2-byte boundaries instead of the 4-byte boundaries that it is using by using #pragma pack.
typedef struct
{
unsigned char msg_id : 8;
unsigned char msg_num : 8;
} Message_Header_Type;
#pragma pack(push, 2) // save current alignment and set to 2-byte boundaries
typedef struct
{
Message_Header_Type header;
int msg[32];
} Raw_Message_Type;
#pragma pack(pop) // restore the previous alignment
Special packing like this is often necessary when using fixed-size structures in files, but be aware that there may be a (light) performance penalty for using a different packing than what the processor prefers. In this specific case, your 32 msg[] ints are now all 2 bytes off from the preferred alignment for your platform.
C implementations are free to insert padding into struct layouts between members, at the end, or both. It is common for them to do so for alignment purposes. It is also common for compilers to provide a mechanism to influence or override the padding, but details are necessarily implementation-specific.
With GCC, for example, you could apply the packed attribute to both your structs:
typedef struct __attribute__((__packed__))
{
unsigned char msg_id : 8;
unsigned char msg_num : 8;
} Message_Header_Type;
typedef struct __attribute__((__packed__))
{
Message_Header_Type header;
int msg[32];
} Raw_Message_Type;
Some other implementations borrow GCC's approach; some use pragmas for the same purpose; and some provide other mechanisms. You'll need to check your compiler's documentation.
Structure objects are not guaranteed the size of the sum of bits used in member bit fields. This is due to padding.
In your case though Raw_Message_Type has another member int msg[32];. It seems logical to use two more bytes for the purpose alignment so that msg[0] can be aligned to a 4-byte boundary.
There is a good chance that
Message_Header_Type obj;
printf("Size : %zu\n",sizeof(obj));
would give you 2 as a the result.

How to override C compiler aligning word-sized variable in struct to word boundary

I have a structure specified as following
Member 1, 16 bits
Member 2, 32 bits
Member 3, 32 bits
which I shall be reading from a file. I want to read straight from the file into the struct.
The problem is that the C compiler will align the variables m1, m2 and m3 to word boundaries which are at 32 bits since I am working on an ARM Cortex M3 for the following struct declaration:
typedef struct
{
uint16_t m1;
uint32_t m2;
uint32_t m3;
}something;
Reading directly from file will put wrong values in m2 and m3, and reads 2 extra bytes too.
I have hacked around and am currently using the following which works just fine:
typedef struct
{
uint16_t m1;
struct
{
uint16_t lo;
uint16_t hi;
}m2;
struct
{
uint16_t lo;
uint16_t hi;
}m3;
}something;
However, this looks like a really dirty hack. I cannot help wishing for a cleaner way to force the compiler to put halves of m2 and m3 in different words, however sub-optimal it may be.
I am using arm-none-eabi-gcc. I know about bit packing, but am unable to work around this optimisation.
Edit: Turns out I didn't know enough about bit-packing :D
What you are looking for is the packed attribute. This will force gcc to not do any padding around members. Taken from the GCC Online docs:
packed
This attribute, attached to an enum, struct, or union type definition, specified that the minimum required memory be used to represent the type.
Specifying this attribute for struct and union types is equivalent to specifying the packed attribute on each of the structure or union members. Specifying the -fshort-enums flag on the line is equivalent to specifying the packed attribute on all enum definitions.
You may only specify this attribute after a closing curly brace on an enum definition, not in a typedef declaration, unless that declaration also contains the definition of the enum.
So what you want is something like:
typedef struct
{
uint16_t m1;
uint32_t m2;
uint32_t m3;
} __attribute__ ((packed)) something;
In addition I would recommend using a compile time assertion check to ensure that the size of the struct is really what you want it to be.
Perhaps #pragma pack(2). That should force the compiler to use 2-byte alignment
You cannot directly read such a struct from a file, and you should never try to. Misalignement can cause traps on certain architecture, and you should not rely on pragma to fix that.
The almost(*) portable way if to read file elements into struct elements unless your are sure that the struct was written with same architecture and alignement (at least compatible) as your are using for reading.
So for your use case, I would recommend :
fread(&something.m1, sizeof(something.m1), 1, fd);
fread(&something.m2, sizeof(something.m2), 1, fd);
fread(&something.m3, sizeof(something.m3), 1, fd);
(*) it is almost portable because it assume that there are no endian problems which can be correct or not depending on your needs. If you are on one single machine or one single architecture it is fine, but if you write struct on a big endian machine and read it on a little endian one, bad things will occur ...
__attribute__ ((aligned (2)));

sizeof(struct) different for different compilers

Supposing I have a code like this:
#include <stdio.h>
#include <stdint.h>
int main(int argc, char *argv[]) {
typedef struct{
uint16_t x : 9;
uint8_t y : 7;
} z;
printf("sizeof(z) = %lu\n",sizeof(z));
}
I have different results for clang on Mac (2) and someone told me on Windows it returned (3). Not sure if I understand it well, but I see that while first compiler compresses the struct to 9+7 = 16 bits, the other uses 16 bits of uint16_t and 8 of uint8_t. Could you advise?
Not sure if I understand it well, but I see that while first compiler compresses the struct to 9+7 = 16 bits, the other uses 16 bits of uint16_t and 8 of uint8_t. Could you advise?
The first thing to remember about bit-field is this phrase from K&R, 2nd:
(6.9 Bit-fields) "Almost everything about fields is implementation-dependent."
It includes padding, alignment and bit endianness.
There are two possible problems that might be occurring:
Bit-fields are very poorly standardized part within the ANSI C specification. The compiler chooses how bits are allocated within the bit-field container.You should avoid using them inside structures instead you can use #define or enum.
The second possible issue is that the compiler will lay the structure in memory by adding padding to ensure that the next object is aligned to the size of that object.It is a good practices to place elements of the struct according to their size:
typedef struct{
uint8_t x : 7;
uint16_t y : 9;
} z;

long long alignment problem (MSVC vs. GCC)

I'm writing C cross-platform library but eventually I've got error in my unittests, but only on Windows machines. I've tracked the problem and found it's related to alignment of structures (I'm using arrays of structures to hold data for multiple similar objects). The problem is: memset(sizeof(struct)) and setting structures members one by one produce different byte-to-byte result and therefore memcmp() returns "not equal" result.
Here the code for illustration:
#include <stdio.h>
#include <string.h>
typedef struct {
long long a;
int b;
} S1;
typedef struct {
long a;
int b;
} S2;
S1 s1, s2;
int main()
{
printf("%d %d\n", sizeof(S1), sizeof(S2));
memset(&s1, 0xFF, sizeof(S1));
memset(&s2, 0x00, sizeof(S1));
s1.a = 0LL; s1.b = 0;
if (0 == memcmp(&s1, &s2, sizeof(S1)))
printf("Equal\n");
else
printf("Not equal\n");
return 0;
}
This code with MSVC 2003 # Windows produce following output:
16 8
Not equal
But the same code with GCC 3.3.6 # Linux works as expected:
12 8
Equal
This makes my unit-testing very hard.
Am I understand correctly that MSVC uses size of biggest native type (long long) to determine alignment to structure?
Can somebody give me advice how can I change my code to make it more robust against this strange alignment problem? In my real code I'm working with arrays of structures via generic pointers to execute memset/memcmp and I'm usually don't know exact type, I have only sizeof(struct) value.
Your unit test's expectation is wrong. It (or the code it tests) should not scan the structure's buffer byte-by-byte. For byte-precise data the code should create a byte buffer explicitly on stack or on heap and fill it with the extracts from each member. The extracts can be obtained in CPU-endianness-independent way by using the right shift operation against the integer values and casting the result by the byte type such as (unsigned char).
BTW, your snippet writes past s2. You could fix that by changing this
memset(&s2, 0x00, sizeof(S1));
s1.a = 0LL; s1.b = 0;
if (0 == memcmp(&s1, &s2, sizeof(S1)))
to this,
memset(&s2, 0x00, sizeof(S2));
s1.a = 0LL; s1.b = 0;
if (0 == memcmp(&s1, &s2, sizeof(S2)))
but the result is technically "undefined" because the alignment of members in the structures is compiler-specific.
GCC Manual:
Note that the alignment of any given struct or union type is required by the ISO C standard to be at least a perfect multiple of the lowest common multiple of the alignments of all of the members of the struct or union in question.
Also, this typically introduces an element of padding (i.e. filler bytes to have the structure aligned). You can use the #pragma with an argument of packed. Note, #pragmas are NOT a portable way of working. Unfortunately, this is also about the only way of working in your case.
References:
Here GCC on structure alignment.
MSDN structure alignment.
What we have done is used the #pragma pack to specify how big the objects should be:
#pragma pack(push, 2)
typedef struct {
long long a;
int b;
} S1;
typedef struct {
long a;
int b;
} S2;
#pragma pack(pop)
If you do this, the structures will be the same size on both platforms.
Note that this is not a 'strange' alignment problem. MSVC has chosen to ensure that the struct is aligned on a 64-bit boundary since it has a 64-bit member so it adds some padding at the end of the struct to ensure that arrays of those objects will have each element properly aligned. I'm actually surprised that GCC doesn't do the same.
I'm curious what you're unit testing does that hits a snag with this - most of the time alignment of structure members isn't necessary unless you need to match a binary file format or a wire protocol or you really need to reduce the memory used by a structure (especially used in embedded systems). Without knowing what you're trying to do in your tests I don't think a good suggestion can be given. Packing the structure might be a solution, but it comes at some cost - performance (especially on non-Intel platforms) and portability (how struct packing is set up is can be different from compiler to compiler). These may not matter to you, but there might be a better way to solve the problem in your case.
You can either do something like
#ifdef _MSC_VER
#pragma pack(push, 16)
#endif
/* your struct defs */
#ifdef _MSC_VER
#pragma pack(pop)
#endif
to give a compiler directive forcing alignment
Or go into the project options and change the default struct alignment [under Code Generation]
Structure padding for 64-bit values is different on different compilers. I've seen differences between even between gcc targets.
Note that explicitly padding to 64-bit alignment will only hide the problem. It will come back if you begin naively nesting structures, because the compilers will still disagree on the natural alignment of the inner structures.

Resources