I have the following structure:
typedef struct LOG_EVENT
{
time_t time;
uint32_t count;
int32_t error_type;
uint16_t gm_state;
uint8_t gm_mode;
uint8_t mc_state;
} LOG_EVENT;
On 32-bit system , the structure's strict alignment is 4bytes so the members are aligned across 4-byte boundary. There is no padding added in this case because all members together are 4-byte aligned.
But this is not true on 64-bit system, where time_t is 64bit. The strict alignment in that case is 8-byte.
How can I change the alignment to 4-byte ? I want to align across 4-byte boundary on 64-bit system because I want to make sure no padding is done.
From the gcc attributes page https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/Variable-Attributes.html , it says that The aligned attribute can only increase the alignment; but you can decrease it by specifying packed as well.
I don't see packed attribute taking in any arguments.
Also, if I use the byte-alignment like below, would it cause any issues compared to 4-byte alignment:
typedef struct __attribute__ ((packed)) LOG_EVENT
{
time_t time;
uint32_t count;
int32_t error_type;
uint16_t gm_state;
uint8_t gm_mode;
uint8_t mc_state;
} LOG_EVENT;
#pragma pack(4) will set alignment to 4 bytes.
Notice this directive is not part of the standard, this was originally introduced in MSVC and later adopted by gcc for compatibility with the Microsoft's compiler.
Also, notice the size of types like time_t, size_t and all pointer types will vary between those architectures. If your intention is to make a structure that is intelligible between applications running on those two architectures this will be a problem.
Also know that there is no benefit in using 64-bit applications other than the fact that you can address more than 4GB of memory, if you don't need all that memory you can stick to 32-bit, there is no sin in that.
The most straightforward thing to do is ... not use time_t. Use fixed-width types only.
typedef struct LOG_EVENT
{
int32_t time;
uint32_t count;
int32_t error_type;
uint16_t gm_state;
uint8_t gm_mode;
uint8_t mc_state;
} LOG_EVENT;
You give up the ability to handle timestamps outside the range of signed 32-bit time_t (usually, but not always, 1901-12-13T20:45:52Z through 2038-01-19T03:14:07Z) but if you're trying to work with an on-disk record array with 32-bit timestamps, which is the most plausible explanation here, that's not an issue.
Related
So, I'm writing a struct that's going to be used for de-serializing a binary stream of data. To get the point across, here is a cut-down version:
typedef struct
{
bool flag1 : 1;
bool flag2 : 1;
bool flag3 : 1;
bool flag4 : 1;
uint32_t reserved : 28;
} frame_flags_t;
typedef struct
{
/* Every frame starts with a magic value. */
uint32_t magic;
frame_flags_t flags;
uint8_t reserved_1;
/* A bunch of other things */
uint32_t crc;
} frame_t;
My question is, if do the following:
frame_t f;
memcpy(&f, raw_data_p, sizeof(frame_t));
Am I guaranteed that f.flags.flag1 is really the first bit (after the magic member, assuming a neatly packed struct (which it is))? And that .flags2 will be the one following that, and etc?
From what I understand the C and C++ standards don't guarantee this. Does GCC?
Am I guaranteed that f.flags.flag1 is really the first bit (after the magic member, assuming a neatly packed struct (which it is))?
The C language does not guarantee that, no.
And that .flags2 will be the one following that, and etc?
The C language does require that consecutive bitfields assigned to the same addressable storage unit be laid out without gaps between them. That is likely to mean that the flags end up occupying adjacent bits in the same byte, but it does not have to mean that.
From what I understand the C and C++ standards don't guarantee this. Does GCC?
No. Structure layout rules are a characteristic of an application binary interface (ABI), which is a property of operating system + hardware combinations. For example, there is an ABI for Linux running on x86_64, a different one for Linux running on 32-bit x86, and still different ones for Windows running on those platforms. GCC supports a wide variety of ABIs, and it lays out structures according to the rules of the target ABI. It cannot make any blanket guarantees about details of structure layout.
For example, the relevant ABI for Linux / x86_64 is https://www.intel.com/content/dam/develop/external/us/en/documents/mpx-linux64-abi.pdf. With respect to bitfield layout, it says:
Bit-fields obey the same size and alignment rules as other structure
and union members.
Also:
bit-fields are allocated from right to left
bit-fields must be contained in a storage unit appropriate for its declared type
bit-fields may share a storage unit with other struct / union members
That's actually not altogether consistent, but the way it's interpreted for your frame_flags_t is that:
the structure has size 4, consisting of a single "addressible storage unit" of that size into which all the bitfields are packed
flag1 uses the least-significant bit
flag2 uses the second-least-significant bit
flag3 uses the third-least-significant bit
flag4 uses the fourth-least-significant bit
Furthermore, the overall frame_t structure has a 4-byte alignment requirement on Linux / x86_64, and it will be laid out with the minimum padding required to align all members. On such a machine, therefore, there will be no padding between the magic member and the flags member. x86_64 is little-endian, so that will indeed put the flag bits in the first byte following magic on Linux / x86_64.
Also, this is for ARM v7,
On this target you can safely use the bitfields and their behaviour is clearly specified by ABI.
In my code, I have something like this:
#include <stdint.h>
typedef struct __attribute__((packed)) {
uint8_t test1;
uint16_t test2;
} test_struct_t;
test_struct_t test_struct;
int main(void)
{
uint32_t *ptr = (uint32_t*) &test_struct;
return 0;
}
When I compile this using arm-none-eabi-gcc, I get the warning
.\test.c:11:2: warning: converting a packed 'test_struct_t' pointer
(alignment 1) to a 'uint32_t' {aka 'long unsigned int'} pointer
(alignment 4) may result in an unaligned pointer value
[-Waddress-of-packed-member]
Can anyone tell me why this is happening? Taking the address of a packed struct member is of course dangerous. But the whole struct itself should always be aligned, shouldn't it?
There is an answer in the comments, but since it's author didn't post it, I take the liberty to post it myself. All the credit is due to #Clifford.
By default, when the struct is packed, compilers also change alignment of the struct to 1 byte. However, for your case you need the struct to be both packed and aligned as 32-bit unsigned integer. This can be done by changing the packing attribute as following:
#include <stdint.h>
struct __attribute__((packed, aligned(sizeof(uint32_t)))) TestStruct {
uint8_t test1;
uint16_t test2;
};
struct TestStruct test_struct;
int32_t* p = (int32_t*)(&test_struct);
This compiles for ARM platform without any warnings.
In my experience, "packed" structs are almost always a bad idea. They don't always do what people think they do, and they might do other things as well. Depending on compilers, target processors, options, etc., you might find the compiler generating code that uses multiple byte accesses to things that you expect to be 16-bit or 32-bit accesses.
Hardware registers are always going to be properly aligned on a microcontroller. (Bitfields may be a different matter, but you are not using bitfields here.) But there might be gaps or padding.
And the whole idea of trying to access this using a pointer to a uint32_t is wrong. Don't access data via pointer casts like this - in fact, if you see a pointer cast at all, be highly suspicious.
So how do you get a structure that matches a hardware structure exactly? You write it out explicitly, and you use compile-time checks to be sure:
#pragma GCC diagnostic error "-Wpadded"
struct TestStruct {
uint8_t test1;
uint8_t padding;
uint16_t test2;
};
_Static_assert(sizeof(struct TestStruct) == 4, "Size check");
The padding is explicit. Any mistakes will be caught by the compiler.
What if you really, really want an unaligned 16-bit field in the middle here, and you haven't made a mistake in reading the datasheets? Use bitfields:
#pragma GCC diagnostic error "-Wpadded"
struct TestStruct2 {
uint32_t test1 : 8;
uint32_t test2 : 16;
uint32_t padding : 8;
};
_Static_assert(sizeof(struct TestStruct2) == 4, "Size check");
Put the padding in explicitly. Tell the compiler to complain about missing padding, and also check the size. The compiler doesn't charge for the extra microsecond of work.
And what if you really, really, really need to access this as a uint32_t ? You use a union for type-punning (though not with C++) :
union TestUnion {
uint32_t raw;
struct TestStruct2 s;
};
Your packed structure has a size of 3 bytes, and there can be no padding in it. Thus, if we were to create an array of such structures, with the first element having a 4-byte aligned address then, by the definition of arrays (contiguous memory), the second element would be three bytes (sizeof(struct test_struct_t)) from that. Thus, the second element would have only single byte alignment – so, the alignment requirement of your structure is, by deduction, one byte.
On your ARM platform, a unit32_t requires 4 byte alignment, hence the warning.
I am working on an embedded application where RAM is extremely tight.
For this purpose I need to create a 24 bit unsigned integer data type. I am doing this using a struct:
typedef struct
{
uint32_t v : 24;
} uint24_t;
However when I interrogate the size of a variable of this type, it returns "4", i.e.:
uint24_t x;
x.v = 0;
printf("Size = %u", sizeof(x));
Is there a way I can force this variable to have 3 bytes?
Initially I thought it was because it is forcing datatypes to be word aligned, but I can for example do this:
typedef struct
{
uint8_t blah[3];
} mytype;
And in that case the size comes out at 3.
Well, you could try to ensure that the structure only takes up the space you need, with something like:
#pragma pack(push, 1)
typedef struct { uint8_t byt[3]; } UInt24;
#pragma pack(pop)
You may have to provide those compiler directives (like the #pragma lines above) to ensure there's no padding but this will probably be the default for a structure with only eight-bit fields(a).
You would probably then have to pack/unpack real values to and from the structure, something like:
// Inline suggestion used to (hopefully) reduce overhead.
inline uint32_t unpack(UInt24 x) {
uint32_t retVal = x.byt[0];
retVal = retVal << 8 | x.byt[1];
retVal = retVal << 8 | x.byt[2];
return retVal;
}
inline UInt24 pack(uint32_t x) {
UInt24 retVal;
retVal.byt[0] = (x >> 16) & 0xff;
retVal.byt[1] = (x >> 8) & 0xff;
retVal.byt[2] = x & 0xff;
return retVal;
}
Note that this gives you big-endian values regardless of your actual architecture. This won't matter if you're exclusively packing and unpacking yourself, but it may be an issue if you want to use the memory blocks elsewhere in a specific layout (in which case you can just change the pack/unpack code to use the desired format).
This method adds a little code to your system (and a probably minimal performance penalty) so you'll have to decide if that's worth the saving in data space used.
(a) For example, both gcc 7.3 and clang 6.0 show 3 6 for the following program, showing that there is no padding either within or following the structure:
#include <stdio.h>
#include <stdint.h>
typedef struct { uint8_t byt[3]; } UInt24;
int main() {
UInt24 x, y[2];
printf("%zd %zd\n", sizeof(x), sizeof(y));
return 0;
}
However, that is just a sample so you may want to consider, in the interest of portable code, using something like #pragma pack(1), or putting in code to catch environments where this may not be the case.
A comment by João Baptista on this site says that you can use #pragma pack. Another option is to use __attribute__((packed)):
#ifndef __GNUC__
# define __attribute__(x)
#endif
struct uint24_t { unsigned long v:24; };
typedef struct uint24_t __attribute__((packed)) uint24_t;
This should work on GCC and Clang.
Note, however, that this will probably screw up alignment unless your processor supports unaligned access.
Initially I thought it was because it is forcing datatypes to be word aligned
Different datatypes can have different alignment. See for example the Objects and alignment doc.
You can use alignof to check, but it's totally normal for char or uint8_t to have 1-byte (ie, effectively no) alignment, but for uint32_t to have 4-bye alignment. I don't know if the alignment of bitfields is explicitly described, but inheriting it from the storage type seems reasonable enough.
NB. The reason for having alignment requirements is generally that it works better with the underlying hardware. If you do use #pragma pack or __attribute__((packed)) or whatever, you may take a performance hit as the compiler - or the memory hardware - silently handle misaligned accesses.
Just explicitly storing a 3-byte array is probably better, IMO.
To begin with, don't use bit-fields or structs. They may include padding as they please and bit-fields are non-portable in general.
Unless your CPU explicitly got 24 bit arithmetic instructions - which doesn't seem very likely unless it's some oddball DSP - then a custom data type will achieve nothing but extra stack bloat.
Most likely you will have to use uint32_t for all arithmetic. Meaning that your 24 bit type might not achieve much when it comes to saving RAM. If you invent some custom ADT with setter/getter (serialization/de-serialization) access, you are probably just wasting RAM since you get higher stack peak usage if the functions can't be inlined.
To actually save RAM you should rather revise your program design.
That being said, you can create a custom type based on an array:
typedef unsigned char u24_t[3];
Whenever you need to access the data, you memcpy it to/from a 32 bit type and then do all arithmetic on 32 bits:
u24_t u24;
uint32_t u32;
...
memcpy(&u32, u24, sizeof(u24));
...
memcpy(&u24, &u32, sizeof(u24));
But note that this assumes little endian, since we are only working with bits 0 to 2. In case of a big endian system, you will have to do memcpy((uint8_t*)&u32+1, ... to discard the MS byte.
I have the following structs in a C program.
typedef struct
{
unsigned char msg_id : 8;
unsigned char msg_num : 8;
} Message_Header_Type;
typedef struct
{
Message_Header_Type header;
int msg[32];
} Raw_Message_Type;
What I'm seeing is that the header in the Raw_Message_Type is taking up 4 bytes but I only want it to take only 2. How would I go about doing this?
What you are looking for is struct packing, which is platform and compiler dependent, the details are not specified in the C standard.
With GCC, on a 32-bit platform, you could do the following to get a struct of size 2:
typedef struct __attribute__((__packed__))
{
unsigned char msg_id;
unsigned char msg_num;
} Message_Header_Type;
From the GCC documentation:
packed
This attribute, attached to an enum, struct, or union type definition, specified that the minimum required memory be used to represent the type.
Message_Header_type inside of Raw_Message_Type is still taking 2 bytes as you would expect, but the compiler is padding the structure for alignment reasons, so your Raw_Message_Type is 132 bytes long instead of the 130 bytes you expected. This is probably because 32-bit aligned accesses are more efficient on your processor.
If you are using gcc, you can tell the compiler to align on 2-byte boundaries instead of the 4-byte boundaries that it is using by using #pragma pack.
typedef struct
{
unsigned char msg_id : 8;
unsigned char msg_num : 8;
} Message_Header_Type;
#pragma pack(push, 2) // save current alignment and set to 2-byte boundaries
typedef struct
{
Message_Header_Type header;
int msg[32];
} Raw_Message_Type;
#pragma pack(pop) // restore the previous alignment
Special packing like this is often necessary when using fixed-size structures in files, but be aware that there may be a (light) performance penalty for using a different packing than what the processor prefers. In this specific case, your 32 msg[] ints are now all 2 bytes off from the preferred alignment for your platform.
C implementations are free to insert padding into struct layouts between members, at the end, or both. It is common for them to do so for alignment purposes. It is also common for compilers to provide a mechanism to influence or override the padding, but details are necessarily implementation-specific.
With GCC, for example, you could apply the packed attribute to both your structs:
typedef struct __attribute__((__packed__))
{
unsigned char msg_id : 8;
unsigned char msg_num : 8;
} Message_Header_Type;
typedef struct __attribute__((__packed__))
{
Message_Header_Type header;
int msg[32];
} Raw_Message_Type;
Some other implementations borrow GCC's approach; some use pragmas for the same purpose; and some provide other mechanisms. You'll need to check your compiler's documentation.
Structure objects are not guaranteed the size of the sum of bits used in member bit fields. This is due to padding.
In your case though Raw_Message_Type has another member int msg[32];. It seems logical to use two more bytes for the purpose alignment so that msg[0] can be aligned to a 4-byte boundary.
There is a good chance that
Message_Header_Type obj;
printf("Size : %zu\n",sizeof(obj));
would give you 2 as a the result.
If I debug the following code then I see the size value is 12 (as expected).
#include <cstdint>
int main(int argc, char *argv[])
{
typedef struct __attribute__((__packed__)) { int8_t value; } time;
typedef struct __attribute__((__packed__)) {
uint8_t msg[8];
// time t1;
uint32_t count;
} theStruct;
theStruct s;
int size = sizeof(s);
return 0;
}
Interestingly, removing the comment at "time t1;", the value of size goes to 16. I was expecting 13.
I know (more or less) that this is explained by the data structure padding story...
But, is there some way to avoid this issue?
What to do in order to read size = 13?
There are some issues with MinGW's emulation of MSVC struct packing.
The workaround is to pass -mno-ms-bitfields flag to the compiler; this will cause it to use its own layout algorithm rather than attempt to emulate MSVC.
See also Struct packing and alignment with mingw (ARM but may be the same issue).
It is clearly an alignement problem, meaning it has nothing to do with the language itself but all with the underlying platform.
If the platform knows (or thinks) that int32_t need an alignement of 4, it should add 3 bytes of padding after time.
Anyway __attribute__((__packed__)) is non standard C and will only be interpreted by gcc (*). Worse it leads to non portable programs : according to this answer on another question, it would cause a bus error on a Sparc architecture because of a misaligned int32_t.
I do know that x86 (and derivatives) architecture are now the most common, but other architecture may still exist ...
(*) according to Visual C++ equivalent of GCC's __attribute__ ((__packed__)), the MSVC equivalent is #pragma pack(push, 1)