I have looked around this site to try to figure out if my use of casting to different unions is violating strict aliasing or otherwise UB.
I have packets coming in on a serial line and I store/get them like:
union uart_data {
struct {
uint8_t start;
uint8_t addr;
uin16_t length;
uint8_t data[];
};
uint8_t bytes[BUFFER_SIZE];
};
void store_byte(uint8_t byte) {
uart_data->start = byte;
/* and so on with the other named fields. */
}
uint8_t * get_buffer() {
return uart_data->bytes;
}
My understanding is that this is, at least with GCC and GNU extensions an valid way to do type punning.
However, I then want to cast the return value from get_buffer() to a more specific type of packet that the uart doesn't need to know the details about.
union spec_pkt {
struct {
uint8_t start;
uint8_t addr;
uin16_t length;
uint8_t command;
uint8_t some_field;
uint16_t data_length;
uint8_t data[];
};
uint8_t bytes[BUFFER_SIZE];
};
void process(uint8_t *data) {
union specific_pkt *pkt = (union specific_pkt *)data;
}
I recall having read somewhere that this is valid since I'm casting from a type that exists in the union but I can't find the source.
My rationale for doing this it this way is that I can have a uart driver that only needs to know about the lowest level details. I'm on an MCU so I only have access to pre-allocated buffers to data and this way I don't have to memcpy between buffers, wasting space. And in my application code I can handle the packet in a nicer way than:
uint8_t data[BUFFER_SIZE];
data[START_POS];
data[LEN_POS];
data[DATA_POS];
If this is violating the SA rule or is UB I'd love some alternatives to achieve the same.
I'm using GCC on a target that supports unaligned access and GCC allows type punning through unions.
The Standard completely fails to specify the circumstances under which a structure or union object may be accessed via a non-character lvalue whose type is not that of the structure or union. If one recognizes that the purpose of the Standard is to purely indicate when a compiler must recognize that an object is being accessed by a seemingly-unrelated lvalue, but is not meant to apply to situations where a compiler would be able to see that an lvalue or pointer of one type is used to derive another which is then used to access storage associated with the first, without any intervening conflicting action on that storage, this omission would make sense. For example, given:
struct sizedPointer { int length,size; int *dat; };
void storeThing(struct sizedPointer *dest, int n)
{
if (dest->length < dest->size)
{
dest->dat[dest->length] = n;
dest->length++;
}
}
such an interpretation would allow a compiler to assume that dest->length will not be written using dest->dat, since its value has been observed after dest->dat was formed, but would require that a compiler recognize that given:
union blob { uint16_t hh[8]; uint64_t oo[2]; } myBblob;
an operation like
sscanf(someString, "%4x", &myBlob.hh[1]);
might interact with any lvalues that are derived from myBlob after the function returns.
Unfortunately, gcc and clang instead interpret the rule as only mandating recognition in cases where failure to do so would completely gut the language. Because the Standard doesn't mandate that member-type lvalues be usable in any fashion whatsoever, and gcc and clang have explicitly stated that they should not be relied upon to do anything beyond what the Standard requires, support for anything useful should be viewed as being at the whim of the maintainers of clang and gcc.
Related
Follow-up to extended discussion in How can I determine if I'm overparenthesizing?
I'm trying to emulate a Z80 in C, where several 8-bit registers can be combined to create 16-bit registers.
This is the logic I'm trying to use:
struct {
uint8_t b;
uint8_t c;
uint16_t *bc;
} regs[1];
...
regs->bc = (uint16_t *)&(regs->b);
Why is this incorrect, and how can I do it correctly (using type-punning if needed)?
I need to do this multiple times, preferably within the same structure.
It's incorrect because b is of type uint8_t and a pointer to uint16_t cannot be used for accessing such a variable. It might not be correctly aligned and it is a strict aliasing violation.
You are however free to do (uint8_t *)®s or (struct reg_t*)®s->b, since (6.7.2.1/15)
A pointer to a structure object, suitably converted, points to its initial member and vice versa.
When doing hardware-related programming, make sure to never use signed types. That means changing intn_t to uintn_t.
As for how to type pun properly, use a union:
typedef union
{
struct /* standard C anonymous struct */
{
uint8_t b;
uint8_t c;
};
uint16_t bc;
} reg_t;
You can then assign this to point at a 16 bit hardware register like this:
volatile reg_t* reg = (volatile reg_t*)0x1234;
where 0x1234 is the hardware register address.
NOTE: this union is endianess-dependent. b will access the MS byte of bc on big endian systems, but the LS byte of bc on little endian systems.
To emulate a hardware register that can be accessed as two eight-bit registers or one 16-bit register, you can use:
union
{
struct { int8_t b, c; };
int16_t bc;
} regs[1];
Then regs->bc will be the 16-bit register, and regs->b and regs->c will be 8-bit registers.
Note: This uses an anonymous struct so that b and c appears as if they were members of the union. If the struct had a name, like this:
union
{
struct { int8_t b, c; } s;
int16_t bc;
} regs[1];
then you would have to include its name when accessing b or c, as with regs->s.b. However, C has a feature that allows you to use a declaration without a name for this purpose.
Also note this requires a C compiler. C allows using unions to reinterpret data. C++ has different rules.
The correct way is through anonymous unions in C as already shown in other answers. But as you want to process bytes, you may use the special handling of characters in the strict aliasing rule: whatever the type, is is always legal to use a char pointer to access the bytes of its representation. So this is conformant C
struct {
uint16_t bc;
uint8_t *b;
uint8_t *c;
} regs[1];
regs->b = (uint8_t *) &(regs->bc);
regs->c = regs->b + 1
Interestingly enough, it is still valid for a C++ compiler...
The correct way to type-pun in C (or do almost anything, for that matter), is to use an implementation that is configured to be suitable for one's intended purpose. The Standard deliberately allows implementations that are intended for various purposes to behave in ways that would make them unsuitable for other purposes. According to the authors, it was never intended to suggest that programs whose behavior isn't mandated by the Standard (but would be defined on the implementations for which they were intended) should be viewed as "broken". Compilers whose authors seek to support the needs of their customers will recognize straightforward type-punning constructs whether or not the Standard requires them to do so, and optimizers who authors view their customers' needs with contempt should not be trusted to reliably handle anything complicated.
What most people are concerned about is what happens if they receive a byte array with data and they want to cast the array to a struct pointer - this can violate strict aliasing rules. I'm not sure whether initializing an empty byte array of sufficient size, casting it to a struct pointer, and then populate the struct members would violate the strict aliasing rules.
The details:
Say I have 2 packed structs:
#pragma pack(1)
typedef struct
{
int a;
char b[2];
uint16_t c : 8;
uint16_t d : 7;
uint16_t e : 1;
} in_t;
typedef struct
{
int x;
char y;
} out_t;
#pragma pack()
I have many types of in/out packed structs for different messages so please ignore the specific members I put for the example. The structs can contain bitfields, other structs, and unions. Also, endianess is taken care of. Also, I can't use new c standards (>= c99) features.
I'm receiving a buffer containing in_t (the buffer is large enough to contain out_t, however big it'll be) as void *
void recv_msg(void *data)
{
in_t *in_data = (in_t*)data;
out_t *out_data = (out_t*)data;
// ... do something with in_data then set values in out_t.
// make sure values aren't overwritten.
}
Now I have a new type of in struct
#pragma pack(1)
typedef struct
{
int a;
char b[3];
uint32_t c;
} in_new_api_t;
typedef struct
{
int x;
char y[2];
} out_new_api_t;
#pragma pack()
Now, when moving to the new api but keeping the old api for backward compatibility, I want to copy values from the old in_t to in_new_api_t, use in_new_api_t, set values in out_new_api_t, and then copy the values to out_t.
The way I thought of doing it is to allocate an empty byte array the size of max(sizeof(in_new_api_t), sizeof(out_new_api_t));, cast it to in_new_api_t *, translate values from in_t to in_new_api_t, send the new api struct to the new api function, then translate values from out_new_api_t to out_t.
void recv_msg(void *data)
{
uint8_t new_api_buf[max(sizeof(in_new_api_t), sizeof(out_new_api_t))] = {0};
in_new_api_t *new_in_data = (in_new_api_t*)new_api_buf;
in_t *in_data = (in_t*)data;
// ... copy values from in_data to new_in_data
// I'M NOT SURE I CAN ACCESS MEMBERS OF new_in_data WITHOUT VIOLATING STRICT ALIASING RULES.
new_in_data->a = in_data->a;
memcpy(new_in_data->b, in_data->b, 2);
// ...
new_recv_msg((void*)new_in_data);
out_new_api_t *new_out_data = (out_new_api_t*)new_api_buf;
out_t *out_data = (out_t*)data;
// ... copy values from new_out_data to out_data
}
The point I'm just not sure about is whether casting from 'uint8_t []' to 'in_new_api_t *' would violate the strict aliasing rules or cause any other issues. Also Access performance issues are a concern.
And if so, what is the best solution?
I can make copies of in_t and out_t and make in_new_api_t point to data but then I need to copy the data 4 times to make sure I'm not overwriting values: from data to in_t tmp_in, from tmp_in to in_new_api, then from out_new_api to out_t tmp_out and from tmp_out to out_t out.
It sounds like what you want is a couple of union types. The common initial sequences of the struct members of a union are layout-compatible, per the standard, and can be mapped onto each other, in exactly the same way as the family field of every sockaddr_* type. Type-punning on a union is legal in C, but not in C++ (although it works with POD on every compiler, no compiler that tries to be compatible with existing code will ever break it, and any possible alternative is undefined behavior too). This might possibly obviate the need for a copy.
A union is guaranteed to be properly-aligned for both. If you do use pointers, it is probably a good idea to Alignas the object to both types, just in case.
A memcpy() to and from arrays of unsigned char is also legal; the language standards call the contents of the array after the copy the object representation.
It is fairly straight-forward:
Casting to a pointer-to-struct type, when the pointed-at data by the void* is of any different type, is a strict aliasing violation.
Casting to a pointer-to-struct from a pointer to raw character buffer is a strict aliasing violation. (You may however go the other way around: from pointer-to-struct into pointer-to-char.)
So your code looks wildly unsafe and is also a bit confusing because of the void pointer. So number one is to get rid of that icky, dangerous void pointer! You can create a type such as:
typedef union
{
in_t old;
in_new_api_t new;
uint8_t bytes [sizeof(in_new_api_t)];
} in_api_t;
Then use this as parameter to your function.
This will first of all allow you to access the initial parts of each struct in a safe manner that doesn't violate aliasing (6.5.2.3, the rule about common initial sequence). That is, the members a and b will correspond to each other in both structs. The only thing you can't rely on is the members that aren't the same - those will have to be copied explicitly with memcpy.
Second, you can now use the bytes member when you need to serialize the data. If you write the "out" structures as unions in a similar manner, and they too contain a bytes member of exactly the same size, you can safely cast from one type to the other, without strict aliasing violations. This is allowed by C11 6.5:
An object shall have its stored value accessed only by an lvalue expression that has one of the following types:
- a type compatible with the effective type of the object
/--/
- an aggregate or union type that includes one of the aforementioned types among its members
If your union is accessed by a pointer to union type, that includes a byte array of exactly the same size (a compatible type), then that's allowed.
What you are doing in recv_msg() clearly is undefined behaviour and will likely break your code some day, as the compiler is entitled to do whatever it wants when moving from *in_data to *out_data. Also, if the void* data argument doesn't come from either a malloc() (and cousins) or from an object that originally was an in_t then you have UB and alignment problems even there.
Your method to save RAM is extremely risky. Even if you are bold enough to ignore the more theoretical UB case of accessing memory with an illegal but correctly aligned type, you still will get problems as there simply is no guarantee that the order of operations of copying in-place from one struct to the other won't trash your data.
I'm fixing some code not written by me, so I found this:
#define get_u_int16_t(X,O) (*(u_int16_t *)(((u_int8_t *)X) + O))
How can I change it to keep the rule, if it violating it ?
The macro is called in this way:
if(get_u_int16_t(packet->payload, i) == ...) { ... }
where payload is a const unsigned char * and i is an unsigned int .
The situation is:
struct orig {
[...]
struct pkt packet;
}*;
struct pkt {
[...]
const u_int8_t *payload;
}*;
Called in this way:
struct orig * flow;
struct pkt * packet = &flow->packet;
payload is a string
i begins with a value of 0 and it is inside a for that loop for the lenght of payload ( u_int16_t len ):
for(i = 0; i < len; i++) {
if(get_u_int16_t(packet->payload, a) == /*value*/) {
// do stuff
}
The macro itself doesn't violate the strict aliasing rule; it depends how you use it. If you only use it to read already existing objects of type u_int16_t or a compatible type then it's fine; if on the other hand you use it to read e.g. parts of a 64-bit integer or a floating-point object then that would be a strict aliasing violation, as well as (possibly) an alignment violation.
As always, you can make the code safe using memcpy:
inline u_int16_t read_u_int16_t(const void *p) {
u_int16_t val;
memcpy(&val, p, sizeof(val));
return val;
}
#define get_u_int16_t(X,O) (read_u_int16_t(((u_int8_t *)X) + O))
As #2501 points out this may be invalid if u_int8_t is not a character type, so you should just use char * for pointer arithmetic:
#define get_u_int16_t(X,O) (read_u_int16_t(((char *)X) + O))
There are two ways to write code of the type you're interested in:
Use pointers of type "unsigned char*" to access anything, assemble values out of multiple bytes, and tolerate the performance hit.
If you know that pointer alignment won't be an issue, use a dialect of C that doesn't semantically gut it. GCC can implement such a dialect if invoked with -fno-strict-aliasing, but since gcc has no way to prevent obtuse "optimizations" without blocking useful and sensible optimizations, getting good performance from such a dialect may require learning how to use restrict.
Some people would argue that approach #1 is better, and for many PC-side programming purposes I'd tend to agree. For embedded software, however, I would suggest staying away from gcc's default dialect since there's no guarantee that the things gcc regards as defined behavior today will remain so tomorrow. I'm not sure where the authors of gcc got the notion that the authors of the Standard were trying to specify all of the forms of aliasing that a quality microprocessor implementation should support, rather than establish a minimum baseline for platforms where no forms of type punning other than "unsigned char" would be useful.
Our team is currently using some ported code from an old architecture to a new product based on the ARM Cortex M3 platform using a customized version of GCC 4.5.1. We are reading data from a communications link, and attempting to cast the raw byte array to a struct to cleanly parse the data. After casting the pointer to a struct and dereferencing, we are getting a warning: "dereferencing type-punned pointer will break strict-aliasing rules".
After some research, I've realized that since the char array has no alignment rules and the struct have to be word aligned, casting the pointers causes undefined behavior (a Bad Thing). I'm wondering if there is a better way to do what we're trying.
I know we can explicitly word-align the char array using GCC's "attribute ((aligned (4)))". I believe this will make our code "safer", but the warnings will still clutter up our builds, and I don't want to disable the warnings in case this situation arises again. What we want is a way to safely do what we are trying, that will still inform us if we attempt to do something unsafe in another place later. Since this is an embedded system, RAM usage and flash usage are important to some degree.
Portability (compiler and architecture) is not a huge concern, this is just for one product. However, if a portable solution exists, it would be preferred.
Here is the a (very simplified) example of what we are currently doing:
#define MESSAGE_TYPE_A 0
#define MESSAGE_TYPE_B 1
typedef struct MessageA __attribute__((__packed__))
{
unsigned char messageType;
unsigned short data1;
unsigned int data2;
}
typedef struct MessageB __attribute__((__packed__))
{
unsigned char messageType;
unsigned char data3;
unsigned char data4;
}
// This gets filled by the comm system, assume from a UART interrupt or similar
unsigned char data[100];
// Assume this gets called once we receive a full message
void ProcessMessage()
{
MessageA* messageA;
unsigned char messageType = data[0];
if (messageType == MESSAGE_TYPE_A)
{
// Cast data to struct and attempt to read
messageA = (MessageA*)data; // Not safe since data may not be word aligned
// This may cause undefined behavior
if (messageA->data1 == 4) // warning would be here, when we use the data at the pointer
{
// Perform some action...
}
}
// ...
// process different types of messages
}
As has already been pointed out, casting pointers about is a dodgy practice.
Solution: use a union
struct message {
unsigned char messageType;
union {
struct {
int data1;
short data2;
} A;
struct {
char data1[5];
int data2;
} B;
} data;
};
void func (...) {
struct message msg;
getMessage (&msg);
switch (msg.messageType) {
case TYPEA:
doStuff (msg.data.A.data1);
break;
case TYPEB:
doOtherStuff (msg.data.B.data1);
break;
}
}
By this means the compiler knows you're accessing the same data via different means, and the warnings and Bad Things will go away.
Of coure, you'll need to make sure the structure alignment and packing matches your message format. Beware endian issues and such if the machine on the other end of the link doesn't match.
Type punning through cast of types different than char * or a pointer to a signed/unsigned variant of char is not strictly conforming as it violates C aliasing rules (and sometimes alignment rules if no care is given).
However, gcc permits type punning through union types. Manpage of gcc explicitly documents it:
The practice of reading from a different union member than the one most recently written to (called "type-punning") is common. Even with
-fstrict-aliasing, type-punning is allowed, provided the memory is accessed through the union type.
To disable optimizations related to aliasing rules with gcc (and thus allow the program to break C aliasing rules), the program can be compiled with: -fno-strict-aliasing. Note that with this option enabled, the program is no longer strictly conforming, but you said portability is not a concern. For information, the Linux kernel is compiled with this option.
GCC has a -fno-strict-aliasing flag that will disable strict-aliasing-based optimizations and make your code safe.
If you're really looking for a way to "fix" it, you have to rethink the way your code works. You can't just overlay the structure the way you're trying, so you need to do something like this:
MessageA messageA;
messageA.messageType = data[0];
// Watch out - endianness and `sizeof(short)` dependent!
messageA.data1 = (data[1] << 8) + data[2];
// Watch out - endianness and `sizeof(int)` dependent!
messageA.data2 = (data[3] << 24) + (data[4] << 16)
+ (data[5] << 8) + data[6];
This method will let you avoid packing your structure, which might also improve its performance characteristics elsewhere in your code. Alternately:
MessageA messageA;
memcpy(&messageA, data, sizeof messageA);
Will do it with your packed structures. You would do the reverse operations to translate the structures back into a flat buffer if necessary.
Stop using packed structures and memcpy the individual fields into variables of the correct size and type. This is the safe, portable, clean way to do what you're trying to achieve. If you're lucky, gcc will optimize the tiny fixed-size memcpy into a few simple load and store instructions.
The Cortex M3 can handle unaligned accesses just fine. I have done this in similar packet processing systems with the M3. You don't need to do anything, you can just use the flag -fno-strict-aliasing to get rid of the warning.
For unaligned accesses, look at the linux macros get_unaligned/put_unaligned.
Other then memory casting tricks is there any way to use an untagged union
(a data type that explicitly hold one of a set of types that isn't a tagged union,
ie. one that is forced by the compiler to hold an associated type tag and possibly only allowed by the language to get the value of the proper type)
without an associated type tag in the container that holds it?
Are there any other advantage an untagged union holds over a typed union?
edit: to show what I mean example tagged union in haskell
data U = I Int | S String
manually tagged union in c
enum u_types {INT,STRING};
typedef struct {
u_types tag;
union u{
int i;
char s[STRING_BUFFER_SIZE];
} d;
}tagged union;
untagged union in c
union u{
int i;
char s[STRING_BUFFER_SIZE];
} d;
One use for untagged unions is to allow easy access to smaller parts of a larger type:
union reg_a {
uint32_t full;
struct { /* little-endian in this example */
uint16_t low;
uint16_t high;
} __attribute__((__packed__));
};
union reg_a a;
a.full = 0x12345678; /* set all whole 32-bits */
a.high = 0xffff; /* change the upper 16-bits */
union pix_rgba {
uint32_t pix; /* to access the whole 32-bit pixel at once */
struct {
uint8_t red; /* red component only */
uint8_t green; /* green component only */
uint8_t blue; /* blue only */
uint8_t alpha; /* alpha only */
} __attribute__((__packed__));
};
These sorts of uses are not necessarily completely portable though, since they may depend on specific representations of types, endianness, etc. Still, they're often portable enough that one or two alternate versions will cover all the platforms one cares about, and they can be quite useful.
Untagged unions are also useful when what is stored in the union will be known anyway, even without checking the tag, and you don't want the extra overhead of storing and updating a tag. Possibly information in another place, that may also serve another purpose, may indicate what kind of data should be in the union -- in which case there's no need tag the union itself.
You don't need the tag if you only plan to define one variable, or use it inside a struct.
For example:
union
{
int x;
int y;
} u;
void test(void)
{
u.x = 10;
}
You only need the tag if you plan to use it in more than one place, if you need to create a pointer to it etc.
Note: The answer above assumed that the question was about what the standard calls a tag. However, after the answer was given, the question was updated to indicate that the tag in question was an extra type field used to record which of the fields in the union was active.
What you have posted as "manually tagged" is not valid C syntax , I suppose you meant it to be:
typedef enum {INT,STRING} u_types;
typedef struct {
u_types tag;
union u{
int i;
char s[1];
} d;
}tagged_union;
Please note that the formal definition of a struct/union tag in C is the name after the struc/union keyword. In your second example, u is a union tag. This is what had me quite confused.
What you describe as "tagged union" is known as a variant in computer science: a variable which can hold multiple types. Variants are generally frowned upon in programming in general and in C in particular. They are banned in MISRA-C:2004, rules 18.3 and 18.4.
Languages with support for variants, like VB (and probably Haskell?) typically present then as: this variable can hold anything, but you should be careful with using it, because it is very inefficient.
In C, variants are not only inefficient, they are a safety hazard. MISRA-c recognizes this in rule 18.3:
For example: a program might try to access data of one type from the location when actually it is storing a value of the other type (e.g. due to an interrupt). The two types of data may align differently in the storage, and encroach upon other data. Therefore the data may not be correctly initialised every time the usage switches. The practice is particularly dangerous in concurrent systems.
So the question should rather be, are there any uses for tagged unions (variants)? No, there is not. I haven't used a single one in any C program I have ever written, there is no use for them. Since C has void pointers, there are far better and safer ways in C to create generic data types:
void ADT_add_generic_type (void* data, some_enum_t type, size_t size);
Take a look at how the C standard implements the functions qsort() and bsearch() for some good examples of generic C programming (ISO 9899:1999 7.20.5.1):
void *bsearch (const void *key,
const void *base,
size_t nmemb,
size_t size,
int (*compar)(const void *, const void *));
Description The bsearch function searches an array of nmemb objects,
the initial element of which is pointed to by base, for an element
that matches the object pointed to by key. The size of each element of
the array is specified by size.
The uses for "untagged" unions are several, however. Data protocols, packing, hardware register access etc etc. See Dmitri's answer for a good example.
Here's a trick:
static __attribute__((const, always_inline))
int32_t floatToIntBits(float f) {
union {
float value;
int32_t bits;
};
value = f;
return bits;
}
Wherever you have a generic implementation using void *, you could use an untagged union instead. Since you are using void * the true object type has to be known from context.
This is a maximally portable way to implement a generic datastructure that can store union { void *ptr; unsigned x; } for example (on C platforms where there is no uintptr_t).