casting from empty byte array to struct pointer can violate strict aliasing? - c

What most people are concerned about is what happens if they receive a byte array with data and they want to cast the array to a struct pointer - this can violate strict aliasing rules. I'm not sure whether initializing an empty byte array of sufficient size, casting it to a struct pointer, and then populate the struct members would violate the strict aliasing rules.
The details:
Say I have 2 packed structs:
#pragma pack(1)
typedef struct
{
int a;
char b[2];
uint16_t c : 8;
uint16_t d : 7;
uint16_t e : 1;
} in_t;
typedef struct
{
int x;
char y;
} out_t;
#pragma pack()
I have many types of in/out packed structs for different messages so please ignore the specific members I put for the example. The structs can contain bitfields, other structs, and unions. Also, endianess is taken care of. Also, I can't use new c standards (>= c99) features.
I'm receiving a buffer containing in_t (the buffer is large enough to contain out_t, however big it'll be) as void *
void recv_msg(void *data)
{
in_t *in_data = (in_t*)data;
out_t *out_data = (out_t*)data;
// ... do something with in_data then set values in out_t.
// make sure values aren't overwritten.
}
Now I have a new type of in struct
#pragma pack(1)
typedef struct
{
int a;
char b[3];
uint32_t c;
} in_new_api_t;
typedef struct
{
int x;
char y[2];
} out_new_api_t;
#pragma pack()
Now, when moving to the new api but keeping the old api for backward compatibility, I want to copy values from the old in_t to in_new_api_t, use in_new_api_t, set values in out_new_api_t, and then copy the values to out_t.
The way I thought of doing it is to allocate an empty byte array the size of max(sizeof(in_new_api_t), sizeof(out_new_api_t));, cast it to in_new_api_t *, translate values from in_t to in_new_api_t, send the new api struct to the new api function, then translate values from out_new_api_t to out_t.
void recv_msg(void *data)
{
uint8_t new_api_buf[max(sizeof(in_new_api_t), sizeof(out_new_api_t))] = {0};
in_new_api_t *new_in_data = (in_new_api_t*)new_api_buf;
in_t *in_data = (in_t*)data;
// ... copy values from in_data to new_in_data
// I'M NOT SURE I CAN ACCESS MEMBERS OF new_in_data WITHOUT VIOLATING STRICT ALIASING RULES.
new_in_data->a = in_data->a;
memcpy(new_in_data->b, in_data->b, 2);
// ...
new_recv_msg((void*)new_in_data);
out_new_api_t *new_out_data = (out_new_api_t*)new_api_buf;
out_t *out_data = (out_t*)data;
// ... copy values from new_out_data to out_data
}
The point I'm just not sure about is whether casting from 'uint8_t []' to 'in_new_api_t *' would violate the strict aliasing rules or cause any other issues. Also Access performance issues are a concern.
And if so, what is the best solution?
I can make copies of in_t and out_t and make in_new_api_t point to data but then I need to copy the data 4 times to make sure I'm not overwriting values: from data to in_t tmp_in, from tmp_in to in_new_api, then from out_new_api to out_t tmp_out and from tmp_out to out_t out.

It sounds like what you want is a couple of union types. The common initial sequences of the struct members of a union are layout-compatible, per the standard, and can be mapped onto each other, in exactly the same way as the family field of every sockaddr_* type. Type-punning on a union is legal in C, but not in C++ (although it works with POD on every compiler, no compiler that tries to be compatible with existing code will ever break it, and any possible alternative is undefined behavior too). This might possibly obviate the need for a copy.
A union is guaranteed to be properly-aligned for both. If you do use pointers, it is probably a good idea to Alignas the object to both types, just in case.
A memcpy() to and from arrays of unsigned char is also legal; the language standards call the contents of the array after the copy the object representation.

It is fairly straight-forward:
Casting to a pointer-to-struct type, when the pointed-at data by the void* is of any different type, is a strict aliasing violation.
Casting to a pointer-to-struct from a pointer to raw character buffer is a strict aliasing violation. (You may however go the other way around: from pointer-to-struct into pointer-to-char.)
So your code looks wildly unsafe and is also a bit confusing because of the void pointer. So number one is to get rid of that icky, dangerous void pointer! You can create a type such as:
typedef union
{
in_t old;
in_new_api_t new;
uint8_t bytes [sizeof(in_new_api_t)];
} in_api_t;
Then use this as parameter to your function.
This will first of all allow you to access the initial parts of each struct in a safe manner that doesn't violate aliasing (6.5.2.3, the rule about common initial sequence). That is, the members a and b will correspond to each other in both structs. The only thing you can't rely on is the members that aren't the same - those will have to be copied explicitly with memcpy.
Second, you can now use the bytes member when you need to serialize the data. If you write the "out" structures as unions in a similar manner, and they too contain a bytes member of exactly the same size, you can safely cast from one type to the other, without strict aliasing violations. This is allowed by C11 6.5:
An object shall have its stored value accessed only by an lvalue expression that has one of the following types:
- a type compatible with the effective type of the object
/--/
- an aggregate or union type that includes one of the aforementioned types among its members
If your union is accessed by a pointer to union type, that includes a byte array of exactly the same size (a compatible type), then that's allowed.

What you are doing in recv_msg() clearly is undefined behaviour and will likely break your code some day, as the compiler is entitled to do whatever it wants when moving from *in_data to *out_data. Also, if the void* data argument doesn't come from either a malloc() (and cousins) or from an object that originally was an in_t then you have UB and alignment problems even there.
Your method to save RAM is extremely risky. Even if you are bold enough to ignore the more theoretical UB case of accessing memory with an illegal but correctly aligned type, you still will get problems as there simply is no guarantee that the order of operations of copying in-place from one struct to the other won't trash your data.

Related

Is address of structure members defined behavior?

Sometimes I do pointer arithmetic on structures and I'm not certain if it's defined behavior or not..
If I have this structure
typedef struct SCustomData{
uint32_t a;
uint32_t b;
}CustomData;
Rather than having to constantly type out different fields of the structure sometimes to me it makes sense to just give it a constant size and fit whatever I need to in there.
My first question is: Is it defined behavior that the address of the structure is also the address of the first variable in the structure?
int main(){
CustomData cd = {0};
cd.a = 0xDEADBEEF;
uint32_t a = 0;
memcpy(&a,&cd,sizeof(uint32_t));
assert(a==0xDEADBEEF);
return 0;
}
Will that always succeed?
My next question is, can I use pointer arithmetic to access any member of the structure so long as I know the sizes of each member?
int main(){
CustomData cd = {0};
cd.b = 0xABCD0000;
uint16_t highshort = *(uint16_t*)((uint8_t*)&cd+sizeof(uint32_t)+sizeof(uint16_t));
assert(highshort == 0xABCD);
uint8_t highbyte = *(uint8_t*)((uint8_t*)&cd+sizeof(uint32_t)+sizeof(uint16_t)+sizeof(uint8_t));
assert(highbyte == 0xAB);
return 0;
}
Aside from maybe looking ugly or bad code practice, is at least defined behavior? Should I avoid it?
And if I didn't use pointer casting and used memcpy instead is it still a bad idea?
int main(){
CustomData cd = {0};
cd.b = 0xABCD0000;
uint16_t highshort = 0;
memcpy(&highshort, ((uint8_t*)&cd)+sizeof(uint32_t)+sizeof(uint16_t), sizeof(uint16_t));
assert(highshort == 0xABCD);
return 0;
}
First Question
It's actually is defined in the C standard! Section 6.7.2.1 (in this standard draft)
15 Within a structure object, the non-bit-field members and the units in which bit-fields
reside have addresses that increase in the order in which they are declared. A pointer to a
structure object, suitably converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.
Second Question
It's not defined according to the standard (again section 6.7.2.1):
Each non-bit-field member of a structure or union object is aligned in an implementation-
defined manner appropriate to its type.
However, most compilers align the same way (per-architecture and bitness at least)
Why It's still a bad idea
Maintainability - Your code will be much easier to maintain if you use the members themselves. It won't be dependent on the compiler and bitness, and won't be sensitive to changes in your code (such as changing types or reordering members).
Readability - Your intentions will of course be much clearer.
Sometimes compilers do weird stuff ;)

C: Is accessing initial member of nested struct using pointer cast to "outer" struct type defined? [duplicate]

This question already has answers here:
Are C-structs with the same members types guaranteed to have the same layout in memory?
(4 answers)
Closed 1 year ago.
I'm trying to understand the so-called "common initial sequence" rule for C aliasing analysis. This question does not concern C++.
Specifically, according to resources (for example the CPython PEP 3123),
[A] value of a struct type may also be accessed through a pointer to the first field. E.g. if a struct starts with an int, the struct * may also be cast to an int *, allowing to write int values into the first field.
(emphasis mine).
My question can be roughly phrased as "does the ability to access a struct by pointer to first-member-type pierce nested structs?" That is, what happens if access is via a pointer whose pointed-to type (let's say type struct A) isn't exactly the same type as that of the first member (let's say type struct B), but that pointed-to type (struct A) has common first initial sequence with struct B, and the "underlying" access is only done to that common initial sequence?
(I'm chiefly interested in structs, but I can imagine this question may also pertain to unions, although I imagine unions come with their own tricky bits w.r.t. aliasing.)
This phrasing may not clear, so I tried to illustrate my intention with the code as follows (also available at godbolt.org, and the code seem to compile just fine with the intended effect):
/* Base object as first member of extension types. */
struct base {
unsigned int flags;
};
/* Types extending the "base" by including it as first member */
struct file_object {
struct base attr;
int index;
unsigned int size;
};
struct socket_object {
struct base attr;
int id;
int type;
int status;
};
/* Another base-type with an additional member, but the first member is
* compatible with that of "struct base" */
struct extended_base {
unsigned int flags;
unsigned int mode;
};
/* A type that derives from extended_base */
struct extended_socket_object {
struct extended_base e_attr; /* Using "extended" base here */
int e_id;
int e_type;
int e_status;
int some_other_field;
};
/* Function intended for structs "deriving from struct base" */
unsigned int set_flag(struct base *objattr, unsigned int flag)
{
objattr->flags |= flag;
return objattr->flags;
}
extern struct file_object *file;
extern struct socket_object *sock;
extern struct extended_socket_object *esock;
void access_files(void)
{
/* Cast to pointer-to-first-member-type and use it */
set_flag((struct base *)file, 1);
set_flag((struct base *)sock, 1);
/* Question: is the following access defined?
* Notice that it's cast to (struct base *), rather than
* (struct extended_base *), although the two structs share the same common
* initial member and it is this member that's actually accessed. */
set_flag((struct base *)esock, 1);
return;
}
This is not safe as you're attempting to access an object of type struct extended_base as though it were an object of type struct base.
However, there are rules that allow access to two structures initial common sequence via a union. From section 6.5.2.3p6 of the C standard:
One special guarantee is made in order to simplify the use of unions: if a union contains several structures that share a common initial sequence (see below), and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union is visible. Two structures share a common initial sequence if corresponding members have compatible types (and, for bit-fields, the same widths) for a sequence of one or more initial members
So if you change the definition of struct extended_socket_object to this:
struct extended_socket_object {
union u_base {
struct base b_attr;
struct extended_base e_attr;
};
int e_id;
int e_type;
int e_status;
int some_other_field;
};
Then a struct extended_socket_object * may be converted to union u_base * which may in turn be converted to a struct base *. This is allowed as per section 6.7.2.1 p15 and p16:
15 Within a structure object, the non-bit-field members and the units in which bit-fields reside have addresses that increase
in the order in which they are declared. A pointer to a structure
object, suitably converted, points to its initial member (or
if that member is a bit-field, then to the unit in which it
resides), and vice versa. There may be unnamed padding within
a structure object, but not at its beginning.
16 The size of a union is sufficient to contain the largest of its members. The value of at most one of the
members can be stored in a union object at any time. A
pointer to a union object, suitably converted, points to each
of its members (or if a member is a bit-field, then to the
unit in which it resides), and vice versa.
It is then allowed to access b_attr->flags because of the union it resides in via 6.5.2.3p6.
According to the C Standard (6.7.2.1 Structure and union specifiers, paragraph 13):
A pointer to a structure object, suitably converted, points to its
initial member (or if that member is a bit-field, then to the unit in
which it resides), and vice versa.
So, converting esock to struct extended_base * and then converting it to unsigned int * must give us a pointer to the flags field, according to the Standard.
I'm not sure if converting to to struct base * counts as "suitably converted" or not. My guess is that it would work at any machine you will try it on, but I wouldn't recommend it.
I think it would be safest (and also make the code more clear) if you simply keep a member of type struct base inside struct extended_base (instead of the member of type unsigned int). After doing that, you have two options:
When you want to send it to a function, write explicitly: esock->e_attr.base (instead of (struct base *)esock). This is what I would recommend.
You can also write: (struct base *) (struct extended_base *) esock which is guaranteed to work, but I think it is less clear, and also more dangerous (if in the future you will want to add or accidentaly add another member in the beginning of the struct).
After reading up into the standard's text following the other answers (thanks!!) I think I may try to answer my own question (which was a bit misleading to begin with, see below)
As the other answers pointed out, there appear to be two somewhat overlapping concerns in this question -
"common initial sequence" -- in the standard documents this specifically refers to the context of a union having several structs as member and when these member structs share some compatible members beginning from the first. (§6.5.2.3 " Structure and union members", p6 -- Thanks, #dbush!).
My reading: the language spec suggests that, if at the site of access to these "apparently" different structs it is made clear that they actually belong to the same union, and that the access is done through the union, it is permitted; otherwise, it is not.
I think the requirement is meant to work with type-based aliasing rules: if these structs do indeed alias each other, this fact must be made clear at compile time (by involving the union). When the compiler sees pointers to different types of structs, it can't, in the most general case, deduce whether they may have belonged to some union somewhere. In that case, if it invokes type-based alias analysis, the code will be miscompiled. So the standard requires that the union is made visible.
"a pointer (to struct), when suitably converted, points to its initial member" (§6.7.2.1 "Structure and union specifiers", p15) -- this sounds tantalizingly close to 1., but it's less about aliasing than about a) the implementation requirements for struct and b) "suitable conversion" of pointers. (Thanks, #Orielno!)
My reading: the "suitable conversion" appears to mean "see everything else in the standard", that is, no matter if the "conversion" is performed by type cast or assignment (or a series of them), being "suitable" suggests "all constraints must be satisfied at all steps". The "initial-member" rule, I think, simply says that the actual location of the struct is exactly the same as the initial member: there cannot be padding in front of the first member (this is explicitly stated in the same paragraph).
But no matter how we make use of this fact to convert pointers, the code must still be subject to constraints governing conversion, because a pointer is not just a machine representation of some location -- its value still has to be correctly interpreted in the context of types. A counterexample would be a conversion involving an assignment that discards const from the pointed-to type: this violates a constraint and cannot be suitable.
The somewhat misleading thing in my original post was to suggest that rule 2 had something to do with "common initial sequence", where it is not directly related to that concept.
So for my own question, I tend to answer, to my own surprise, "yes, it is valid". The reason is that the pointer conversion by cast in expression (struct base *)esock is "legal in the letter of the law" -- the standard simply says that (§6.5.4 "Cast operators", p3)
Conversions that involve pointers, other than where permitted by the constraints of 6.5.16.1 (note: constraints governing simple assignment), shall be specified by means of an explicit cast.
Since the expression is indeed an explicit cast, in and by itself it doesn't contradict the standard. The "conversion" is "suitable". Further function call to set_flag() correctly dereferences the pointer by virtue of the suitable conversion.
But! Indeed the "common initial sequence" becomes important when we want to improve the code. For example, in #dbush's answer, if we want to "inherit from multiple bases" via union, we must make sure that access to base is done where it's apparent that the struct is a member of the union. Also, as #Orielno pointed out, when the code makes us worry about its validity, perhaps switching to an explicitly safe alternative is better even if the code is valid in the first place.
In the language the C Standard was written to describe, an lvalue of the form ptr->memberName would use ptr's type to select a namespace in which to look up memberName, add the offset of that member to the address in ptr, and then access an object of that member type at that address. Once the address and type of the member were determined, the original structure object would play no further rule in the processing of the expression.
When C99 was being written, there was a desire to avoid requiring that a compiler given something like:
struct position {double x,y,z; };
struct velocity {double dx,dy,dz; };
void update_positions(struct positions *pp, struct velocity *vv, int count)
{
for (int i=0; i<count; i++)
{
positions[i].x += vv->dx;
positions[i].y += vv->dy;
positions[i].z += vv->dz;
}
}
must allow for the possibility that a write to e.g. positions[i].y might affect the object of vv->dy even when there is no evidence of any relationship between any object of type struct position and any object of type struct velocity. The Committee agreed that compilers shouldn't be required to accommodate interactions between different structure types in such cases.
I don't think anyone would have seriously disputed the notion that in situations where storage is accessed using a pointer which is freshly and visibly converted from one structure type to another, a quality compiler should accommodate the possibility that the operation might access a structure of the original type. The question of exactly when an implementation would accommodate such possibilities should depend upon what its customers were expecting to do, and was thus left as a quality-of-implementation issue outside the Standard's jurisdiction. The Standard wouldn't forbid implementations from being willfully blind to even the most obvious cases, but that's because the dumber something would be, the less need there should be to prohibit it.
Unfortunately, the authors of clang and gcc have misinterpreted the Standard's failure to forbid them from being obtusely blind to the possibility that a freshly-type-converted pointer might be used to access the same object as a pointer of the original type, as an invitation to behave in such fashion. When using clang or gcc to process any code which would need to make use of the Common Initial Sequence guarantees, one must use -fno-strict-aliasing. When using optimization without that flag, both clang nor gcc are prone to behave in ways inconsistent with any plausible interpretation of the Standard's intent. Whether one views such behaviors as being a result of a really weird interpretation of the Standard, or simply as bugs, I see no reason to expect that gcc or clang will ever behave meaningfully in such cases.

Struct type aliasing / tagged-union without union

For two (or more) structs: Base and Sub with a common first (unnamed) struct, is it safe to convert/cast from Base to Sub and vice versa?
struct Base{
struct{
int id;
// ...
};
char data[]; // necessary?
}
struct Sub{
struct{
int id;
// same '...'
};
// actual data
};
Are these functions guaranteed to be safe and technically correct? (Also: is the zero-length char data[] member necessary and useful?)
struct Base * subToBase(struct Sub * s){
return (struct Base*)s;
}
struct Sub * baseToSub(struct Base * b){
if(b->id != SUB_ID){
return NULL;
}
return (struct Sub*)b;
}
Edit
I have no plans to nest any further than Base within Sub, but rather leave the possibility to add other sub-types (directly under Base) later without needing to change Base. My main concern is whether pointers to the structs can be safely converted back and forth between Base and any sub. References to the (C11) standard would be most appreciated.
Edit v2
Changed the wording slightly to discourage OOP/inheritance discussions. What I want is a tagged-union, without the union so it can be extended later. I have no plans for doing additional nesting. Sub-types that need other sub-types' functionality can do so explicitly, without doing any further nesting.
Context
For a script interpreter1 I have made a pseudo object-oriented tagged-union type system, without the union. It has an (abstract) generic base type Object with several (specific) sub-types, such as String, Number, List etc. Every type-struct has the following unnamed struct as the first member:
#define OBJHEAD struct{ \
int id; \
int line; \
int column; \
}
The id identifies the type of object, line and column should (also) be self-explanatory. A simplified implementation of various objects:
typedef struct Object{
OBJHEAD;
char data[]; // necessary?
} Object;
typedef struct Number{
OBJHEAD;
int value; // only int for simplicity
} Number;
typedef struct String{
OBJHEAD;
size_t length;
char * string;
} String;
typedef struct List{
OBJHEAD;
size_t size;
Object * elements; // may be any kind and mix of objects
} List;
Object * Number_toObject(Number * num){
return (Object*)num;
}
Number * Number_fromObject(Object * obj){
if(obj->type != TYPE_NUMBER){
return NULL;
}
return (Number*)obj;
}
I know that the most elegant and technically correct way to do this would be to use an enum for the id and a union for the various sub-types. But I want the type-system to be extensible (through some form of type-registry) so that types can be added later without changing all the Object-related code.
A later/external addition could be:
typedef struct File{
OBJHEAD;
FILE * fp;
} File;
without needing to change Object.
Are these conversions guaranteed to be safe?
(As for the small macro-abuse: the OBJHEAD will of course be extensively documented so additional implementers will know what member-names not to use. The idea is not to hide the header, but to save pasting it every time.)
Converting a pointer to one object type to a pointer to a different object type (via a cast, for instance) is permitted, but if the resulting pointer is not correctly aligned then behavior is undefined (C11 6.3.2.3/7). Depending on the members of Base and Sub and on implentation-dependent behavior, it is not necessarily the case that a Base * converted to a Sub * is correctly aligned. For example, given ...
struct Base{
struct{
int id;
};
char data[]; // necessary?
}
struct Sub{
struct{
int id;
};
long long int value;
};
... it may be that the implementation permits Base objects to be aligned on 32-bit boundaries but requires Sub objects to be aligned on 64-bit boundaries, or even on stricter ones.
None of this is affected by whether Base has a flexible array member.
It is a different question whether it is safe to dereference a pointer value of one type that was obtained by casting a pointer value of a different type. For one thing, C places rather few restrictions on how implementations choose to lay out structures: members must be laid out in the order they are declared, and there must not be any padding before the first one, but otherwise, implementations have free reign. To the best of my knowledge, in your case there is no requirement that the anonymous struct members of your two structures must be laid out the same way as each other if they have more than one member. (And if they have only one member then why use an anonumous struct?) It is also not safe to assume that Base.data starts at the same offset as the first element following the anonymous struct in Sub.
In practice, dereferencing the result of your subToBase() is probably ok, and you can certainly implement tests to verify that. Also, if you have a Base * that was obtained by conversion from a Sub *, then the result of converting it back, for instance via baseToSub(), is guaranteed to be the same as the original Sub * (C11 6.3.2.3/7 again). In that case, the conversion to Base * and back has no effect on the safety of dereferencing the the pointer as a Sub *.
On the other hand, though I'm having trouble finding a reference for it in the standard, I have to say that baseToSub() is very dangerous in the general context. If a Base * that does not actually point to a Sub is converted to Sub * (which in itself is permitted), then it is not safe to dereference that pointer to access members not shared with Base. In particular, given my declarations above, if the referenced object is in fact a Base, then Base.data being declared in no way prevents ((Sub *)really_a_Base_ptr)->value from producing undefined behavior.
To avoid all undefined and implementation-defined behavior, you want an approach that avoids casting and ensures consistent layout. #LoPiTaL's suggestion to embed a typed Base structure inside your Sub structures is a good approach in that regard.
No it is not safe, at least not under all circumstances. If your compiler sees two pointers p and q that have different base type, it may always assume that they don't alias, or stated in other words it may always assume that *p and *q are different objects.
Your cast punches a hole in that assumption. That is if you have a function
double foo(struct A* p, struct B* q) {
double b = q->field0;
*p = (struct A*){ 0 };
return b + q->field0; // compiler may return 2*b
}
the optimizer is allowed to avoid the additional read from memory.
If you'd know that no function will ever see the same object through differently typed pointers, you would be safe. But such an assertion is not made easily, so you'd better avoid such hackery.
It is correct, since it is guaranteed to have the same alignment on the first member of the struct, so you can cast from one struct to another.
Nevertheless, the common way to implement your behaviour is to "inherit" the base class:
//Base struct definition
typedef struct Base_{
int id;
// ...
//char data[]; //This is not needed.
}Base;
//Subclass definition
typedef struct Sub_{
Base base; //Note: this is NOT a pointer
// actual data
}Sub;
So now, you can cast a Sub struct into a Base struct, or just return the first member, which already is of type Base, so there is no need of casting anymore.
One word of caution: do not abuse MACROS. MACROS are nice and good for a lot of things, but abusing them may lead to difficult to read and maintain code.
In this case, the macro is easily replaced with the base member.
One final word, your macro is error prone, since the member names are now hidden. On the end, you may be adding new members with the same name, and getting weird errors without knowing why.
When you further expand your hierarchy into sub-subclasses, you will end up having to write ALL the base classes MACRO, while if you use the "inherit" aproach, you will have to write only the direct base.
Neither of these solutions actually solve your problem: inheritance. The only real solution you would have (the preferred) would be to change to a trully OO language. Due to similarity to C, the best match would be C++, but could do any other language.

Struct padding for single member structs

I was looking to make a struct which was an arbitrary size, known at compile time. (for use in a macro).
eg:
/* assume sizeof(SomeStruct) could be an odd number,
* if it is using GCC's 'packed' attribute for eg */
struct {
unsigned char data[sizeof(SomeStruct)];
} a, *tmp;
tmp = (void *)some_data
a = *tmp;
However I was concerned that struct padding may increase the size of the struct so it is larger than the member, I was assured that the size of a single member struct will always be the size of its member.
So my question is:
Can I rely on single member structs always being the same size as their member? Is this apart of the C spec? or is it just how most compilers behave?
C11 6.7.2.1 paragraph 17:
There may be unnamed padding at the end of a structure or union.
No special case is listed for structs with only one member. This is implementation-defined.
It's not obvious why a compiler would put padding there though, since the size of the member should already be as padded as is necessary for all purposes, as it has a complete type of its own.
For your usage though, you don't need to rely on the compiler's behaviour - you can enforce it yourself by adding a static assertion to your macro that sizeof *tmp == sizeof (SomeStruct). If you have an up-to-date compiler and your macro allows for declaration statements, just use _Static_assert; there are also several C99-compatible hacks you can use, such as:
#define STATIC_EXPR_ASSERT(COND) (sizeof (char[(COND) ? 1 : -1]))
... which you can use as the lhs of a comma-expression.
You should also note that if the body of the temporary struct is made up of a char array, it may not have sufficient alignment to represent the incoming data correctly. From C11 onwards you should specify the alignment of the array with _Alignas:
struct {
_Alignas (max_align_t) unsigned char data[sizeof(SomeStruct)];
} a, *tmp;
...which will ensure it can safely store the data for any type regardless of alignment. In C99 you don't have any way to explicitly request max alignment, but you can force the alignment to match a named type by making the temporary struct a member of a union alongside it (you might try long double and hope that's the most-aligned type).

Representing dynamic typing in C

I'm writing a dynamically-typed language. Currently, my objects are represented in this way:
struct Class { struct Class* class; struct Object* (*get)(struct Object*,struct Object*); };
struct Integer { struct Class* class; int value; };
struct Object { struct Class* class; };
struct String { struct Class* class; size_t length; char* characters; };
The goal is that I should be able to pass everything around as a struct Object* and then discover the type of the object by comparing the class attribute. For example, to cast an integer for use I would simply do the following (assume that integer is of type struct Class*):
struct Object* foo = bar();
// increment foo
if(foo->class == integer)
((struct Integer*)foo)->value++;
else
handleTypeError();
The problem is that, as far as I know, the C standard makes no promises about how structures are stored. On my platform this works. But on another platform struct String might store value before class and when I accessed foo->class in the above I would actually be accessing foo->value, which is obviously bad. Portability is a big goal here.
There are alternatives to this approach:
struct Object
{
struct Class* class;
union Value
{
struct Class c;
int i;
struct String s;
} value;
};
The problem here is that the union uses up as much space as the size of the largest thing that can be stored in the union. Given that some of my types are many times as large as my other types, this would mean that my small types (int) would take up as much space as my large types (map) which is an unacceptable tradeoff.
struct Object
{
struct Class* class;
void* value;
};
This creates a level of redirection that will slow things down. Speed is a goal here.
The final alternative is to pass around void*s and manage the internals of the structure myself. For example, to implement the type test mentioned above:
void* foo = bar();
// increment foo
if(*((struct Class*) foo) == integer)
(*((int*)(foo + sizeof(struct Class*))))++;
else
handleTypeError();
This gives me everything I want (portability, different sizes for different types, etc.) but has at least two downsides:
Hideous, error-prone C. The code above only calculates a single-member offset; it will get much worse with types more complex than integers. I might be able to alleviate this a bit using macros, but this will be painful no matter what.
Since there is no struct that represents the object, I don't have the option of stack allocations (at least without implementing my own stack on the heap).
Basically, my question is, how can I get what I want without paying for it? Is there a way to be portable, have variance in size for different types, not use redirection, and keep my code pretty?
EDIT: This is the best response I've ever received for an SO question. Choosing an answer was hard. SO only allows me to choose one answer so I chose the one that lead me to my solution, but you all received upvotes.
See Python PEP 3123 (http://www.python.org/dev/peps/pep-3123/) for how Python solves this problem using standard C. The Python solution can be directly applied to your problem. Essentially you want to do this:
struct Object { struct Class* class; };
struct Integer { struct Object object; int value; };
struct String { struct Object object; size_t length; char* characters; };
You can safely cast Integer* to Object*, and Object* to Integer* if you know that your object is an integer.
C gives you sufficient guarantees that your first approach will work. The only modification you need to make is that in order to make the pointer aliasing OK, you must have a union in scope that contains all of the structs that you are casting between:
union allow_aliasing {
struct Class class;
struct Object object;
struct Integer integer;
struct String string;
};
(You don't need to ever use the union for anything - it just has to be in scope)
I believe the relevant part of the standard is this:
[#5] With one exception, if the value
of a member of a union object is used
when the most recent store to the
object was to a different member, the
behavior is implementation-defined.
One special guarantee is made in order
to simplify the use of unions: If a
union contains several structures that
share a common initial sequence (see
below), and if the union object
currently contains one of these
structures, it is permitted to inspect
the common initial part of any of them
anywhere that a declaration of the
completed type of the union is
visible. Two structures share a common
initial sequence if corresponding
members have compatible types (and,
for bit-fields, the same widths) for a
sequence of one or more initial
members.
(This doesn't directly say it's OK, but I believe that it does guarantee that if two structs have a common intial sequence and are put into a union together, they'll be laid out in memory the same way - it's certainly been idiomatic C for a long time to assume this, anyway).
There are 3 major approaches for implementing dynamic types and which one is best depends on the situation.
1) C-style inheritance: The first one is shown in Josh Haberman's answer. We create a type-hierarchy using classic C-style inheritance:
struct Object { struct Class* class; };
struct Integer { struct Object object; int value; };
struct String { struct Object object; size_t length; char* characters; };
Functions with dynamically typed arguments receive them as Object*, inspect the class member, and cast as appropriate. The cost to check the type is two pointer hops. The cost to get the underlying value is one pointer hop. In approaches like this one, objects are typically allocated on the heap since the size of objects is unknown at compile time. Since most `malloc implementations allocate a minimum of 32 bytes at a time, small objects can waste a significant amount of memory with this approach.
2) Tagged union: We can remove a level of indirection for accessing small objects using the "short string optimization"/"small object optimization":
struct Object {
struct Class* class;
union {
// fundamental C types or other small types of interest
bool as_bool;
int as_int;
// [...]
// object pointer for large types (or actual pointer values)
void* as_ptr;
};
};
Functions with dynamically typed arguments receive them as Object, inspect the class member, and read the union as appropriate. The cost to check the type is one pointer hop. If the type is one of the special small types, it is stored directly in the union, and there is no indirection to retrieve the value. Otherwise, one pointer hop is required to retrieve the value. This approach can sometimes avoid allocating objects on the heap. Although the exact size of an object still isn't known at compile time, we now know the size and alignment (our union) needed to accommodate small objects.
In these first two solutions, if we know all the possible types at compile time, we can encode the type using an integer type instead of a pointer and reduce type check indirection by one pointer hop.
3) Nan-boxing: Finally, there's nan-boxing where every object handle is only 64 bits.
double object;
Any value corresponding to a non-NaN double is understood to simply be a double. All other object handles are a NaN. There are actually large swaths of bit values of double precision floats that correspond to NaN in the commonly used IEEE-754 floating point standard. In the space of NaNs, we use a few bits to tag types and the remaining bits for data. By taking advantage of the fact that most 64-bit machines actually only have a 48-bit address space, we can even stash pointers in NaNs. This method incurs no indirection or extra memory use but constrains our small object types, is awkward, and in theory is not portable C.
Section 6.2.5 of ISO 9899:1999 (the C99 standard) says:
A structure type describes a sequentially allocated nonempty set of member objects (and, in certain circumstances, an incomplete array), each of which has an optionally specified name and possibly distinct type.
Section 6.7.2.1 also says:
As discussed in 6.2.5, a structure is a type consisting of a sequence of members, whose storage is allocated in an ordered sequence, and a union is a type consisting of a sequence of members whose storage overlap.
[...]
Within a structure object, the non-bit-field members and the units in which bit-fields
reside have addresses that increase in the order in which they are declared. A pointer to a
structure object, suitably converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.
This guarantees what you need.
In the question you say:
The problem is that, as far as I know, the C standard makes no promises about how structures are stored. On my platform this works.
This will work on all platforms. It also means that your first alternative - what you are currently using - is safe enough.
But on another platform struct StringInteger might store value before class and when I accessed foo->class in the above I would actually be accessing foo->value, which is obviously bad. Portability is a big goal here.
No compliant compiler is allowed to do that. [I replaced String by Integer assuming you were referring to the first set of declarations. On closer examination, you might have been referring to the structure with an embedded union. The compiler still isn't allowed to reorder class and value.]
The problem is that, as far as I know, the C standard makes no promises about how structures are stored. On my platform this works. But on another platform struct String might store value before class and when I accessed foo->class in the above I would actually be accessing foo->value, which is obviously bad. Portability is a big goal here.
I believe you're wrong here. First, because your struct String doesn't have a value member. Second, because I believe C does guarantee the layout in memory of your struct's members. That's why the following are different sizes:
struct {
short a;
char b;
char c;
}
struct {
char a;
short b;
char c;
}
If C made no guarantees, then compilers would probably optimize both of those to be the same size. But it guarantees the internal layout of your structs, so the natural alignment rules kick in and make the second one larger than the first.
I appreciate the pedantic issues raised by this question and answers, but I just wanted to mention that CPython has used similar tricks "more or less forever" and it's been working for decades across a huge variety of C compilers. Specifically, see object.h, macros like PyObject_HEAD, structs like PyObject: all kinds of Python Objects (down at the C API level) are getting pointers to them forever cast back and forth to/from PyObject* with no harm done. It's been a while since I last played sea lawyer with an ISO C Standard, to the point that I don't have a copy handy (!), but I do believe that there are some constraints there that should make this keep working as it has for nearly 20 years...

Resources