Consider the following code:
int main()
{
typedef struct { int first; float second; } type;
type whole = { 1, 2.0 };
void * vp = &whole;
struct { int first; } * shorn = vp;
printf("values: %d, %d\n", ((type *)vp)->first, shorn->first);
if (vp == shorn)
printf("ptrs compare the same\n");
return 0;
}
Two questions:
Is the pointer equality comparison UB?
Regarding the "shearing" away of the second member on the line that initializes shorn: is it valid C to cast away struct members like this and then dereference the manipulated pointer to access the remaining member?
Comparing two pointers with == when one is a void * is well defined.
Section 6.5.9 of the C standard regarding the equality operator == says the following:
2 One of the following shall hold:
both operands have arithmetic type;
both operands are pointers to qualified or unqualified versions of compatible types;
one operand is a pointer to an object type and the other is a pointer to a qualified or unqualified version of void; or
one operand is a pointer and the other is a null pointer constant
...
5 Otherwise, at least one operand is a pointer. If one operand is a pointer and the other is a null pointer constant, the null pointer
constant is converted to the type of the pointer. If one operand
is a pointer to an object type and the other is a pointer
to a qualified or unqualified version of void, the former is
converted to the type of the latter.
The usage of shorn->first works because a pointer to a struct can be converted to a pointer to its first member. For both type and the unnamed struct type their first member is an int so it works out.
Section 6.2.5 Types paragraph 28 of the C standard says:
[...] All pointers to structure types shall have the same representation and alignment requirements as each other. [...]
Section 6.3.2.3 Pointers paragraph 1 says:
A pointer to void may be converted to or from a pointer to any object type. A pointer to any object type may be converted to a pointer to void and back again; the result shall compare equal to the original pointer.
And paragraph 7 says:
A pointer to an object type may be converted to a pointer to a different object type. If the resulting pointer is not correctly aligned68) for the referenced type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer. [...]
And footnote 68 says:
In general, the concept "correctly aligned" is transitive: if a pointer to type A is correctly aligned for a pointer to type B, which in turn is correctly aligned for a pointer to type C, then a pointer to type A is correctly aligned for a pointer to type C.
Because all pointers to structure types have the same representation, the conversions between pointers to void and pointers to structure types must be the same for all pointers to structure types. So it seems that a pointer to structure type A could be converted by a cast operator directly to a pointer to structure type B without an intermediate conversion to a pointer to void as long as the pointer is "correctly aligned" for structure type B. (This may be a weak argument.)
The question remains when, in the case of two structure types A and B where the initial sequence of structure type A consists of all the members of structure type B, a pointer to structure type A is guaranteed to be correctly aligned for structure type B (the reverse is obviously not guaranteed). As far as I can tell, the C standard makes no such guarantee. So strictly speaking, a pointer to the larger structure type A might not be correctly aligned for the smaller structure type B, and if it is not, the behavior is undefined. For a "sane" compiler, the larger structure type A would not have weaker alignment than the smaller structure type B, but for an "insane" compiler, that might not be the case.
Regarding the second question about accessing members of the truncated (shorter) structure using the pointer derived from the full (longer) structure, then as long as the pointer is correctly aligned for the shorter structure (see above for why that might not be true for an "insane" compiler), and as long as strict aliasing rules are avoided (for example, by going through an intermediate pointer to void in an intermediate external function call across compilation unit boundaries), then accessing the members through the pointer to the shorter structure type should be perfectly fine. There is a special guarantee for that when objects of both structure types appear as members of the same union type. Section 6.3.2.3 Structure and union members paragraph 6 says:
One special guarantee is made in order to simplify the use of unions: if a union contains several structures that share a common initial sequence (see below), and if the union
object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union
is visible. Two structures share a common initial sequence if corresponding members have compatible types (and, for bit-fields, the same widths) for a sequence of one or more initial members.
However, since the offsets of members within a structure type does not depend on whether an object of the structure type appears in a union type or not, the above implies that any structures with a common initial sequence of members will have those common members at the same offsets within their respective structure types.
In the language the C89 Standard was written to describe, it was well established that if two structures share a common initial sequence, a pointer to either may be cast to the other and used to inspect members of that common initial sequence. Code which relied upon this was commonplace and not considered even remotely controversial.
In the interest of optimization, the authors of the C99 Standard deliberately allowed compilers to assume that structures of different types won't alias in cases where such assumption would be useful for their customers. Because there are many good means by which implementations could recognize cases where such assumptions would be needlessly break what had been perfectly good code, and because they the authors of the Standard expected that compiler writers would make a bona fide effort to behave in ways useful to the programmers using their products, the Standard doesn't mandate any particular means of making such distinctions. Instead, it regards the ability to support constructs that had been universally supported as a "quality of implementation" issue, which would be reasonable if compiler writers made a bona fide effort to treat it as such.
Unfortunately, some compiler writers who aren't interested in selling their products to paying customers have taken the Standard's failure to mandate useful behavior as an invitation to behave in needlessly useless fashion. Code which relies upon Common Initial Sequence guarantees can thus not be meaningfully processed by clang or gcc without using either non-standard syntax or disabling type-based aliasing entirely.
Related
N2479 C17..C2x working draft — February 5, 2020 ISO/IEC 9899:202x (E) (emphasis added):
6.7.2.1 Structure and union specifiers
17 Within a structure object, the non-bit-field members and the units in which bit-fields reside have addresses that increase in the order in which they are declared. A pointer to a structure object, suitably converted, points to its initial member (or if that member is a bit-field, then to the unit in which it resides), and vice versa. There may be unnamed padding within a structure object, but not at its beginning.
18 The size of a union is sufficient to contain the largest of its members. The value of at most one of the members can be stored in a union object at any time. A pointer to a union object, suitably converted, points to each of its members (or if a member is a bit-field, then to the unit in which it resides), and vice versa.
Question: what is the exact definition of suitably converted?
Extra: if there is no exact definition of suitably converted, then shall the C implementation document its understanding? For example (C/C++ preprocessor domain), Microsoft understands the term single item (C++, N4713) as single, permanently indivisible preprocessor token (which leads to issues while porting code from gcc/clang/other, which has different understanding), however, it seems that they don't document their understanding of the single item.
In this context "suitable converted" means converted to a proper compatible type. For example:
#include <stdio.h>
struct mystruct {
double a;
int b;
};
int main()
{
struct mystruct s = { 2.5, 4 };
double *d = (double *)&s;
printf("%f\n", *d); // prints 2.500000
return 0;
}
Here the first member of struct mystruct has type double. So a "suitable conversion" in this case means a struct mystruct * may be converted via explicit cast to double * and it will point to the a member.
This isn't a term with special meaning.
You are dealing with conversion from type A to type B. Type A is a pointer-to-structure type. Type B is not specified except that it must be suitable for use as a pointer to the initial member. Therefore, it must be a pointer (so we are dealing only with conversions between two pointer types) and it must follow the strict aliasing rule (the destination type can be pointer to a narrow character type, std::byte, the actual type of the first member, or a type that is representation compatible such as differing only in signedness).
Any pointer conversion that results in a suitable pointer satisfied "suitably converted".
Question: what is the exact definition of suitably converted?
The C standard does not give any “exact definition” of “suitably converted.”
I interpret it to mean any sequence of conversions to a type of “pointer to type of initial member” or “pointer to type of structure” such that the specifications of the conversions ensure the final pointer is pointing to the appropriate address. (E.g., has not passed through a conversion with possibly incorrect alignment.)
Extra: if there is no exact definition of suitably converted, then shall the C implementation document its understanding?
The C standard does not impose any requirement on a C implementation to document its understanding or interpretation of “suitably converted.”
It's not a formal term, but between the lines we can tell that it is used to mean a valid pointer conversion. And it needs to be carried out explicitly by the programmer by means of a cast.
In C, pretty much any object pointer can be converted to another object pointer casting. But what happens if you would de-reference such a pointer through the wrong type is a whole different story though. Most of the time it's poorly-defined behavior.
Valid pointer conversions in this case:
A pointer to a type compatible with the type of the first member.1) 2) 3)
A void pointer. 1) 2)
A pointer to character type.2)
A pointer to another structure type, where both structures are part of a union and share common initial member(s) of compatible type.4)
Optionally the pointer in any of the above cases can be type qualified. Unless the first member is a qualified type, in which case the pointer needs to share all qualifiers of that type. 1) 3)
1) The rules of simple assignment 6.5.16.1.
2) The rules regarding pointer conversions (6.3.2.3).
3) The rules of compatible type qualifiers (6.7.3).
4) The rule of common initial sequence (6.5.2.3).
The common initial sequence one is an oddball rule which apparently got poor compiler support, but it's in line with "strict aliasing".
Considering
T1 *p1;
T2 *p2;
Can we assign p1 to p2 or vice versa? If so, can it be done without the casting or we must use a cast?
First, let’s consider assignment without casting. C 2018 6.5.16.1 1 lists constraints for simple assignment, saying that one of them must hold. The first two are for arithmetic, structure, and union types. The last two deal involve null pointer constants or _Bool. The middle two deal with assigning pointers to pointers:
the left operand has atomic, qualified, or unqualified pointer type, and … both operands are pointers to qualified or unqualified versions of compatible types, and the type pointed to by the left has all the qualifiers of the type pointed to by the right
the left operand has atomic, qualified, or unqualified pointer type, and … one operand is a pointer to an object type, and the other is a pointer to a qualified or unqualified version of void, and the type pointed to by the left has all the qualifiers of the type pointed to by the right
The latter says we can assign void * to any object pointer and vice-versa, as long as no qualifiers (const, volatile, restrict, or __Atomic) are removed.
The former says we can assign pointers to compatible types, as long as no qualifiers are removed. What are compatible types?
6.2.7 1 says:
Two types are compatible if they are the same.
Additional rules are in 6.7.2, 6.7.3, and 6.7.6.
Two structure, union, or enumerated types declared in separate translation units are compatible if they are, in essence, declared identically. (Interestingly, two such types declared in the same translation unit are not compatible.)
6.7.2 4 says each enumerated type (an enum type) is compatible with an implementation-defined choice of char or a signed or unsigned integer type. So a pointer to some enum can be assigned to a pointer to one char or integer type (and vice-versa), but you cannot know which one without knowing something about the particular C implementation.
6.7.3 11 says qualified types must have the same qualifiers to be compatible. Thus an int is not compatible with a const int, and this prevents an int * from being assigned to a const int *.
6.7.6.1 2 says for two pointer types to be compatible, they must be identically qualified pointers to compatible types. This tells us, for example, that int * is not compatible with char *, and, therefore, by assignment constraints above, a char ** may not be assigned to an int **.
6.7.6.2 6 says for two array types to be compatible, they must have compatible element types and, if they both have integer constant sizes, they must be the same. (This allows that an array with unknown size may be compatible with an array of known size. However, additional text says that if the arrays ultimately have different sizes, using them in a context that requires them to be compatible has undefined behavior. So an assignment of pointers to such arrays may satisfy its constraints and compile without error, but the resulting program may misbehave.)
6.7.6.3 15 presents somewhat complicated rules for the compatibility of function types. These rules are complicated because functions may be declared with or without parameter lists, with ellipses, and so on. I will omit complete discussion of these.
Those are the rules that tell you what pointer assignments may be made without casts.
6.5.4 discusses casts. Its constraints do not restrict which pointer types may be converted to which other pointer types. (They do prohibit other things involving pointers, such as converting a pointer type to a floating-point type.) So you can specify any pointer conversion you want in a cast, and, as long as the resulting type is compatible with the type to which it is being assigned, no assignment or cast constraint is violated. However, there is a still a question about whether the conversion is proper.
6.3.2.3 specifies rules for pointer conversions. Those dealing with conversion from pointers to pointers (excluding integers and null pointer constants) say:
Any pointer to an object type (not to function types) may be converted to a pointer to void and vice-versa. The result of converting an object pointer to a void pointer and back compares equal to the original.
A pointer may be converted to the same type with more qualifiers, and the result compares equal to the original.
A pointer to an object type may be converted to a pointer to a different object type if the resulting pointer is correctly aligned for its type (otherwise, the behavior is undefined). When converted back, the result compares equal to the original pointer. (Note that, while you are allowed to make this conversion, this rule does not say the resulting pointer may be used to access an object of the new type. There are other rules in C about that.)
A pointer to a function type may be converted to a pointer to another function type. When converted back, the result compares equal to the original pointer. (As with objects, you are allowed to make this conversion, but using the resulting pointer to call an incompatible function has undefined behavior.)
So, when casts are used, you may convert any object pointer type to any object pointer type and assign it as long as the alignment requirement is satisfied, and you may convert any function pointer type to any function pointer type and assign it.
When types T1 and T2 are different, assignments between T1 *p1 and T2 *p2 are formally disallowed, unless at least one of T1 and T2 is void and the other is an object (not function) type.
In many cases incompatible assignments will work in practice, especially on machines (such as all popular ones today) with "flat" address spaces and where all pointer types share the same internal representation.
After a "mixed-mode" pointer assignment, however, problems may very well occur when the pointers are dereferenced, due to (1) alignment issues and (2) strict aliasing.
Since "mixed mode" pointer assignments are formally illegal and often a bad idea, most compilers will warn about them. Most compilers allow the warnings to be suppressed with an explicit cast. Most of the time, the cast serves only to suppress the warning; it does not introduce any actual conversion that wouldn't have been performed anyway. (That is, changing p1 = p2 to p1 = (T1 *)p2 is a lot like changing i = f to i = (int)f, where i is an int and f is a float.)
Addendum: I wrote, "When types T1 and T2 are different," but a more precise statement would be when they are incompatible. For example, types char and unsigned char are compatible, so assignment between pointers of those types is fine. See Eric Postpischil's longer answer for more details.
AFAIK, there are three situations where aliasing is ok
Types that only differ by qualifier or sign can alias each other.
struct or union types can alias types contained inside them.
casting T* to char* is ok. (the opposite is not allowed)
These makes sense when reading simple examples from John Regehrs blog posts but I'm not sure how to reason about aliasing-correctness for larger examples, such as malloc-like memory arrangements.
I'm reading Per Vognsens re-implementation of Sean Barrets stretchy buffers. It uses a malloc-like schema where a buffer has associated metadata just before it.
typedef struct BufHdr {
size_t len;
size_t cap;
char buf[];
} BufHdr;
The metadata is accessed by subtracting an offset from a pointer b:
#define buf__hdr(b) ((BufHdr *)((char *)(b) - offsetof(BufHdr, buf)))
Here's a somewhat simplified version of the original buf__grow function that extends the buffer and returns the buf as a void*.
void *buf__grow(const void *buf, size_t new_size) {
// ...
BufHdr *new_hdr; // (1)
if (buf) {
new_hdr = xrealloc(buf__hdr(buf), new_size);
} else {
new_hdr = xmalloc(new_size);
new_hdr->len = 0;
}
new_hdr->cap = new_cap;
return new_hdr->buf;
}
Usage example (the buf__grow is hidden behind macros but here it's in the open for clarity):
int *ip = NULL;
ip = buf__grow(ip, 16);
ip = buf__grow(ip, 32);
After these calls, we have 32 + sizeof(BufHdr) bytes large memory area on the heap. We have ip pointing into that area and we have new_hdr and buf__hdr pointing into it at various points in the execution.
Questions
Is there a strict-aliasing violation here? AFAICT, ip and some variable of type BufHdr shouldn't be allowed to point to the same memory.
Or is it so that the fact that buf__hdr not creating an lvalue means it's not aliasing the same memory as ip? And the fact that new_hdr is contained within buf__grow where ip isn't "live" means that those aren't aliasing either?
If new_hdr were in global scope, would that change things?
Do the C compiler track the type of storage or only the types of variables? If there is storage, such as the memory area allocated in buf__grow that doesn't have any variable pointing to it, then what is the type of that storage? Are we free to reinterpret that storage as long as there is no variable associated with that memory?
Is there a strict-aliasing violation here? AFAICT, ip and some variable of type BufHdr shouldn't be allowed to point to the same memory.
What's important to remember is that a strict aliasing violation only occurs when you do a value access of a memory location, and the compiler believes that what's stored at that memory location is of a different type. So it is not so important to speak of the types of the pointers, as to speak of the effective type of whatever they point at.
An allocated chunk of memory has no declared type. What applies is C11 6.5/6:
The effective type of an object for an access to its stored value is the declared type of the
object, if any. 87)
Where note 87 clarifies that allocated objects have no declared type. That is the case here, so we continue to read the definition of effective type:
If a value is stored into an object having no declared type through an
lvalue having a type that is not a character type, then the type of the lvalue becomes the
effective type of the object for that access and for subsequent accesses that do not modify
the stored value.
This means that as soon as we do an access to the chunk of allocated memory, the effective type of whatever is stored there, becomes the type of whatever we stored there.
The first time access happens in your case, is the lines new_hdr->len = 0; and new_hdr->cap = new_cap;, making the effective type of the data at those addresses size_t.
buf remains inaccessed, so that part of the memory does not yet have an effective type. You return new_hdr->buf and set an int* to point there.
The next thing that will happen, I assume is buf__hdr(ip). In that macro, the pointer is cast to (char *), then some pointer subtraction occurs:
(b) - offsetof(BufHdr, buf) // undefined behavior
Here we formally get undefined behavior, but for entirely different reasons than strict aliasing. b is not a pointer pointing to the same array as whatever is stored before b. The relevant part is the specification of the additive operators 6.5.6:
For subtraction, one of the following shall hold:
— both operands have arithmetic type;
— both operands are pointers to qualified or unqualified versions of compatible complete object types; or
— the left operand is a pointer to a complete object type and the right operand has integer type.
The first two clearly don't apply. In the third case, we don't point to a complete object type, as buf has not yet gotten an effective type. As I understand it, this means we have a constraint violation, I'm not entirely sure here. I am however very sure that the following is violated, 6.5.6/9:
When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object; the result is the difference of the subscripts of the two array elements. The size of the result is implementation-defined,
and its type (a signed integer type) is ptrdiff_t defined in the <stddef.h> header.
If the result is not representable in an object of that type, the behavior is undefined
So that's definitely a bug.
If we ignore that part, the actual access (BufHdr *) is fine, since BufHdr is a struct ("aggregate") containing the effective type of the object accessed (2x size_t). And here the memory of buf is accessed for the first time, getting the effective type char[] (flexible array member).
There is no strict aliasing violation unless you would after invoking the above macro go and access ip as an int.
If new_hdr were in global scope, would that change things?
No, the pointer type does not matter, only the effective type of the pointed-at object.
Do the C compiler track the type of storage or only the types of variables?
It needs to track the effective type of the object if it wishes to do optimizations like gcc, assuming strict aliasing violations never occur.
Are we free to reinterpret that storage as long as there is no variable associated with that memory?
Yes you can point at it with any kind of pointer - since it is allocated memory, it doesn't get an effective type until you do a value access.
The Standard does not define any means by which an lvalue of one type can be used to derive an lvalue of a second type that can be used to access the storage, unless the latter has a character type. Even something as basic as:
union foo { int x; float y;} u = {0};
u.x = 1;
invokes UB because it uses an lvalue of type int to access the storage associated with an object of type union foo and float. On the other hand, the authors of the Standard probably figured that since no compiler writer would be so obtuse as to use the lvalue-type rules as justification for not processing the above in useful fashion, there was no need to try to craft explicit rules mandating that they do so.
If a compiler guarantees not to "enforce" the rule except in cases where:
an object is modified during a particular execution of a function or loop;
lvalues of two or more different types are used to access storage during such execution; and
neither lvalue has been visibly and actively derived from the other within such execution
such a guarantee would be sufficient to allow a malloc() implementation that would be free of "aliasing"-related problems. While I suspect the authors of the Standard probably expected compiler writers to naturally uphold such a guarantee whether or not it was mandated, neither gcc nor clang will do so unless the -fno-strict-aliasing flag is used.
Unfortunately, when asked in Defect Report #028 to clarify what the C89 rules meant, the Committee responded by suggesting that an lvalue formed by dereferencing a pointer to a unions member will mostly behave like an lvalue formed directly with the member-access operator, except that actions which would invoke Implementation-Defined Behavior if done directly on a union member should invoke UB if done on a pointer. When writing C99, the Committee decided to "clarify" things by codifying that principle into C99's "Effective Type" rules, rather than recognizing any cases where an lvalue of a derived type may be used to access the parent object [an omission which the Effective Type rules do nothing to correct!].
The C99 standard states that:
When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object
Consider the following code:
struct test {
int x[5];
char something;
short y[5];
};
...
struct test s = { ... };
char *p = (char *) s.x;
char *q = (char *) s.y;
printf("%td\n", q - p);
This obviously breaks the above rule, since the p and q pointers are pointing to different "array objects", and, according to the rule, the q - p difference is undefined.
But in practice, why should such a thing ever result in undefined behaviour? After all, the struct members are laid out sequentially (just as array elements are), with any potential padding between the members. True, the amount of padding will vary across implementations and that would affect the outcome of the calculations, but why should that result be "undefined"?
My question is, can we suppose that the standard is just "ignorant" of this issue, or is there a good reason for not broadening this rule? Couldn't the above rule be rephrased to "both shall point to elements of the same array object or members of the same struct"?
My only suspicion are segmented memory architectures where the members might end up in different segments. Is that the case?
I also suspect that this is the reason why GCC defines its own __builtin_offsetof, in order to have a "standards compliant" definition of the offsetof macro.
EDIT:
As already pointed out, arithmetic on void pointers is not allowed by the standard. It is a GNU extension that fires a warning only when GCC is passed -std=c99 -pedantic. I'm replacing the void * pointers with char * pointers.
Subtraction and relational operators (on type char*) between addresses of member of the same struct are well defined.
Any object can be treated as an array of unsigned char.
Quoting N1570 6.2.6.1 paragraph 4:
Values stored in non-bit-field objects of any other object type
consist of n × CHAR_BIT bits, where n is the size of an object of that
type, in bytes. The value may be copied into an object of type
unsigned char [ n ] (e.g., by memcpy); the resulting set of bytes is
called the object representation of the value.
...
My only suspicion are segmented memory architectures where the members
might end up in different segments. Is that the case?
No. For a system with a segmented memory architecture, normally the compiler will impose a restriction that each object must fit into a single segment. Or it can permit objects that occupy multiple segments, but it still has to ensure that pointer arithmetic and comparisons work correctly.
Pointer arithmetic requires that the two pointers being added or subtracted to be part of the same object because it doesn't make sense otherwise.
The quoted section of standard specifically refers to two unrelated objects such as int a[b]; and int b[5]. The pointer arithmetic requires to know the type of the object that the pointers pointing to (I am sure you are aware of this already).
i.e.
int a[5];
int *p = &a[1]+1;
Here p is calculated by knowing the that the &a[1] refers to an int object and hence incremented to 4 bytes (assuming sizeof(int) is 4).
Coming to the struct example, I don't think it can possibly be defined in a way to make pointer arithmetic between struct members legal.
Let's take the example,
struct test {
int x[5];
char something;
short y[5];
};
Pointer arithmatic is not allowed with void pointers by C standard (Compiling with gcc -Wall -pedantic test.c would catch that). I think you are using gcc which assumes void* is similar to char* and allows it.
So,
printf("%zu\n", q - p);
is equivalent to
printf("%zu", (char*)q - (char*)p);
as pointer arithmetic is well defined if the pointers point to within the same object and are character pointers (char* or unsigned char*).
Using correct types, it would be:
struct test s = { ... };
int *p = s.x;
short *q = s.y;
printf("%td\n", q - p);
Now, how can q-p be performed? based on sizeof(int) or sizeof(short) ? How can the size of char something; that's in the middle of these two arrays be calculated?
That should explain it's not possible to perform pointer arithmetic on objects of different types.
Even if all members are of same type (thus no type issue as stated above), then it's better to use the standard macro offsetof (from <stddef.h>) to get the difference between struct members which has the similar effect as pointer arithmetic between members:
printf("%zu\n", offsetof(struct test, y) - offsetof(struct test, x));
So I see no necessity to define pointer arithmetic between struct members by the C standard.
Yes, you are allowed to perform pointer arithmetric on structure bytes:
N1570 - 6.3.2.3 Pointers p7:
... When a pointer to an object is converted to a pointer to a character type,
the result points to the lowest addressed byte of the object. Successive increments of the
result, up to the size of the object, yield pointers to the remaining bytes of the object.
This means that for the programmer, bytes of the stucture shall be seen as a continuous area, regardless how it may have been implemented in the hardware.
Not with void* pointers though, that is non-standard compiler extension. As mentioned on paragraph from the standard, it applies only to character type pointers.
Edit:
As mafso pointed out in comments, above is only true as long as type of substraction result ptrdiff_t, has enough range for the result. Since range of size_t can be larger than ptrdiff_t, and if structure is big enough, it's possible that addresses are too far apart.
Because of this it's preferable to use offsetof macro on structure members and calculate result from those.
I believe the answer to this question is simpler than it appears, the OP asks:
but why should that result be "undefined"?
Well, let's see that the definition of undefined behavior is in the draft C99 standard section 3.4.3:
behavior, upon use of a nonportable or erroneous program construct or
of erroneous data, for which this International Standard imposes no
requirements
it is simply behavior for which the standard does not impose a requirement, which perfectly fits this situation, the results are going to vary depending on the architecture and attempting to specify the results would have probably been difficult if not impossible in a portable manner. This leaves the question, why would they choose undefined behavior as opposed to let's say implementation of unspecified behavior?
Most likely it was made undefined behavior to limit the number of ways an invalid pointer could be created, this is consistent with the fact that we are provided with offsetof to remove the one potential need for pointer subtraction of unrelated objects.
Although the standard does not really define the term invalid pointer, we get a good description in Rationale for International Standard—Programming Languages—C which in section 6.3.2.3 Pointers says (emphasis mine):
Implicit in the Standard is the notion of invalid pointers. In
discussing pointers, the Standard typically refers to “a pointer to an
object” or “a pointer to a function” or “a null pointer.” A special
case in address arithmetic allows for a pointer to just past the end
of an array. Any other pointer is invalid.
The C99 rationale further adds:
Regardless how an invalid pointer is created, any use of it yields
undefined behavior. Even assignment, comparison with a null pointer
constant, or comparison with itself, might on some systems result in
an exception.
This strongly suggests to us that a pointer to padding would be an invalid pointer, although it is difficult to prove that padding is not an object, the definition of object says:
region of data storage in the execution environment, the contents of
which can represent values
and notes:
When referenced, an object may be interpreted as having a particular
type; see 6.3.2.1.
I don't see how we can reason about the type or the value of padding between elements of a struct and therefore they are not objects or at least is strongly indicates padding is not meant to be considered an object.
I should point out the following:
from the C99 standard, section 6.7.2.1:
Within a structure object, the non-bit-field members and the units in which bit-fields
reside have addresses that increase in the order in which they are declared. A pointer to a
structure object, suitably converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.
It isn't so much that the result of pointer subtraction between members is undefined so much as it is unreliable (i.e. not guaranteed to be the same between different instances of the same struct type when the same arithmetic is applied).
I can't seem to wrap my head around certain parts of the C standard, so I'm coming here to clear up that foggy, anxious uncertainty that comes when I have to think about what such tricks are defined behaviour and what are undefined or violate the standard. I don't care whether or not it will WORK, I care if the C standard considers it legal, defined behaviour.
Such as this, which I am fairly certain is UB:
struct One
{
int Hurr;
char Durr[2];
float Nrrr;
} One;
struct Two
{
int Hurr;
char Durr[2];
float Nrrr;
double Wibble;
} Two;
One = *(struct One*)&Two;
This is not all I am talking about. Such as casting the pointer to One to int*, and dereferencing it, etc. I want to get a good understanding of what such things are defined so I can sleep at night. Cite places in the standard if you can, but be sure to specify whether it's C89 or C99. C11 is too new to be trusted with such questions IMHO.
I think that technically that example is UB, too. But it will almost certainly work, and neither gcc nor clang complain about it with -pedantic.
To start with, the following is well-defined in C99 (§6.5.2.3/6): [1]
union OneTwo {
struct One one;
struct Two two;
};
OneTwo tmp = {.two = {3, {'a', 'b'}, 3.14f, 3.14159} };
One one = tmp.one;
The fact that accessing the "punned" struct One through union must work implies that the layout of the prefix of struct Two is identical to struct One. This cannot be contingent on the existence of a union because the a given composite type can only have one storage layout, and its layout cannot be contingent on its use in a union because the union does not need to be visible to every translation unit in which the struct is used.
Furthermore, in C all types are no more than a sequence of bytes (unlike, for example, C++) (§6.2.6.1/4) [2]. Consequently, the following is also guaranteed to work:
struct One one;
struct Two two = ...;
unsigned char tmp[sizeof one];
memcpy(tmp, two, sizeof one);
memcpy(one, tmp, sizeof one);
Given the above and the convertibility of any pointer type to a void*, I think it is reasonable to conclude that the temporary storage above is unnecessary, and it could have been written directly as:
struct One one;
struct Two two = ...;
unsigned char tmp[sizeof one];
memcpy(one, two, sizeof one);
From there to the direct assignment through an aliased pointer as in the OP is not a very big leap, but there is an additional problem for the aliased pointer: it is theoretically possible for the pointer conversion to create an invalid pointer, because it's possible that the bit format of a struct Two* differs from a struct One*. Although it is legal to cast one pointer type to another pointer type with looser alignment (§6.3.2.3/7) [3] and then convert it back again, it is not guaranteed that the converted pointer is actually usable, unless the conversion is to a character type. In particular, it is possible that the alignment of struct Two is different from (more strict than) the alignment of struct One, and that the bit format of the more strongly-aligned pointer is not directly usable as a pointer to the less strongly-aligned struct. However, it is hard to see an argument against the almost equivalent:
one = *(struct One*)(void*)&two;
although this may not be explicitly guaranteed by the standard.
In comments, various people have raised the spectre of aliasing optimizations. The above discussion does not touch on aliasing at all because I believe that it is irrelevant to a simple assignment. The assignment must be sequenced after any preceding expressions and before any succeeding ones; it clearly modifies one and almost as clearly references two. An optimization which made a preceding legal mutation of two invisible to the assignment, would be highly suspect.
But aliasing optimizations are, in general, possible. Consequently, even though all of the above pointer casts should be acceptable in the context of a single assignment expression, it would certainly not be legal behaviour to retain the converted pointer of type struct One* which actually points into an object of type struct Two and expect it to be usable either to mutate a member of its target or to access a member of its target which has otherwise been mutated. The only context in which you could get away with using a pointer to struct One as though it were a pointer to the prefix of struct Two is when the two objects are overlaid in a union.
--- Standard references:
[1] "if a union contains several structures that share a common initial sequence (see below), and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union is visible."
[2] "Values stored in non-bit-field objects of any other object type consist of n × CHAR_BIT
bits, where n is the size of an object of that type, in bytes. The value may be copied into
an object of type unsigned char [n] (e.g., by memcpy)…"
[3] "A pointer to an object type may be converted to a pointer to a different object type… When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers to the remaining bytes of the object."
C99 6.7.2.1 says:
Para 5
As discussed in 6.2.5, a structure is a type consisting of a sequence
of members, whose storage is allocated in an ordered sequence
Para 12
Each non-bit-field member of a structure or union object is aligned in
an implementation-defined manner appropriate to its type.
Para 13
Within a structure object, the non-bit-field members and the units in
which bit-fields reside have addresses that increase in the order in
which they are declared. A pointer to a structure object, suitably
converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There
may be unnamed padding within a structure object, but not at its
beginning
That last paragraph covers your second question (casting the pointer to One to int*, and dereferencing it).
The first point - whether it is valid to "Downcast" a Two* to a One* - I could not find specifically addressed. It boils down to whether the other rules ensure that the memory layout of the fields of One and the initial fields of Two are identical in all cases.
The members have to be packed in ordered sequence, no padding is allowed at the beginning, and they have to be aligned according to type, but the standard does not actually say that the layout needs to be the same (even though in most compilers I am sure it is).
There is, however, a better way to define these structures so that you can guarantee it:
struct One
{
int Hurr;
char Durr[2];
float Nrrr;
} One;
struct Two
{
struct One one;
double Wibble;
} Two;
You might think you can now safely cast a Two* to a One* - Para 13 says so. However strict aliasing might bite you somewhere unpleasant. But with the example above you don't need to anyway:
One = Two.one;
A1. Undefined behaviour, because of Wibble.
A2. Defined.
S9.2 in N3337.
Two standard-layout struct (Clause 9) types are layout-compatible if
they have the same number of non-static data members and corresponding
non-static data members (in declaration order) have layout-compatible
types
Your structs would be layout compatible and thus interchangeable but for Wibble. There is a good reason too: Wibble might cause different padding in struct Two.
A pointer to a standard-layout struct object, suitably converted using
a reinterpret_cast, points to its initial member (or if that member is
a bit-field, then to the unit in which it resides) and vice versa.
I think that guarantees that you can dereference the initial int.