Related
With an anonymous union declared in a struct, you can access the members directly. This made sense and I thought, like a normal union, you could only read from the most recent value that has been written to. Then I saw this
#include<stdio.h>
struct Scope
{
// Anonymous union
union
{
char alpha;
int num;
};
};
int main()
{
struct Scope x;
x.num = 65;
// Note that members of union are accessed directly
printf("x.alpha = %c, x.num = %d", x.alpha, x.num);
return 0;
}
What then, is the point if I can just access all of the variables all of the time? Why not just declare the variables in the scope of "Scope"?
According to C11 6.2.6.1, the value of the accessed member is unspecified. In general, it could be a trap representation, which triggers undefined behavior.
(If you changed char to unsigned char, it would be safe, since unsigned char cannot have trap representations. So your program would run to completion and print something, but the C standard does not specify what value would be printed for x.alpha.)
Of course, any given implementation may specify what value you actually get (e.g. the low byte, or the high byte, of x.num). So such code is most likely intended to work only on such implementations, and not meant to be portable or standard-conforming.
As Peter notes, all this is independent of whether you use anonymous unions or the old-fashioned kind.
The point is that struct Scope could have other members. A more realistic example:
struct Scope
{
union
{
int num;
uint8_t num_byte [sizeof(int)];
};
int foo;
};
Now you can access struct Scope members as obj.num or obj.num_byte[i]. After that union in memory, there will be a different variable foo, so clearly the union members can't get moved out to the struct.
If not for anonymous union, we'd have to type something like obj.name.num, where name is potentially just clutter.
Regarding reading different union members, your statement "you could only read from the most recent value that has been written to" is not true in C. Note that C and C++ are different here.
C17 6.5.2.3 states:
A postfix expression followed by the . operator and an identifier designates a member of
a structure or union object. The value is that of the named member,95) and is an lvalue if
the first expression is an lvalue.
where the foot note 95) is helpful:
95) If the member used to read the contents of a union object is not the same as the member last used to
store a value in the object, the appropriate part of the object representation of the value is reinterpreted
as an object representation in the new type as described in 6.2.6 (a process sometimes called ‘‘type
punning’’). This might be a trap representation.
The part in 6.2.6 referred to by the foot note:
When a value is stored in a member of an object of union type, the bytes of the object
representation that do not correspond to that member but do correspond to other members
take unspecified values.
Where an operator is applied to a value that has more than one object representation,
which object representation is used shall not affect the value of the result. Where a
value is stored in an object using a type that has more than one object representation for
that value, it is unspecified which representation is used, but a trap representation shall
not be generated.
What this means in plain English is that C allows type punning, but it's the programmers responsibility to ensure that it is feasible, with regards to alignment/padding, trap representations, endianess etc.
You can interpret same memory segment differently. Its widely used in protocols implementations. Where you receive buffer of bytes and based on various flags treat/decode it as needed.
In this example int and char are different size and byte order
I'm trying to understand how type-punning works when it comes to storing a value into a member of structure or union.
The Standard N1570 6.2.6.1(p6) specifies that
When a value is stored in an object of structure or union type,
including in a member object, the bytes of the object representation
that correspond to any padding bytes take unspecified values.
So I interpreted it as if we have an object to store into a member such that the size of the object equals the sizeof(declared_type_of_the_member) + padding the bytes related to padding will have unspecified value (even in spite of the fact that we had the bytes in the original object defined). Here is an example:
struct first_member_padded_t{
int a;
long b;
};
int a = 10;
struct first_member_padded_t s;
char repr[offsetof(struct first_member_padded_t, b)] = //some value
memcpy(repr, &a, sizeof(a));
memcpy(&(s.a), repr, sizeof(repr));
s.b = 100;
printf("%d%ld\n", s.a, s.b); //prints 10100
On my machine sizeof(int) = 4, offsetof(struct first_member_padded_t, b) = 8.
Is the behavior of printing 10100 well defined for such a program? I thing that it is.
What the memcpy Calls Do
The question is poorly posed. Let’s look first at the code:
char repr[offsetof(struct first_member_padded_t, b)] = //some value
memcpy(repr, &a, sizeof(a));
memcpy(&(s.a), repr, sizeof(repr));
First note that repr is initialized, so all the elements in it are given values.
The first memcpy is fine—it copies the bytes of a into repr.
If the second memcpy were memcpy(&s, repr, sizeof repr);, it would copy bytes from repr into s. This would write bytes into s.a and, due to the size of repr, into any padding between s.a and s.b. Per C 2018 6.5 7 and other pats of the standard, it is permitted to access the bytes of an object (and “access” means both reading and writing, per 3.1 1). So this copy into s is fine, and it results in s.a taking on the same value that a has.
However, the memcpy uses &(s.a) rather than &s. It uses the address of s.a rather than the address of s. We know that converting s.a to a pointer to a character type would allow us to access the bytes of s.a (6.5 7 and more) (and passing it to memcpy has the same effect as such a conversion, as memcpy is specified to have the effect of copying bytes), but it is not clear it allows us to access other bytes in s. In other words, we have a question of whether we can use &s.a to access bytes other than those in s.a.
6.7.2.1 15 tells us that, if a pointer to the first member of a structure is “suitably converted,” the result points to the structure. So, if we converted &s.a to a pointer to struct first_member_padding_t, it would point to s, and we can certainly use a pointer to s to access all the bytes in s. Thus, this would also be well defined:
memcpy((struct first_member_padding t *) &s.a, repr, sizeof repr);
However, memcpy(&s.a, repr, sizeof repr); only converts &s.a to void * (because memcpy is declared to take a void *, so &s.a is automatically converted during the function call) and not to a pointer to the structure type. Is that a suitable conversion? Note that if we did memcpy(&s, repr, sizeof repr);, it would convert &s to void *. 6.2.5 28 tells us that a pointer to void has the same representation as a pointer to a character type. So consider these two statements:
memcpy(&s.a, repr, sizeof repr);
memcpy(&s, repr, sizeof repr);
Both of these statements pass a void * to memcpy, and those two void * have the same representation as each other and point to the same byte. Now, we might interpret the standard pedantically and strictly so that they are different in that the latter may be used to access all the bytes of s and the former may not. Then it is bizarre that we have two necessarily identical pointers that behave differently.
Such a severe interpretation of the C standard seems possible in theory—the difference between the pointers could arise during optimization rather than in the actual implementation of memcpy—but I am not aware of any compiler that would do this. Note that such an interpretation is at odds with section 6.2 of the standard, which tells us about types and representations. Interpreting the standard so that (void *) &s.a and (void *) &s behave differently means that two things with the same value and type may behave differently, which means a value consists of something more than its value and type, which does not seem to be the intent of 6.2 or the standard generally.
Type-Punning
The question states:
I'm trying to understand how type-punning works when it comes to storing a value into a member of structure or union.
This is not type-punning as the term is commonly used. Technically, the code does access s.a using lvalues of a different type than its definition (because it uses memcpy, which is defined to copy as if with character type, while the defined type is int), but the bytes originate in an int and are copied without modification, and this sort of copying the bytes of an object is generally regarded as a mechanical procedure; it is done to effect a copy and not to reinterpret the bytes in a new type. “Type-punning” usually refers to using different lvalues for the purpose of reinterpreting the value, such as writing an unsigned int and reading a float.
In any case, type-punning is not really the subject of the question.
Values In Members
The title asks:
What values can we store in a struct or union members?
This title seems off from the content of the question. The title question is easily answered: The values we can store in a member are those values the member’s type can represent. But the question goes on to explore the padding between members. The padding does not affect the values in the members.
Padding Takes Unspecified Values
The question quotes the standard:
When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values.
and says:
So I interpreted it as if we have an object to store into a member such that the size of the object equals the sizeof(declared_type_of_the_member) + padding the bytes related to padding will have unspecified value…
The quoted text in the standard means that, if the padding bytes in s have been set to some values, as with memcpy, and we then do s.a = something;, then the padding bytes are no longer required to hold their previous values.
The code in the question explores a different situation. The code memcpy(&(s.a), repr, sizeof(repr)); does not store a value in a member of the structure in the sense meant in 6.2.6.1 6. It is not storing into either of the members s.a or s.b. It is copying bytes in, which is a different thing from what is discussed in 6.2.6.1.
6.2.6.1 6 means that, for example, if we execute this code:
char repr[sizeof s] = { 0 };
memcpy(&s, repr, sizeof s); // Set all the bytes of s to known values.
s.a = 0; // Store a value in a member.
memcpy(repr, &s, sizeof s); // Get all the bytes of s to examine them.
for (size_t i = sizeof s.a; i < offsetof(struct first_member_padding_t, b); ++i)
printf("Byte %zu = %d.\n", i, repr[i]);
then it is not necessarily true that all zeros will be printed—the bytes in the padding may have changed.
In many implementations of the language the C Standard was written to describe, an attempt to write an N-byte object within a struct or union would affect the value of at most N bytes within the struct or union. On the other hand, on a platform which supported 8-bit and 32-bit stores, but not 16-bit stores, if someone declared a type like:
struct S { uint32_t x; uint16_t y;} *s;
and then executed s->y = 23; without caring about what happened to the two bytes following y, it would be faster to performs a 32-bit store to y, blindly overwriting the two bytes following it, than to perform a pair of 8-bit writes to update the upper and lower halves of y. The authors of the Standard didn't want to forbid such treatment.
It would have been helpful if the Standard had included a means by which implementations could indicate whether writes to structure or union members might disturb storage beyond them, and programs that would be broken by such disturbance could refuse to run on implementations where it could occur. The authors of the Standard, however, likely expected that programmers who would be interested in such details would know what kinds of hardware their program was expected to run on, and thus know whether such memory disturbances would be an issue on such hardware.
Unfortunately, modern compiler writers seem to interpret freedoms that were intended to assist implementations for unusual hardware as an open invitation to get "creative" even when targeting platforms that could process code efficiently without such concessions.
As #user694733 said, in case there is padding between s.a and s.b, memcpy() is accessing a memory area that cannot be accessed by &a:
int a = 1;
int b;
b = *((char *)&a + sizeof(int));
This is Undefined Behaviour, and it is basically what is happening inside memcpy().
This is a quote from the C11 Standard:
6.5 Expressions
...
6 The effective type of an object for an access to its stored value is the declared type of the object, if any. If a value is stored into an object having no declared type through an lvalue having a type that is not a character type, then the type of the lvalue becomes the effective type of the object for that access and for subsequent accesses that do not modify the stored value. If a value is copied into an object having no declared type using memcpy or memmove, or is copied as an array of character type, then the effective type of the modified object for that access and for subsequent accesses that do not modify the value is the effective type of the object from which the value is copied, if it has one. For all other accesses to an object having no declared type, the effective type of the object is simply the type of the lvalue used for the access.
7 An object shall have its stored value accessed only by an lvalue expression that has one of the following types:
— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the object,
— a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object,
— an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or
— a character type.
Does this imply that memcpy cannot be used for type punning this way:
double d = 1234.5678;
uint64_t bits;
memcpy(&bits, &d, sizeof bits);
printf("the representation of %g is %08"PRIX64"\n", d, bits);
Why would it not give the same output as:
union { double d; uint64_t i; } u;
u.d = 1234.5678;
printf("the representation of %g is %08"PRIX64"\n", d, u.i);
What if I use my version of memcpy using character types:
void *my_memcpy(void *dst, const void *src, size_t n) {
unsigned char *d = dst;
const unsigned char *s = src;
for (size_t i = 0; i < n; i++) { d[i] = s[i]; }
return dst;
}
EDIT: EOF commented that The part about memcpy() in paragraph 6 doesn't apply in this situation, since uint64_t bits has a declared type. I agree, but, unfortunately, this does not help answer the question whether memcpy can be used for type punning, it just makes paragraph 6 irrelevant to assess the validity of the above examples.
Here here is another attempt at type punning with memcpy that I believe would be covered by paragraph 6:
double d = 1234.5678;
void *p = malloc(sizeof(double));
if (p != NULL) {
uint64_t *pbits = memcpy(p, &d, sizeof(double));
uint64_t bits = *pbits;
printf("the representation of %g is %08"PRIX64"\n", d, bits);
}
Assuming sizeof(double) == sizeof(uint64_t), Does the above code have defined behavior under paragraph 6 and 7?
EDIT: Some answers point to the potential for undefined behavior coming from reading a trap representation. This is not relevant as the C Standard explicitly excludes this possibility:
7.20.1.1 Exact-width integer types
1 The typedef name intN_t designates a signed integer type with width N, no padding bits, and a two’s complement representation. Thus, int8_t denotes such a signed integer type with a width of exactly 8 bits.
2 The typedef name uintN_t designates an unsigned integer type with width N and no padding bits. Thus, uint24_t denotes such an unsigned integer type with a width of exactly 24 bits.
These types are optional. However, if an implementation provides integer types with widths of 8, 16, 32, or 64 bits, no padding bits, and (for the signed types) that have a two’s complement representation, it shall define the corresponding typedef names.
Type uint64_t has exactly 64 value bits and no padding bits, thus there cannot be any trap representations.
There are two cases to consider: memcpy()ing into an object that has a declared type, and memcpy()ing into an object that does not.
In the second case,
double d = 1234.5678;
void *p = malloc(sizeof(double));
assert(p);
uint64_t *pbits = memcpy(p, &d, sizeof(double));
uint64_t bits = *pbits;
printf("the representation of %g is %08"PRIX64"\n", d, bits);
The behavior is indeed undefined, since the effective type of the object pointed to by p will become double, and accessing an object of effective type double though an lvalue of type uint64_t is undefined.
On the other hand,
double d = 1234.5678;
uint64_t bits;
memcpy(&bits, &d, sizeof bits);
printf("the representation of %g is %08"PRIX64"\n", d, bits);
is not undefined. C11 draft standard n1570:
7.24.1 String function conventions
3 For all functions in this subclause, each character shall be interpreted as if it had the type
unsigned char (and therefore every possible object representation is
valid and has a different value).
And
6.5 Expressions
7 An object shall have its stored value accessed only by an lvalue expression that has one of the following types: 88)
— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the object,
— a type that is the signed or unsigned type corresponding to a qualified version of the
effective type of the object,
— an aggregate or union type that includes one of the aforementioned types
among its members (including, recursively, a member of a subaggregate or contained union), or
— a character type.
Footnote 88) The intent of this list is to specify those circumstances in which an object may or may not be aliased.
So the memcpy() itself is well-defined.
Since uint64_t bits has a declared type, it retains its type even though its object representation was copied from a double.
As chqrlie points out, uint64_t cannot have trap representations, so accessing bits after the memcpy() is not undefined, provided sizeof(uint64_t) == sizeof(double). However, the value of bits will be implementation-dependent (for example due to endianness).
Conclusion: memcpy() can be used for type-punning, provided that the destination of the memcpy() does have a declared type, i.e. is not allocated by [m/c/re]alloc() or equivalent.
You propose 3 ways which all have different problems with C standard.
standard library memcpy
double d = 1234.5678;
uint64_t bits;
memcpy(&bits, &d, sizeof bits);
printf("the representation of %g is %08"PRIX64"\n", d, bits);
The memcpy part is legal (provided in your implementation sizeof(double) == sizeof(uint64_t) which is not guaranteed per standard): you access two objects through char pointers.
But the printf line is not. The representation in bits is now a double. it might be a trap representation for an uint64_t, as defined in 6.2.6.1 General §5
Certain object representations need not represent a value of the object type. If the stored
value of an object has such a representation and is read by an lvalue expression that does
not have character type, the behavior is undefined. If such a representation is produced
by a side effect that modifies all or any part of the object by an lvalue expression that
does not have character type, the behavior is undefined. Such a representation is called
a trap representation.
And 6.2.6.2 Integer types says explicitely
For unsigned integer types other than unsigned char, the bits of the object
representation shall be divided into two groups: value bits and padding bits ... The values of any padding bits are unspecified.53
With note 53 saying:
Some combinations of padding bits might generate trap representations,
If you know that in your implementation there are no padding bits (still never seen one...) every representation is a valid value, and the print line becomes valid again. But it is only implementation dependant and can be undefined behaviour in the general case
union
union { double d; uint64_t i; } u;
u.d = 1234.5678;
printf("the representation of %g is %08"PRIX64"\n", d, u.i);
The members of the union do not share a common subsequence, and you are accessing a member which is not the last value written. Ok common implementation will give expected results but per standard it is not explicitely defined what should happen. A footnote in 6.5.2.3 Structure and union members §3 says that if leads to same problems as previous case:
If the member used to access the contents of a union object is not the same as the member last used to
store a value in the object, the appropriate part of the object representation of the value is reinterpreted
as an object representation in the new type as described in 6.2.6 (a process sometimes called "type
punning"). This might be a trap representation.
custom memcpy
Your implementation only does character accesses which is always allowed. It is exactly the same thing as the first case: implementation defined.
The only way that would be explicitely defined per standard would be to store the representation of the double in an char array of the correct size, and then display the bytes values of the char array:
double d = 1234.5678;
unsigned char bits[sizeof(d)];
memcpy(&bits, &d, sizeof(bits));
printf("the representation of %g is ", d);
for(int i=0; i<sizeof(bits); i++) {
printf("%02x", (unsigned int) bits[i]);
}
printf("\n");
And the result will only be useable if the implementation uses exactly 8 bits for a char. But it would be visible because it would display more than 8 hexa digits if one of the bytes had a value greater than 255.
All of the above is only valid because bits has a declared type. Please see #EOF's answer to understand why it would be different for an allocated object
I read paragraph 6 as saying that using the memcpy() function to copy a series of bytes from one memory location to another memory location can be used for type punning just as using a union with two different types can be used for type punning.
The first mention of using memcpy() indicates that if it copies the specified number of bytes and that those bytes will have the same type as the variable at the source destination when that variable (lvalue) was used to store the bytes there.
In other words if you have a variable double d; and you then assign a value to this variable (lvalue) the type of the data stored in that variable is type double. If you then use the memcpy() function to copy those bytes to another memory location, say a variable uint64_t bits; the type of those copied bytes is still double.
If you then access the copied bytes through the destination variable (lvalue), the uint64_t bits; in the example, then the type of that data is seen as the type of the lvalue used to retrieve the data bytes from that destination variable. So the bytes are interpreted (not converted but interpreted) as the destination variable type rather than the type of the source variable.
Accessing the bytes through a different type means the bytes are now interpreted as the new type even though the bytes have not actually changed in any way.
This is also the way a union works. A union does not do any kind of conversion. You store bytes into a union member which is of one type and then you pull the same bytes back out through a different union member. The bytes are the same however the interpretation of the bytes depends on the type of the union member that is used to access the memory area.
I have seen the memcpy() function used in older C source code to help divide up a struct into pieces by using struct member offset along with the memcpy() function to copy portions of the struct variable into other struct variables.
Because the type of the source location used in the memcpy() is the type of the bytes stored there the same kinds of problems that you can run into with the use of a union for punning also apply to using memcpy() in this way such as the Endianness of the data type.
The thing to remember is that whether using a union or using the memcpy() approach the type of the bytes copied are the type of the source variable and when you then access the data as another type, whether through a different member of a union or through the destination variable of the memcpy() the bytes are interpreted as the type of the destination lvalue. However the actual bytes are not changed.
CHANGED--SEE BELOW
While I have never observed a compiler to interpret a memcpy of non-overlapping source and destination as doing anything that would not be equivalent to reading all of the bytes of the source as a character type and then writing all of the bytes of the destination as a character type (meaning that if the destination had no declared type, it would be left with no effective type), the language of the Standard would allow obtuse compilers to make "optimizations" which--in those rare instances where a compiler would be able to identify and exploit them--would be more likely to break code which would otherwise work (and would be well-defined if the Standard were better written) than to actually improve efficiency.
As to whether that means that it's better to use memcpy or a manual byte-copy loop whose purpose is sufficiently well-disguised as to be unrecognizable as "copying an array of character type", I have no idea. I would posit that the sensible thing would be to shun anyone so obtuse as to suggest that a good compiler should generate bogus code absent such obfuscation, but since behavior that would have been considered obtuse in years past is presently fashionable, I have no idea whether memcpy will be the next victim in the race to break code which compilers had for decades treated as "well-defined".
UPDATE
GCC as of 6.2 will sometimes omit memmove operations in cases where it sees that the destination and source identify the same address, even if they are pointers of different types. If storage which had been written as the source type is later read as the destination type, gcc will assume that the latter read cannot identify the same storage as the earlier write. Such behavior on gcc's part is justifiable only because of the language in the Standard which allows the compiler to copy the Effective Type through the memmove. It's unclear whether that was an intentional interpretation of the rules regarding memcpy, however, given that gcc will also make a similar optimization in some cases where it is clearly not allowed by the Standard, e.g. when a union member of one type (e.g. 64-bit long) is copied to a temporary and from there to a member of a different type with the same representation (e.g. 64-bit long long). If gcc sees that the destination will be bit-for-bit identical to the temporary, it will omit the write, and consequently fail to notice that the effective type of the storage was changed.
It might give the same result, but the compiler does not need to guarantee it. So you simply cannot rely on it.
The C99 standard states that:
When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object
Consider the following code:
struct test {
int x[5];
char something;
short y[5];
};
...
struct test s = { ... };
char *p = (char *) s.x;
char *q = (char *) s.y;
printf("%td\n", q - p);
This obviously breaks the above rule, since the p and q pointers are pointing to different "array objects", and, according to the rule, the q - p difference is undefined.
But in practice, why should such a thing ever result in undefined behaviour? After all, the struct members are laid out sequentially (just as array elements are), with any potential padding between the members. True, the amount of padding will vary across implementations and that would affect the outcome of the calculations, but why should that result be "undefined"?
My question is, can we suppose that the standard is just "ignorant" of this issue, or is there a good reason for not broadening this rule? Couldn't the above rule be rephrased to "both shall point to elements of the same array object or members of the same struct"?
My only suspicion are segmented memory architectures where the members might end up in different segments. Is that the case?
I also suspect that this is the reason why GCC defines its own __builtin_offsetof, in order to have a "standards compliant" definition of the offsetof macro.
EDIT:
As already pointed out, arithmetic on void pointers is not allowed by the standard. It is a GNU extension that fires a warning only when GCC is passed -std=c99 -pedantic. I'm replacing the void * pointers with char * pointers.
Subtraction and relational operators (on type char*) between addresses of member of the same struct are well defined.
Any object can be treated as an array of unsigned char.
Quoting N1570 6.2.6.1 paragraph 4:
Values stored in non-bit-field objects of any other object type
consist of n × CHAR_BIT bits, where n is the size of an object of that
type, in bytes. The value may be copied into an object of type
unsigned char [ n ] (e.g., by memcpy); the resulting set of bytes is
called the object representation of the value.
...
My only suspicion are segmented memory architectures where the members
might end up in different segments. Is that the case?
No. For a system with a segmented memory architecture, normally the compiler will impose a restriction that each object must fit into a single segment. Or it can permit objects that occupy multiple segments, but it still has to ensure that pointer arithmetic and comparisons work correctly.
Pointer arithmetic requires that the two pointers being added or subtracted to be part of the same object because it doesn't make sense otherwise.
The quoted section of standard specifically refers to two unrelated objects such as int a[b]; and int b[5]. The pointer arithmetic requires to know the type of the object that the pointers pointing to (I am sure you are aware of this already).
i.e.
int a[5];
int *p = &a[1]+1;
Here p is calculated by knowing the that the &a[1] refers to an int object and hence incremented to 4 bytes (assuming sizeof(int) is 4).
Coming to the struct example, I don't think it can possibly be defined in a way to make pointer arithmetic between struct members legal.
Let's take the example,
struct test {
int x[5];
char something;
short y[5];
};
Pointer arithmatic is not allowed with void pointers by C standard (Compiling with gcc -Wall -pedantic test.c would catch that). I think you are using gcc which assumes void* is similar to char* and allows it.
So,
printf("%zu\n", q - p);
is equivalent to
printf("%zu", (char*)q - (char*)p);
as pointer arithmetic is well defined if the pointers point to within the same object and are character pointers (char* or unsigned char*).
Using correct types, it would be:
struct test s = { ... };
int *p = s.x;
short *q = s.y;
printf("%td\n", q - p);
Now, how can q-p be performed? based on sizeof(int) or sizeof(short) ? How can the size of char something; that's in the middle of these two arrays be calculated?
That should explain it's not possible to perform pointer arithmetic on objects of different types.
Even if all members are of same type (thus no type issue as stated above), then it's better to use the standard macro offsetof (from <stddef.h>) to get the difference between struct members which has the similar effect as pointer arithmetic between members:
printf("%zu\n", offsetof(struct test, y) - offsetof(struct test, x));
So I see no necessity to define pointer arithmetic between struct members by the C standard.
Yes, you are allowed to perform pointer arithmetric on structure bytes:
N1570 - 6.3.2.3 Pointers p7:
... When a pointer to an object is converted to a pointer to a character type,
the result points to the lowest addressed byte of the object. Successive increments of the
result, up to the size of the object, yield pointers to the remaining bytes of the object.
This means that for the programmer, bytes of the stucture shall be seen as a continuous area, regardless how it may have been implemented in the hardware.
Not with void* pointers though, that is non-standard compiler extension. As mentioned on paragraph from the standard, it applies only to character type pointers.
Edit:
As mafso pointed out in comments, above is only true as long as type of substraction result ptrdiff_t, has enough range for the result. Since range of size_t can be larger than ptrdiff_t, and if structure is big enough, it's possible that addresses are too far apart.
Because of this it's preferable to use offsetof macro on structure members and calculate result from those.
I believe the answer to this question is simpler than it appears, the OP asks:
but why should that result be "undefined"?
Well, let's see that the definition of undefined behavior is in the draft C99 standard section 3.4.3:
behavior, upon use of a nonportable or erroneous program construct or
of erroneous data, for which this International Standard imposes no
requirements
it is simply behavior for which the standard does not impose a requirement, which perfectly fits this situation, the results are going to vary depending on the architecture and attempting to specify the results would have probably been difficult if not impossible in a portable manner. This leaves the question, why would they choose undefined behavior as opposed to let's say implementation of unspecified behavior?
Most likely it was made undefined behavior to limit the number of ways an invalid pointer could be created, this is consistent with the fact that we are provided with offsetof to remove the one potential need for pointer subtraction of unrelated objects.
Although the standard does not really define the term invalid pointer, we get a good description in Rationale for International Standard—Programming Languages—C which in section 6.3.2.3 Pointers says (emphasis mine):
Implicit in the Standard is the notion of invalid pointers. In
discussing pointers, the Standard typically refers to “a pointer to an
object” or “a pointer to a function” or “a null pointer.” A special
case in address arithmetic allows for a pointer to just past the end
of an array. Any other pointer is invalid.
The C99 rationale further adds:
Regardless how an invalid pointer is created, any use of it yields
undefined behavior. Even assignment, comparison with a null pointer
constant, or comparison with itself, might on some systems result in
an exception.
This strongly suggests to us that a pointer to padding would be an invalid pointer, although it is difficult to prove that padding is not an object, the definition of object says:
region of data storage in the execution environment, the contents of
which can represent values
and notes:
When referenced, an object may be interpreted as having a particular
type; see 6.3.2.1.
I don't see how we can reason about the type or the value of padding between elements of a struct and therefore they are not objects or at least is strongly indicates padding is not meant to be considered an object.
I should point out the following:
from the C99 standard, section 6.7.2.1:
Within a structure object, the non-bit-field members and the units in which bit-fields
reside have addresses that increase in the order in which they are declared. A pointer to a
structure object, suitably converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.
It isn't so much that the result of pointer subtraction between members is undefined so much as it is unreliable (i.e. not guaranteed to be the same between different instances of the same struct type when the same arithmetic is applied).
Suppose I have the following structure:
typedef struct
{
unsigned field1 :1;
unsigned field2 :1;
unsigned field3 :1;
} mytype;
The first 3 bits will be usable but sizeof(mytype) will return 4 which means 29 bits of padding.
My question is, are these padding bits guaranteed by the standard to be zero initialized by the statement:
mytype testfields = {0};
or:
mytype myfields = {1, 1, 1};
Such that it's safe to perform the following memcmp() on the assumption that bits 4..29 will be zero and therefore won't affect the comparison:
if ( memcmp(&myfields, &testfields, sizeof(myfields)) == 0 )
printf("Fields have no bits set\n");
else
printf("Fields have bits set\n");
Yes and no. The actual standard, C11, specifies:
If an object that has static or thread storage duration is not
initialized explicitly, then:
....
if it is an aggregate, every member is initialized (recursively)
according to these rules, and any padding is initialized to zero bits;
So this only holds for objects of static storage, at a first view. But then later it says in addition:
If there are fewer initializers in a brace-enclosed list than there
are elements or members of an aggregate, or fewer characters in a
string literal used to initialize an array of known size than there
are elements in the array, the remainder of the aggregate shall be
initialized implicitly the same as objects that have static storage
duration.
So this means that padding inside sub-structures that are not initialized explicitly is zero-bit initialized.
In summarry, some padding in a structure is guaranteed to be zero-bit initialized, some isn't. I don't think that such a confusion is intentional, I will file a defect report for this.
Older versions didn't have that at all. So with most existing compilers you'd have to be even more careful, since they don't implement C11, yet. But AFAIR, clang already does on that behalf.
Also be aware that this only holds for initialization. Padding isn't necessarily copied on assignment.
The C99 standard doesn't specify the padding bits would be set to zero. In fact, it specifically mentions that the values of any padding bits are unspecified, so that padding need not be copied in an assignment.
Footnote 51 to 6.2.6.1 (6) (n1570):
Thus, for example, structure assignment need not copy any padding bits.
The new C2011 standard - thanks to Jens Gustedt for sharing that knowledge - specifies that padding bits in objects of static or thread storage duration without explicit initialisation are initialised to 0.
There are still no guarantees for assignment.
My question is, are these padding bits guaranteed by the standard to be zero initialized by the statement:
No.
The value of the padding is unspecified:
(C99, 6.2.6.1p6) "When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values"
EDIT: See Jens Gustedt answer, C11 now guarantees the padding is set to 0 in (rare) certain circumstances