I wanted to make something like a binary search tree for mapping addresses to Page *s (except it was actually hexadecimal, and the address is implied by the structure itself), so:
typedef union Map Map;
union Map {
Map (*ps)[16];
Page *p;
};
This makes perfect sense, logically (a union Map containing either a pointer to a Page or a pointer to an array of 16 Maps), but gcc errors at the ps declaration with array type has incomplete element type, so I guess this kind of recursive definition is not allowed in C.
Is there any way to do this without using tricks like pointer aliasing?
C 2018 6.7.6.2 1 specifies the constraints for array declarators. It says, in part:
The element type shall not be an incomplete or function type.
In Map (*ps)[16], the (*ps)[16] is a declarator as shown in the grammar at 6.7.6 1. Since it is an array declarator, it is subject to the rules in 6.7.6.2, and therefore the element type must be complete. This is true even though the ultimate type being declared is a pointer.
As noted in a comment, you can declare Map *ps instead. If this is unsatisfactory because then pointer arithmetic on ps works in units of Map instead of Map [16], an alternative could be to define typedef struct Map16 Map16; before the union, then Map16 *ps; inside the union, and then struct Map16 { Map element[16]; } after the union. That will make pointer arithmetic on ps work in the desired units (supposing the implementation does not pad the structure, which would be unusual), although it does make you use an extra .element when referencing elements.
In considering why Map *ps is accepted while Map (*ps)[16] is not, we can see that both declare pointers to incomplete types, and so it is not the completeness of the pointed-to type that distinguishes them but merely this rule in the C standard. It may be that the rule in 6.7.6.2 1 could have been modified to permit Map (*ps)[16], as it does not appear the compiler needs complete information about the pointed-to type at that point.
Related
Consider the following code:
int main()
{
typedef struct { int first; float second; } type;
type whole = { 1, 2.0 };
void * vp = &whole;
struct { int first; } * shorn = vp;
printf("values: %d, %d\n", ((type *)vp)->first, shorn->first);
if (vp == shorn)
printf("ptrs compare the same\n");
return 0;
}
Two questions:
Is the pointer equality comparison UB?
Regarding the "shearing" away of the second member on the line that initializes shorn: is it valid C to cast away struct members like this and then dereference the manipulated pointer to access the remaining member?
Comparing two pointers with == when one is a void * is well defined.
Section 6.5.9 of the C standard regarding the equality operator == says the following:
2 One of the following shall hold:
both operands have arithmetic type;
both operands are pointers to qualified or unqualified versions of compatible types;
one operand is a pointer to an object type and the other is a pointer to a qualified or unqualified version of void; or
one operand is a pointer and the other is a null pointer constant
...
5 Otherwise, at least one operand is a pointer. If one operand is a pointer and the other is a null pointer constant, the null pointer
constant is converted to the type of the pointer. If one operand
is a pointer to an object type and the other is a pointer
to a qualified or unqualified version of void, the former is
converted to the type of the latter.
The usage of shorn->first works because a pointer to a struct can be converted to a pointer to its first member. For both type and the unnamed struct type their first member is an int so it works out.
Section 6.2.5 Types paragraph 28 of the C standard says:
[...] All pointers to structure types shall have the same representation and alignment requirements as each other. [...]
Section 6.3.2.3 Pointers paragraph 1 says:
A pointer to void may be converted to or from a pointer to any object type. A pointer to any object type may be converted to a pointer to void and back again; the result shall compare equal to the original pointer.
And paragraph 7 says:
A pointer to an object type may be converted to a pointer to a different object type. If the resulting pointer is not correctly aligned68) for the referenced type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer. [...]
And footnote 68 says:
In general, the concept "correctly aligned" is transitive: if a pointer to type A is correctly aligned for a pointer to type B, which in turn is correctly aligned for a pointer to type C, then a pointer to type A is correctly aligned for a pointer to type C.
Because all pointers to structure types have the same representation, the conversions between pointers to void and pointers to structure types must be the same for all pointers to structure types. So it seems that a pointer to structure type A could be converted by a cast operator directly to a pointer to structure type B without an intermediate conversion to a pointer to void as long as the pointer is "correctly aligned" for structure type B. (This may be a weak argument.)
The question remains when, in the case of two structure types A and B where the initial sequence of structure type A consists of all the members of structure type B, a pointer to structure type A is guaranteed to be correctly aligned for structure type B (the reverse is obviously not guaranteed). As far as I can tell, the C standard makes no such guarantee. So strictly speaking, a pointer to the larger structure type A might not be correctly aligned for the smaller structure type B, and if it is not, the behavior is undefined. For a "sane" compiler, the larger structure type A would not have weaker alignment than the smaller structure type B, but for an "insane" compiler, that might not be the case.
Regarding the second question about accessing members of the truncated (shorter) structure using the pointer derived from the full (longer) structure, then as long as the pointer is correctly aligned for the shorter structure (see above for why that might not be true for an "insane" compiler), and as long as strict aliasing rules are avoided (for example, by going through an intermediate pointer to void in an intermediate external function call across compilation unit boundaries), then accessing the members through the pointer to the shorter structure type should be perfectly fine. There is a special guarantee for that when objects of both structure types appear as members of the same union type. Section 6.3.2.3 Structure and union members paragraph 6 says:
One special guarantee is made in order to simplify the use of unions: if a union contains several structures that share a common initial sequence (see below), and if the union
object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union
is visible. Two structures share a common initial sequence if corresponding members have compatible types (and, for bit-fields, the same widths) for a sequence of one or more initial members.
However, since the offsets of members within a structure type does not depend on whether an object of the structure type appears in a union type or not, the above implies that any structures with a common initial sequence of members will have those common members at the same offsets within their respective structure types.
In the language the C89 Standard was written to describe, it was well established that if two structures share a common initial sequence, a pointer to either may be cast to the other and used to inspect members of that common initial sequence. Code which relied upon this was commonplace and not considered even remotely controversial.
In the interest of optimization, the authors of the C99 Standard deliberately allowed compilers to assume that structures of different types won't alias in cases where such assumption would be useful for their customers. Because there are many good means by which implementations could recognize cases where such assumptions would be needlessly break what had been perfectly good code, and because they the authors of the Standard expected that compiler writers would make a bona fide effort to behave in ways useful to the programmers using their products, the Standard doesn't mandate any particular means of making such distinctions. Instead, it regards the ability to support constructs that had been universally supported as a "quality of implementation" issue, which would be reasonable if compiler writers made a bona fide effort to treat it as such.
Unfortunately, some compiler writers who aren't interested in selling their products to paying customers have taken the Standard's failure to mandate useful behavior as an invitation to behave in needlessly useless fashion. Code which relies upon Common Initial Sequence guarantees can thus not be meaningfully processed by clang or gcc without using either non-standard syntax or disabling type-based aliasing entirely.
As mentioned here, here and here a function (in c99 or newer) defined this way
void func(int ptr[static 1]){
//do something with ptr, knowing that ptr != NULL
}
has one parameter (ptr) of type pointer to int and the compiler can assume that the function will never be called with null as argument. (e.g. the compiler can optimize null pointer checks away or warn if func is called with a nullpointer - and yes I know, that the compiler is not required to do any of that...)
C17 section 6.7.6.3 Function declarators (including prototypes) paragraph 7 says:
A declaration of a parameter as “array of type” shall be adjusted to “qualified pointer to type”, where
the type qualifiers (if any) are those specified within the [ and ] of the array type derivation. If the
keyword static also appears within the [ and ] of the array type derivation, then for each call to
the function, the value of the corresponding actual argument shall provide access to the first element
of an array with at least as many elements as specified by the size expression.
In case of the definition above the value of ptr has to provide access to the first element of an array with at least 1 element. It is therefore clear that the argument can never be null.
What I'm wandering is, whether it is valid to call such a function with the address of an int that is not part of an array. E.g. is this (given the definition of func above) technically valid or is it undefined behavior:
int var = 5;
func(&var);
I am aware that this will practically never be an issue, because no compiler I know of differentiates between a pointer to a member of an int array and a pointer to a local int variable. But given that a pointer in c (at least from the perspective of the standard) can be much more than just some integer with a special compile time type I wandered if there is some section in the standard, that makes this valid.
I do suspect, that it is actually not valid, as section 6.5.6 Additive operators paragraph 8 contains:
[...] If both the pointer operand and the result point
to elements of the same array object, or one past the last element of the array object, the evaluation
shall not produce an overflow; otherwise, the behavior is undefined. [...]
To me that sounds as if for any pointer that points to an array element adding 1 is a valid operation while it would be UB to add 1 to a pointer that points to a regular variable. That would mean, that there is indeed a difference between a pointer to an array element and a pointer to a normal variable, which would make the snippet above UB...
Section 6.5.6 Additive operators paragraph 7 contains:
For the purposes of these operators, a pointer to an object that is not an element of an array behaves
the same as a pointer to the first element of an array of length one with the type of the object as its
element type.
As the paragraph begins with "for the purposes of these operators" I suspect that there can be a difference in other contexts?
tl;dr;
Is there some section of the standard, that specifies, that there is no difference between a pointer to a regular variable of type T and a pointer to the element of an array of length one (array of type T[1])?
At face value, I think you have a point. We aren't really passing a pointer to the first element of an array. This may be UB if we consider the standard in a vacuum.
Other than the paragraph you quote in 6.5.6, there is no passage in the standard equating a single object to an array of one element. And there shouldn't be, since the two things are different. An array (of even one element) is implicitly converted to a pointer when appearing in most expressions. That's obviously not a property most object types posses.
The definition of the static keyword in [] mentions that the the pointer being passed, must be to the initial element of an array that contains at least a certain number of elements. There is another problem with the wording you cited, what about
int a[2];
func(a + 1);
Clearly the pointer being passed is not to the first element of an array. That is UB too if we take a literal interpretation of 6.7.6.3p7.
Putting the static keyword aside, when a function accepts a pointer to an object, whether the object is a member of an array (of any size) or not matters in only one context: pointer arithmetic.
In the absence of pointer arithmetic, there is no distinguishable difference in behavior when using a pointer to access an element of an array, or a standalone object.
I would argue that the intent behind 6.7.6.3p7 has pointer arithmetic in mind. And so the semantic being mentioned comes hand in hand with trying to do pointer arithmetic on the pointer being passed into the function.
The use of static 1 simply emerged naturally as useful idiom, and maybe wasn't the intent from get go. While the normative text may do with a slight correction, I think the intent behind it is clear. It isn't meant to be undefined behavior by the standard.
The authors of the Standard almost certainly intended that quality implementations would treat the value of a pointer to a non-array object in the same way as it would treat the value of a pointer to the first element of an array object of length 1. Had it merely said that a pointer to a non-array object was equivalent to a pointer to an array, however, that might have been misinterpreted as applying to all expressions that yield pointer values. This could cause problems given e.g. char a[1],*p=a;, because the expressions a and p both yield pointers of type char* with the same value, but sizeof p and sizeof a would likely yield different values.
The language was in wide use before the Standard was written, and it was hardly uncommon for programs to rely upon such behavior. Implementations that make a bona fide effort to behave in a fashion consistent with the Standard Committee's intentions as documented in the published Rationale document should thus be expected to process such code meaningfully without regard for whether a pedantic reading of the Standard would require it. Implementations that do not make such efforts, however, should not be trusted to process such code meaningfully.
Elements of the struct must be properly aligned, so struct can be padded with non-used bytes.
Size of the struct is always the same (since it's a constant expression).
So I presume compiler must somehow use the same padding every time such as:
placing a single struct instance
passing a struct instance by value to a function
returning a struct instance by value from a function
in the array of structs
If I'm correct with this presumption, can you please point me where does C standard says so? Are there any strict rules for padding placement that guarantee this property?
C locks down what the offsets of fields are in a particular structure type via the specification of the offsetof macro, in §7.17 of this version of the draft, which is describing the required macros in <stddef.h>.
The macros are
NULL
which expands to an implementation-defined null pointer constant; and
offsetof(type, member-designator)
which expands to an integer constant expression that has type size_t, the value of
which is the offset in bytes, to the structure member (designated by member-designator),
from the beginning of its structure (designated by type). The type and member designator
shall be such that given
static type t;
then the expression &(t.member-designator) evaluates to an address constant. (If the
specified member is a bit-field, the behavior is undefined.)
Since it is a constant expression, every use of a particular structure type must use the same offset (or this would all be completely crazy).
Two different struct types, even if declared the same, are not constrained by this to have the same layout. (For example, if the types were declared under the control of different structure layout pragmas, they would not be expected to necessarily have the same layout, yet the declarations would look prima facie identical. typedef is important!)
I can't seem to wrap my head around certain parts of the C standard, so I'm coming here to clear up that foggy, anxious uncertainty that comes when I have to think about what such tricks are defined behaviour and what are undefined or violate the standard. I don't care whether or not it will WORK, I care if the C standard considers it legal, defined behaviour.
Such as this, which I am fairly certain is UB:
struct One
{
int Hurr;
char Durr[2];
float Nrrr;
} One;
struct Two
{
int Hurr;
char Durr[2];
float Nrrr;
double Wibble;
} Two;
One = *(struct One*)&Two;
This is not all I am talking about. Such as casting the pointer to One to int*, and dereferencing it, etc. I want to get a good understanding of what such things are defined so I can sleep at night. Cite places in the standard if you can, but be sure to specify whether it's C89 or C99. C11 is too new to be trusted with such questions IMHO.
I think that technically that example is UB, too. But it will almost certainly work, and neither gcc nor clang complain about it with -pedantic.
To start with, the following is well-defined in C99 (§6.5.2.3/6): [1]
union OneTwo {
struct One one;
struct Two two;
};
OneTwo tmp = {.two = {3, {'a', 'b'}, 3.14f, 3.14159} };
One one = tmp.one;
The fact that accessing the "punned" struct One through union must work implies that the layout of the prefix of struct Two is identical to struct One. This cannot be contingent on the existence of a union because the a given composite type can only have one storage layout, and its layout cannot be contingent on its use in a union because the union does not need to be visible to every translation unit in which the struct is used.
Furthermore, in C all types are no more than a sequence of bytes (unlike, for example, C++) (§6.2.6.1/4) [2]. Consequently, the following is also guaranteed to work:
struct One one;
struct Two two = ...;
unsigned char tmp[sizeof one];
memcpy(tmp, two, sizeof one);
memcpy(one, tmp, sizeof one);
Given the above and the convertibility of any pointer type to a void*, I think it is reasonable to conclude that the temporary storage above is unnecessary, and it could have been written directly as:
struct One one;
struct Two two = ...;
unsigned char tmp[sizeof one];
memcpy(one, two, sizeof one);
From there to the direct assignment through an aliased pointer as in the OP is not a very big leap, but there is an additional problem for the aliased pointer: it is theoretically possible for the pointer conversion to create an invalid pointer, because it's possible that the bit format of a struct Two* differs from a struct One*. Although it is legal to cast one pointer type to another pointer type with looser alignment (§6.3.2.3/7) [3] and then convert it back again, it is not guaranteed that the converted pointer is actually usable, unless the conversion is to a character type. In particular, it is possible that the alignment of struct Two is different from (more strict than) the alignment of struct One, and that the bit format of the more strongly-aligned pointer is not directly usable as a pointer to the less strongly-aligned struct. However, it is hard to see an argument against the almost equivalent:
one = *(struct One*)(void*)&two;
although this may not be explicitly guaranteed by the standard.
In comments, various people have raised the spectre of aliasing optimizations. The above discussion does not touch on aliasing at all because I believe that it is irrelevant to a simple assignment. The assignment must be sequenced after any preceding expressions and before any succeeding ones; it clearly modifies one and almost as clearly references two. An optimization which made a preceding legal mutation of two invisible to the assignment, would be highly suspect.
But aliasing optimizations are, in general, possible. Consequently, even though all of the above pointer casts should be acceptable in the context of a single assignment expression, it would certainly not be legal behaviour to retain the converted pointer of type struct One* which actually points into an object of type struct Two and expect it to be usable either to mutate a member of its target or to access a member of its target which has otherwise been mutated. The only context in which you could get away with using a pointer to struct One as though it were a pointer to the prefix of struct Two is when the two objects are overlaid in a union.
--- Standard references:
[1] "if a union contains several structures that share a common initial sequence (see below), and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union is visible."
[2] "Values stored in non-bit-field objects of any other object type consist of n × CHAR_BIT
bits, where n is the size of an object of that type, in bytes. The value may be copied into
an object of type unsigned char [n] (e.g., by memcpy)…"
[3] "A pointer to an object type may be converted to a pointer to a different object type… When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object. Successive increments of the result, up to the size of the object, yield pointers to the remaining bytes of the object."
C99 6.7.2.1 says:
Para 5
As discussed in 6.2.5, a structure is a type consisting of a sequence
of members, whose storage is allocated in an ordered sequence
Para 12
Each non-bit-field member of a structure or union object is aligned in
an implementation-defined manner appropriate to its type.
Para 13
Within a structure object, the non-bit-field members and the units in
which bit-fields reside have addresses that increase in the order in
which they are declared. A pointer to a structure object, suitably
converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There
may be unnamed padding within a structure object, but not at its
beginning
That last paragraph covers your second question (casting the pointer to One to int*, and dereferencing it).
The first point - whether it is valid to "Downcast" a Two* to a One* - I could not find specifically addressed. It boils down to whether the other rules ensure that the memory layout of the fields of One and the initial fields of Two are identical in all cases.
The members have to be packed in ordered sequence, no padding is allowed at the beginning, and they have to be aligned according to type, but the standard does not actually say that the layout needs to be the same (even though in most compilers I am sure it is).
There is, however, a better way to define these structures so that you can guarantee it:
struct One
{
int Hurr;
char Durr[2];
float Nrrr;
} One;
struct Two
{
struct One one;
double Wibble;
} Two;
You might think you can now safely cast a Two* to a One* - Para 13 says so. However strict aliasing might bite you somewhere unpleasant. But with the example above you don't need to anyway:
One = Two.one;
A1. Undefined behaviour, because of Wibble.
A2. Defined.
S9.2 in N3337.
Two standard-layout struct (Clause 9) types are layout-compatible if
they have the same number of non-static data members and corresponding
non-static data members (in declaration order) have layout-compatible
types
Your structs would be layout compatible and thus interchangeable but for Wibble. There is a good reason too: Wibble might cause different padding in struct Two.
A pointer to a standard-layout struct object, suitably converted using
a reinterpret_cast, points to its initial member (or if that member is
a bit-field, then to the unit in which it resides) and vice versa.
I think that guarantees that you can dereference the initial int.
I have the following code:
int *pa;
int a[3] = {1, 2, 3};
Why pa = a is ok, but a = pa is not allowed?
The main difference is that type of a is still an array but it just decays into a pointer when you do pa=a;. pa will now point to the first element of the array not the entire array itself. When you do a=pa it doesnot make any sense as you are trying point a datatype which is holding 3 integers to a type which can point only to a single integer.
Note: This is purely conceptual, this is not the actual reason why this happens.
I like to think of pointer assignment like OOP & Inheritance.
Imagine int * is a generic object. Now, think of int [] as an object that inherits from int *.
As you can see, you can cast down from int [] to int *, but not casting upwards.
Well, the simple answer is that the language definition simply doesn't allow it - it's a design choice.
Chapter and verse:
6.5.16 Assignment operators
...
Constraints
2 An assignment operator shall have a modifiable lvalue as its left operand.
And what's a modifiable lvalue?
6.3.2.1 Lvalues, arrays, and function designators
1 An lvalue is an expression with an object type or an incomplete type other than void;53)
if an lvalue does not designate an object when it is evaluated, the behavior is undefined.
When an object is said to have a particular type, the type is specified by the lvalue used to
designate the object. A modifiable lvalue is an lvalue that does not have array type, does
not have an incomplete type, does not have a const-qualified type, and if it is a structure
or union, does not have any member (including, recursively, any member or element of
all contained aggregates or unions) with a const-qualified type.
...
53) The name ‘‘lvalue’’ comes originally from the assignment expression E1 = E2, in which the left
operand E1 is required to be a (modifiable) lvalue. It is perhaps better considered as representing an
object ‘‘locator value’’. What is sometimes called ‘‘rvalue’’ is in this International Standard described
as the ‘‘value of an expression’’.
Emphasis added.
Array expressions in C are treated differently than most other expressions. The reason for this is explained in an article Dennis Ritchie wrote about the development of the C language:
NB existed so briefly that no full description of it was written. It supplied the types int and char, arrays of them, and pointers to them, declared in a style typified by
int i, j;
char c, d;
int iarray[10];
int ipointer[];
char carray[10];
char cpointer[];
The semantics of arrays remained exactly as in B and BCPL: the declarations of iarray and carray create cells dynamically initialized with a value pointing to the first of a sequence of 10 integers and characters respectively. The declarations for ipointer and cpointer omit the size, to assert that no storage should be allocated automatically. Within procedures, the language's interpretation of the pointers was identical to that of the array variables: a pointer declaration created a cell differing from an array declaration only in that the programmer was expected to assign a referent, instead of letting the compiler allocate the space and initialize the cell.
Values stored in the cells bound to array and pointer names were the machine addresses, measured in bytes, of the corresponding storage area. Therefore, indirection through a pointer implied no run-time overhead to scale the pointer from word to byte offset. On the other hand, the machine code for array subscripting and pointer arithmetic now depended on the type of the array or the pointer: to compute iarray[i] or ipointer+i implied scaling the addend i by the size of the object referred to.
These semantics represented an easy transition from B, and I experimented with them for some months. Problems became evident when I tried to extend the type notation, especially to add structured (record) types. Structures, it seemed, should map in an intuitive way onto memory in the machine, but in a structure containing an array, there was no good place to stash the pointer containing the base of the array, nor any convenient way to arrange that it be initialized. For example, the directory entries of early Unix systems might be described in C as
struct {
int inumber;
char name[14];
};
I wanted the structure not merely to characterize an abstract object but also to describe a collection of bits that might be read from a directory. Where could the compiler hide the pointer to name that the semantics demanded? Even if structures were thought of more abstractly, and the space for pointers could be hidden somehow, how could I handle the technical problem of properly initializing these pointers when allocating a complicated object, perhaps one that specified structures containing arrays containing structures to arbitrary depth?
The solution constituted the crucial jump in the evolutionary chain between typeless BCPL and typed C. It eliminated the materialization of the pointer in storage, and instead caused the creation of the pointer when the array name is mentioned in an expression. The rule, which survives in today's C, is that values of array type are converted, when they appear in expressions, into pointers to the first of the objects making up the array.
This invention enabled most existing B code to continue to work, despite the underlying shift in the language's semantics. The few programs that assigned new values to an array name to adjust its origin—possible in B and BCPL, meaningless in C—were easily repaired. More important, the new language retained a coherent and workable (if unusual) explanation of the semantics of arrays, while opening the way to a more comprehensive type structure.
It's a good article, and well worth reading if you're interested in the "whys" of C.