I've seen in the ISO C99 committee draft that structs can have an incomplete an array with unspecified size its end, known as Flexible Array Member.
On the other hand C99 also has Variable Length Arrays, which allow declaring arrays with size not constant at compile-time.
I thought that a FAM was a special kind of a VLA, but I've seen two SO users claiming otherwise. Also, reading the Wikipedia section on sizeof, it says that sizeof behaves differently for those two.
Why do both of them exist instead of just one? (Are their use-cases too different?)
Also, which other associated behaviors are different for each of them?
There are two different things that the C99 standard added and they are easy to mix up by mistake:
Flexible array members. This means that a struct can have a member of unknown size at the end. Example from the C standard:
struct s { int n; double d[]; };
int m = /* some value */;
struct s *p = malloc(sizeof (struct s) + sizeof (double [m]));
This was used before C99 as well, but it was then undefined behavior, known as the "struct hack" referred to in another answer. Before C90, there could be unexpected padding bytes at the end of the struct, leading to bugs.
Variable length arrays (VLA). These are arrays with their size set in runtime. They are most likely implemented by the compiler by using dynamic memory allocation. Example:
void func (int n)
{
int array[n];
}
referred from user29079 : https://softwareengineering.stackexchange.com/questions/154089/c-flexible-arrays-when-did-they-become-part-of-the-standard
Related
The C standard states (emphasize mine):
21 EXAMPLE 2 After the declaration:
struct s { int n; double d[]; };
the structure struct s has a flexible array member d. [...]
22 Following the above declaration:
struct s t1 = { 0 }; // valid
struct s t2 = { 1, { 4.2 }}; // invalid
t1.n = 4; // valid
t1.d[0] = 4.2; // might be undefined behavior
The initialization of t2 is invalid (and violates a constraint) because struct s is treated as if it did not contain member d.
Source: C18, §6.7.2.1/20 + /21
I do not understand the explanation of "because struct s is treated as if it did not contain member d"
If I use the initializer of { 1, { 4.2 }};, the { 4.2 } part is to initialize the flexible array member;
To be precise to initialize the flexible array member to be consisted of one element and initialize this element to the value 4.2 and thus stuct s is treated as it has member d or not?
This sentence makes no sense in my eyes.
Why does the standard say, that { 4.2 } wouldn't initialize/denote the flexible array member and thus the structure would be treated as if it has no member d?
If I use a fixed size array, this notation works and initializes the member with no complain:
struct foo {
int x;
double y[1];
};
int main (void)
{
struct foo a = { 1, { 2.3 } };
}
Evidence
Why is this initialization invalid when the structure has an flexible array member but valid when the structure has an fixed size array member?
Could you elaborate that?
I've read:
Why does static initialization of flexible array member work?
and
How to initialize a structure with flexible array member
and
Flexible array members can lead to undefined behavior?
and others but none of them answers me what this sentence wants to explain and why exactly this this is invalid.
Related:
How does an array of structures with flexible array members behave?
What are the real benefits of flexible array member?
I guess this is a language defect. While it might make no sense to initialize a flexible array member, the standard needs to address that issue somewhere. I can't find such normative text anywhere.
The definition of a flexible array member is, C17 6.7.2.1/18:
As a special case, the last element of a structure with more than one named member may have an
incomplete array type; this is called a flexible array member. In most situations, the flexible array
member is ignored. In particular, the size of the structure is as if the flexible array member were
omitted except that it may have more trailing padding than the omission would imply.
From this we learn that a flexible array member is an incomplete array type. We do not however learn in what situations the flexible array member is ignored, save for when calculating the size of the struct. "In most situations" isn't helpful and is the defect - this needed to be expanded to an exhaustive list, including the behavior of flexible array members when part of an initializer list. Otherwise one may assume that it behaves just like any other array of incomplete type.
C17 6.2.5/22:
An array type of unknown size is an incomplete type.
And then the rules for initialization say, C17 6.7.9:
The type of the entity to be initialized shall be an array of unknown size or a complete object type that is not a variable length array type.
So far there is no normative text saying that we are not allowed to provide an initializer for a flexible array member - on the contrary. The example in the question (C17 6.7.2.1 example 21) is not normative, since examples aren't normative in ISO standards. The example doesn't mention which constraint that is violated, nor does it mention where it says that the flexible array member must be ignored.
I suppose I'd probably file a DR about this.
I do not understand the explanation of "because struct s is treated as if it did not contain member d".
The C standard also says “In most situations, the flexible array member is ignored.” It is unclear why you would not understand what the meaning of this is. If struct s is declared struct s { int n; double d[]; };, then, in most situations, the C implementation behaves as if it were declared struct s { int n; };. Therefore, struct s t2 = { 1, { 4.2 }}; fails because the 4.2 is an initializer for something that, in effect, does not exist.
It is sensible to ask why this is the situation. For the most part, I expect a compiler could support a definition in which the array initializers were counted and used to set the structure size. Certainly compilers do this with array definitions such s int a[] = { 3, 4, 5};. However, that is not the typical use case for flexible array members. Typically, a program receives information about how many elements it will need to manage with the structure, allocates space for the structure with space for those elements included, and then puts a structure in the allocated space. That is, the typical use case for structures with flexible array members is with dynamically allocated space. I expect the C committee saw little need to require compilers to support flexible array members in static or automatic objects, instead of dynamic objects.
You've omitted some important language in the example you quoted - here's the full text:
20 EXAMPLE 2 After the declaration:struct s { int n; double d[]; };
the structure struct s has a flexible array member d. A typical way to use this is:int m = /* some value */;
struct s *p = malloc(sizeof (struct s) + sizeof (double [m]));
and assuming that the call to malloc succeeds, the object pointed to by p behaves, for most purposes, as if
p had been declared as:struct { int n; double d[m]; } *p;
(there are circumstances in which this equivalence is broken; in particular, the offsets of member d might
not be the same).
IOW, flexible array members only really come into play if you allocate the struct instance dynamically and allocate additional space for the array member.
A flexible array member has no size, so it doesn't contribute to the size of the struct type - that is, the result of sizeof (struct s) evaluates to the size of the type without the array.
IMO it is because sizeof of the initialized this way struct cannot be determined in another compilation unit when the struct is declared as extern.
OK so I was reading the standard paper (ISO C11) in the part where it explains flexible array members (at 6.7.2.1 p18). It says this:
As a special case, the last element of a structure with more than one
named member may have an incomplete array type; this is called a
flexible array member. In most situations, the flexible array member
is ignored. In particular, the size of the structure is as if the
flexible array member were omitted except that it may have more
trailing padding than the omission would imply. However, when a . (or
->) operator has a left operand that is (a pointer to) a structure with a flexible array member and the right operand names that member,
it behaves as if that member were replaced with the longest array
(with the same element type) that would not make the structure larger
than the object being accessed; the offset of the array shall remain
that of the flexible array member, even if this would differ from that
of the replacement array. If this array would have no elements, it
behaves as if it had one element but the behavior is undefined if any
attempt is made to access that element or to generate a pointer one
past it.
And here are some of the examples given below (p20):
EXAMPLE 2 After the declaration:
struct s { int n; double d[]; };
the structure struct s has a flexible array member d. A typical way to
use this is:
int m = /* some value */;
struct s *p = malloc(sizeof (struct s) + sizeof (double [m]));
and assuming that the call to malloc succeeds, the object pointed to
by p behaves, for most purposes, as if p had been declared as:
struct { int n; double d[m]; } *p;
(there are circumstances in which this equivalence is broken; in
particular, the offsets of member d might not be the same).
Added spoilers as examples inside the standard are not documentation.
And now my example (extending the one from the standard):
#include <stdio.h>
#include <stdlib.h>
int main(void)
{
struct s { int n; double d[]; };
int m = 7;
struct s *p = malloc(sizeof (struct s) + sizeof (double [m])); //create our object
printf("%zu", sizeof(p->d)); //retrieve the size of the flexible array member
free(p); //free out object
}
Online example.
Now the compiler is complaining that p->d has incomplete type double[] which is clearly not the case according the standard paper. Is this a bug in the GCC compiler?
As a special case, the last element of a structure with more than one named member may have an incomplete array type; ... C11dr 6.7.2.1 18
In the following d is an incomplete type.
struct s { int n; double d[]; };
The sizeof operator shall not be applied to an expression that has function type or an incomplete type ... C11dr §6.5.3.4 1
// This does not change the type of field `m`.
// It (that is `d`) behaves like a `double d[m]`, but it is still an incomplete type.
struct s *p = foo();
// UB
printf("%zu", sizeof(p->d));
This looks like a defect in the Standard. We can see from the paper where flexible array members were standardized, N791 "Solving the struct hack problem", that the struct definition replacement is intended to apply only in evaluated context (to borrow the C++ terminology); my emphasis:
When an lvalue whose type is a structure
with a flexible array member is used to access an object, it behaves as
if that member were replaced by the longest array that would not make
the structure larger than the object being accessed.
Compare the eventual standard language:
[W]hen a . (or ->) operator has a left operand that is (a pointer to) a structure with a flexible array member and the right operand names that member, it behaves as if that member were replaced with the longest array (with the same
element type) that would not make the structure larger than the object being accessed [...]
Some form of language like "When a . (or ->) operator whose left operand is (a pointer to) a structure with a flexible array member and whose right operand names that member is evaluated [...]" would seem to work to fix it.
(Note that sizeof does not evaluate its argument, except for variable length arrays, which are another kettle of fish.)
There is no corresponding defect report visible via the JTC1/SC22/WG14 website. You might consider submitting a defect report via your ISO national member body, or asking your vendor to do so.
Standard says:
C11-§6.5.3.4/2
The sizeof operator yields the size (in bytes) of its operand, which may be an expression or the parenthesized name of a type. The size is determined from the type of the operand.
and it also says
C11-§6.5.3.4/1
The sizeof operator shall not be applied to an expression that has function type or an incomplete type, [...]
p->d is of incomplete type and it can't be an operand of sizeof operator. The statement
it behaves as if that member were replaced with the longest array (with the same element type) that would not make the structure larger than the object being accessed
doesn't hold for sizeof operator as it determine size of the object by the type of object which must be a complete type.
First, what is happening is correct in terms of the standard, arrays that are declared [] are incomplete and you can't use the sizeof operator.
But there is also a simple reason for it in your case. You never told your compiler that in that particular case the d member should be viewed as of a particular size. You only told malloc the total memory size to be reserved and placed p to point to that. The compiler has obtained no type information that could help him deduce the size of the array.
This is different from allocating a variable length array (VLA) or a pointer to VLA:
double (*q)[m] = malloc(sizeof(double[m]));
Here the compiler can know what type of array q is pointing to. But not because you told malloc the total size (that information is not returned from the malloc call) but because m is part of the type specification of q.
The C Standard is a bit loosey-goosey when it comes to the definition of certain terms in certain contexts. Given something like:
struct foo {uint32_t x; uint16_t y[]; };
char *p = 1024+(char*)malloc(1024); // Point to end of region
struct foo *q1 = (struct foo *)(p -= 512); // Allocate some space from it
... some code which uses *q1
struct foo *q2 = (struct foo *)(p -= 512); // Allocate more space from it
there's no really clear indication of what storage is occupied by objects
*q1 or *q2, nor by q1->y or q2->y. If *q1 will never be accessed afterward,
then q2->y may be treated as a uint16_t[509], but writing to *q1 will trash
the contents of q2->y[254] and above, and writing q2->y[254] and above will
trash *q1. Since a compiler will generally have no way of knowing what will
happen to *q1 in the future, it will have no way of sensibly reporting a size
for q2->y.
Why does C permit this:
typedef struct s
{
int arr[];
} s;
where the array arr has no size specified?
This is C99 feature called flexible arrays, the main feature is to allow the use variable length array like features inside a struct and R.. in this answer to another question on flexible array members provides a list of benefits to using flexible arrays over pointers. The draft C99 standard in section 6.7.2.1 Structure and union specifiers paragraph 16 says:
As a special case, the last element of a structure with more than one named member may
have an incomplete array type; this is called a flexible array member. In most situations,
the flexible array member is ignored. In particular, the size of the structure is as if the
flexible array member were omitted except that it may have more trailing padding than
the omission would imply. [...]
So if you had a s* you would allocate space for the array in addition to space required for the struct, usually you would have other members in the structure:
s *s1 = malloc( sizeof(struct s) + n*sizeof(int) ) ;
the draft standard actually has a instructive example in paragraph 17:
EXAMPLE After the declaration:
struct s { int n; double d[]; };
the structure struct s has a flexible array member d. A typical way to use this
is:
int m = /* some value */;
struct s *p = malloc(sizeof (struct s) + sizeof (double [m]));
and assuming that the call to malloc succeeds, the object pointed to by p
behaves, for most purposes, as if p had been declared as:
struct { int n; double d[m]; } *p;
(there are circumstances in which this equivalence is broken; in particular, the
offsets of member d might not be the same).
You are probably looking for flexible arrays in C99. Flexible array members are members of unknown size at the end of a struct/union.
As a special case, the last element of a structure with more than one
named member may have an incomplete array type; this is called a
flexible array member. In most situations, the flexible array member
is ignored. In particular, the size of the structure is as if the
flexible array member were omitted except that it may have more
trailing padding than the omission would imply.
You may also look at the reason for the struct hack in the first place.
It's not clear if it's legal or portable, but it is rather popular. An implementation of the technique might look something like this:
#include <stdlib.h>
#include <string.h>
struct name *makename(char *newname)
{
struct name *ret =
malloc(sizeof(struct name)-1 + strlen(newname)+1);
/* -1 for initial [1]; +1 for \0 */
if(ret != NULL) {
ret->namelen = strlen(newname);
strcpy(ret->namestr, newname);
}
return ret;
}
This function allocates an instance of the name structure with the
size adjusted so that the namestr field can hold the requested name
(not just one character, as the structure declaration would suggest).
Despite its popularity, the technique is also somewhat notorious -
Dennis Ritchie has called it "unwarranted chumminess with the C implementation." An official interpretation has deemed that it is NOT
strictly conforming with the C Standard, although it does seem to work
under all known implementations. Compilers that check array bounds
carefully might issue warnings.
I'm programming in C99 and use variable length arrays in one portion of my code. I know in C89 zero-length arrays are not allowed, but I'm unsure of C99 and variable length arrays.
In short, is the following well defined behavior?
int main()
{
int i = 0;
char array[i];
return 0;
}
No, zero-length arrays are explicitly prohibited by C language, even if they are created as VLA through a run-time size value (as in your code sample).
6.7.5.2 Array declarators
...
5 If the size is an expression that is not an integer constant
expression: if it occurs in a declaration at function prototype scope,
it is treated as if it were replaced by *; otherwise, each time it is
evaluated it shall have a value greater than zero.
Zero-length arrays are not allowed in C. Statically typed arrays must have a fixed, non-zero size that is a constant expression, and variable-length-arrays must have a size which evaluates non-zero; C11 6.7.6.2/5:
each time it [the size expression] is evaluated it shall have a value greater than zero
However, C99 and C11 have a notion of a flexible array member of a struct:
struct foo
{
int a;
int data[];
};
From C11, 6.7.21/18:
As a special case, the last element of a structure with more than one named member may
have an incomplete array type; this is called a flexible array member. In most situations,
the flexible array member is ignored. In particular, the size of the structure is as if the
flexible array member were omitted except that it may have more trailing padding than
the omission would imply. However, when a . (or ->) operator has a left operand that is
(a pointer to) a structure with a flexible array member and the right operand names that
member, it behaves as if that member were replaced with the longest array (with the same
element type) that would not make the structure larger than the object being accessed;
Zero-length arrays are not allowed in standard C(not even C99 or C11). But gcc does provide an extension to allow it. See http://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
struct line {
int length;
char contents[0];
};
struct line *thisline = (struct line *)
malloc (sizeof (struct line) + this_length);
thisline->length = this_length;
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Is the “struct hack” technically undefined behavior?
Normally accessing an array beyond its end is undefined behavior in C. For example:
int foo[1];
foo[5] = 1; //Undefined behavior
Is it still undefined behavior if I know that the memory area after the end of the array has been allocated, with malloc or on the stack? Here is an example:
#include <stdio.h>
#include <stdlib.h>
typedef struct
{
int len;
int data[1];
} MyStruct;
int main(void)
{
MyStruct *foo = malloc(sizeof(MyStruct) + sizeof(int) * 10);
foo->data[5] = 1;
}
I have seen this patten used in several places to make a variable length struct, and it seems to work in practice. Is it technically undefined behavior?
What you are describing is affectionately called "the struct hack". It's not clear if it's completely okay, but it was and is widely used.
As of late (C99), it has started to be replaced by the "flexible array member", where you're allowed to put an int data[]; field if it's the last field in the struct.
Under 6.5.6 Additive operators:
Semantics
8 - [...] If the pointer operand points to an element of an array object, and the array is large enough, the result points to an element offset from the original element such that the difference of the subscripts of the resulting and original array elements equals the integer expression. [...] If the result points one past the last element of the array object, it shall not be used as the operand of a unary * operator that is evaluated.
If the memory is allocated by malloc then:
7.22.3 Memory management functions
1 - [...] The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object with a fundamental alignment requirement and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated). The lifetime of an allocated object extends from the allocation until the deallocation.
This does not however countenance the use of such memory without an appropriate cast, so for MyStruct as defined above only the declared members of the object can be used. This is why flexible array members (6.7.2.1:18) were added.
Also note that appendix J.2 Undefined behavior calls out array access:
1 - The behavior is undefined in the following circumstances: [...]
— Addition or subtraction of a pointer into, or just beyond, an array object and an
integer type produces a result that does not point into, or just beyond, the same array
object.
— Addition or subtraction of a pointer into, or just beyond, an array object and an
integer type produces a result that points just beyond the array object and is used as
the operand of a unary * operator that is evaluated.
— An array subscript is out of range, even if an object is apparently accessible with the
given subscript (as in the lvalue expression a[1][7] given the declaration int
a[4][5]).
So, as you note this would be undefined behaviour:
MyStruct *foo = malloc(sizeof(MyStruct) + sizeof(int) * 10);
foo->data[5] = 1;
However, you would be allowed to do the following:
MyStruct *foo = malloc(sizeof(MyStruct) + sizeof(int) * 10);
((int *) foo)[(offsetof(MyStruct, data) / sizeof(int)) + 5] = 1;
C++ is laxer in this regard; 3.9.2 Compound types [basic.compound] has:
3 - [...] If an object of type T is located at an address A, a pointer of type cv T* whose value is the address A is said to point to that object, regardless of how the value was obtained.
This makes sense considered in the light of C's more aggressive optimisation opportunities for pointers, e.g. with the restrict qualifier.
The C99 rationale document talks about this in section 6.7.2.1.
A new feature of C99: There is a common idiom known as the “struct hack” for creating a structure containing a variable-size array:
...
The validity of this construct has always been questionable. In the response to one Defect Report, the Committee decided that it was undefined behavior because the array p->items contains only one item, irrespective of whether the space exists. An alternative construct was
suggested: make the array size larger than the largest possible case (for example, using int items[INT_MAX];), but this approach is also undefined for other reasons.
The Committee felt that, although there was no way to implement the “struct hack” in C89, it was nonetheless a useful facility. Therefore the new feature of “flexible array members” was introduced. Apart from the empty brackets, and the removal of the “-1” in the malloc call,
this is used in the same way as the struct hack, but is now explicitly valid code.
The struct hack is undefined behavior, as supported not only be the C specification itself (I'm sure there are citations in the other answers) but the committee has even recorded its opinion.
So the answer is yes, it is undefined behavior according to the standard document, but it is well defined according to the de facto C standard. I imagine most compiler writers are intimately familiar with the hack. From GCC's tree-vrp.c:
/* Accesses after the end of arrays of size 0 (gcc
extension) and 1 are likely intentional ("struct
hack"). */
I think there's a good chance you might even find the struct hack in compiler test suites.