The following simple code segfaults under gcc 4.4.4
#include<stdio.h>
typedef struct Foo Foo;
struct Foo {
char f[25];
};
Foo foo(){
Foo f = {"Hello, World!"};
return f;
}
int main(){
printf("%s\n", foo().f);
}
Changing the final line to
Foo f = foo(); printf("%s\n", f.f);
Works fine. Both versions work when compiled with -std=c99. Am I simply invoking undefined behavior, or has something in the standard changed, which permits the code to work under C99? Why does is crash under C89?
I believe the behavior is undefined both in C89/C90 and in C99.
foo().f is an expression of array type, specifically char[25]. C99 6.3.2.1p3 says:
Except when it is the operand of the sizeof operator or the unary
& operator, or is a string literal used to initialize an array, an
expression that has type "array of type" is converted to an
expression with type "pointer to type" that points to the initial
element of the array object and is not an lvalue. If the array object
has register storage class, the behavior is undefined.
The problem in this particular case (an array that's an element of a structure returned by a function) is that there is no "array object". Function results are returned by value, so the result of calling foo() is a value of type struct Foo, and foo().f is a value (not an lvalue) of type char[25].
This is, as far as I know, the only case in C (up to C99) where you can have a non-lvalue expression of array type. I'd say that the behavior of attempting to access it is undefined by omission, likely because the authors of the standard (understandably IMHO) didn't think of this case. You're likely to see different behaviors at different optimization settings.
The new 2011 C standard patches this corner case by inventing a new storage class. N1570 (the link is to a late pre-C11 draft) says in 6.2.4p8:
A non-lvalue expression with structure or union type, where the
structure or union contains a member with array type (including,
recursively, members of all contained structures and unions) refers to
an object with automatic storage duration and temporary lifetime.
Its lifetime begins when the expression is evaluated and its initial
value is the value of the expression. Its lifetime ends when the
evaluation of the containing full expression or full declarator ends.
Any attempt to modify an object with temporary lifetime results in
undefined behavior.
So the program's behavior is well defined in C11. Until you're able to get a C11-conforming compiler, though, your best bet is probably to store the result of the function in a local object (assuming your goal is working code rather than breaking compilers):
[...]
int main(void ) {
struct Foo temp = foo();
printf("%s\n", temp.f);
}
printf is a bit funny, because it's one of those functions that takes varargs. So let's break it down by writing a helper function bar. We'll return to printf later.
(I'm using "gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3")
void bar(const char *t) {
printf("bar: %s\n", t);
}
and calling that instead:
bar(foo().f); // error: invalid use of non-lvalue array
OK, that gives an error. In C and C++, you are not allowed to pass an array by value. You can work around this limitation by putting the array inside a struct, for example void bar2(Foo f) {...}
But we're not using that workaround - we're not allowed to pass in the array by value. Now, you might think it should decay to a char*, allowing you to pass the array by reference. But decay only works if the array has an address (i.e. is an lvalue). But temporaries, such as the return values from function, live in a magic land where they don't have an address. Therefore you can't take the address & of a temporary. In short, we're not allowed to take the address of a temporary, and hence it can't decay to a pointer. We are unable to pass it by value (because it's an array), nor by reference (because it's a temporary).
I found that the following code worked:
bar(&(foo().f[0]));
but to be honest I think that's suspect. Hasn't this broken the rules I just listed?
And just to be complete, this works perfectly as it should:
Foo f = foo();
bar(f.f);
The variable f is not a temporary and hence we can (implicitly, during decay) takes its address.
printf, 32-bit versus 64-bit, and weirdness
I promised to mention printf again. According to the above, it should refuse to pass foo().f to any function (including printf). But printf is funny because it's one of those vararg functions. gcc allowed itself to pass the array by value to the printf.
When I first compiled and ran the code, it was in 64-bit mode. I didn't see confirmation of my theory until I compiled in 32-bit (-m32 to gcc). Sure enough I got a segfault, as in the original question. (I had been getting some gibberish output, but no segfault, when in 64 bits).
I implemented my own my_printf (with the vararg nonsense) which printed the actual value of the char * before trying to print the letters pointed at by the char*. I called it like so:
my_printf("%s\n", f.f);
my_printf("%s\n", foo().f);
and this is the output I got (code on ideone):
arg = 0xffc14eb3 // my_printf("%s\n", f.f); // worked fine
string = Hello, World!
arg = 0x6c6c6548 // my_printf("%s\n", foo().f); // it's about to crash!
Segmentation fault
The first pointer value 0xffc14eb3 is correct (it points to the characters "Hello, world!"), but look at the second 0x6c6c6548. That's the ASCII codes for Hell (reverse order - little endianness or something like that). It has copied the array by value into printf and the first four bytes have been interpreted as a 32-bit pointer or integer. This pointer doesn't point anywhere sensible and hence the program crashes when it attempts to access that location.
I think this is in violation of the standard, simply by virtue of the fact that we're not supposed to be allowed to copy arrays by value.
On MacOS X 10.7.2, both GCC/LLVM 4.2.1 ('i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)') and GCC 4.6.1 (which I built) compile the code without warnings (under -Wall -Wextra), in both 32-bit and 64-bit modes. The programs all run without crashing. This is what I'd expect; the code looks fine to me.
Maybe the problem on Ubuntu is a bug in the specific version of GCC that has since been fixed?
Related
Why do GCC and Clang produce different output with this conforming C code:
int (puts) (); int (main) (main, puts) int main;
char *puts[(&puts) (&main["\0April 1"])]; <%%>
Neither compiler produces any warning or error even with -Wall -std=c18 -pedantic, but the program produces no output when built with GCC but prints the current date when built with Clang.
Why do GCC and Clang produce different output with this conforming C
code:
int (puts) (); int (main) (main, puts) int main;
char *puts[(&puts) (&main["\0April 1"])]; <%%>
In the first place, it is conforming code, though it does make use of a variable-length array, which is an optional language feature in C11 and C17. Some of the obfuscations are
use of the obscure digraphs <% and %>, which mean the same thing as { and }, respectively.
parenthesizing the function identifiers in function declarations
a forward declaration of function puts that is not a prototype
a K&R-style definition of function main
with a VLA parameter
whose dimension expression contains a function call
and a reference to another parameter
use of unconventional identifiers for the parameters to function main()
use of identifiers (puts and main) in declarations of an object and a function, respectively, with the same identifier
use of the identifier main for something more than the program's entry-point function
inversion of the conventional order of the operands of the indexing operator ([])
plus, indexing a sting literal
calling a function via an explicit function pointer constant expression
A string literal with an explicit null character within
Unconventional placement (and omission) of line breaks
A less obfuscated equivalent would be
int puts();
int main(
int argc,
char *argv[ puts("\0April 1" + argc) ]
) {
}
But the central question about the difference in behavior between the version compiled with GCC and the one built with Clang comes down to whether the expression for the size of the VLA function parameter is evaluated at runtime.
The language spec says that when a function parameter is declared with array type, its type is "adjusted" to the corresponding pointer type. That applies equally to complete, incomplete, and variable-length array types, but the spec does not explicitly say that the expression(s) for the dimension(s) are not evaluated. It does specify that expressions go unevaluated in certain other cases, and it even makes an exception to such a rule in the case of sizeof expressions involving VLAs, so the omission in this case could be interpreted as meaningful.
That makes a difference only for parameters of VLA type, because only for those can evaluation of the dimension expression(s) produce side effects on the machine state, including, but not limited to, observable program behavior.
GCC does not evaluate the VLA parameter's size expression at runtime, and I am inclined to take this as conforming to the intent of the standard. As a result, the GCC-compiled program does nothing but exit with status 0.
Clang does evaluate the VLA parameter's size expression at runtime. Although I disfavor this interpretation of the spec, I cannot rule it out. When it does evaluate the size expression, it uses the passed value of the first parameter. When the program is run without arguments, then the first parameter has value 1, with the result that the standard library's puts function is called with a pointer to the 'A' in "\0April 1".
int (puts) ();
int (main) (main, puts)
int main;
char *puts[(&puts) (&main["\0April 1"])];
{
}
Somebody's got a compiler bug; I'm just not sure who anymore. I don't understand why any compiler would emit code to evaluate the size parameter of a VLA as an argument.
The clang output is rather bizarre. For it to work, it would have had to find main in the function's scope but puts in the global scope despite having already encountered the declaration for puts. Normally, you can access a variable in its own declaration.
If somebody did this in production code my answer would be rather: "Stop using K&R function definitions."
I have code that looks like this :
Foo* create(args) {
Foo *t = malloc (sizeof (Foo)) ;
// Fill up fields in struct t from args.
return t;
}
The call is
Foo *created = create (args)
Note that the function and the call to the function are two separate modules.
The value of the pointer assigned to t on being malloced is slightly different to what is captured in created. Seemingly the MSB of the address is changed and replaced with fffff. The LSB portion is the same for around 6-7 characters.
I'm at a loss as to what's going on.
I'm using GCC 4.6
The most likely explanation one can come up with from what you provided is that at the point of the call the function create is undeclared. A permissive C compiler assumed that unknown function create returned an int and generated code, which effectively truncated the pointer value or, more precisely, sign-extended the assumed int return value into the MSB of the recipient pointer (assuming a platform where pointers are wider than int in terms of their bit-width, e.g. 64-bit platform).
Most likely the compiler issued a warning about an undeclared function being called, but the warning was ignored by the user.
Make sure declaration (or, better, prototype) of create is visible at the point of the call. If you are compiling in C99 mode (or later), ask your compiler to strictly enforce C99 requirements. And don't ignore compiler diagnostic messages.
Consider the following code:
void doesnt_modify(const int *);
int foo(int *n) {
*n = 42;
doesnt_modify(n);
return *n;
}
where the definition of doesnt_modify isn’t visible for the compiler. Thus, it must assume, that doesnt_modify changes the object n points to and must read *n before the return (the last line cannot be replaced by return 42;).
Assume, doesnt_modify doesn’t modify *n. I thought about the following to allow the optimization:
int foo_r(int *n) {
*n = 42;
{ /* New scope is important, I think. */
const int *restrict n_restr = n;
doesnt_modify(n_restr);
return *n_restr;
}
}
This has the drawback that the caller of doesnt_modify has to tell the compiler *n isn’t modified, rather than that the function itself could tell the compiler via its prototype. Simply restrict-qualifying the parameter to doesnt_modify in the declaration doesn’t suffice, cf. “Is top-level volatile or restrict significant [...]?”.
When compiling with gcc -std=c99 -O3 -S (or Clang with the same options), all functions are compiled to equivalent assembly, all re-reading the 42 from *n.
Would a compiler be allowed to do this optimization (replace the last line by return 42;) for foo_r? If not, is there a (portable, if possible) way to tell the compiler doesnt_modify doesn’t modify what its argument points to? Is there a way compilers do understand and make use of?
Does any function have UB (provided doesnt_modify doesn’t modify its argument’s pointee)?
Why I think, restrict could help here (From C11 (n1570) 6.7.3.1 “Formal definition of restrict”, p4 [emph. mine]):
[In this case, B is the inner block of foo_r, P is n_restr, T is const int, and X is the object denoted by *n, I think.]
During each execution of B, let L be any lvalue that has &L based on P. If L is used to access the value of the object X that it designates, and X is also modified (by any means), then the following requirements apply: T shall not be const-qualified. […]
$ clang --version
Ubuntu clang version 3.5.0-4ubuntu2 (tags/RELEASE_350/final) (based on LLVM 3.5.0)
Target: x86_64-pc-linux-gnu
Gcc version is 4.9.2, on an x86 32bit target.
Version 1 seems clearly specified by the formal definition of restrict (C11 6.7.3.1). For the following code:
const int *restrict P = n;
doesnt_modify(P);
return *P;
the symbols used in 6.7.3.1 are:
B - that block of code
P - the variable P
T - the type of *P which is const int
X - the (non-const) int being pointed to by P
L - the lvalue *P is what we're interested in
6.7.3.1/4 (partial):
During each execution of B, let L be any lvalue that has &L based on P. If L is used to access the value of the object X that it designates, and X is also modified (by any means), then the following requirements apply: T shall not be const-qualified
[...]
If these requirements are not met, then the behavior is undefined.
Note that T is const-qualified. Therefore, if X is modified in any way during this block (which includes during the call to a function in that block), the behaviour is undefined.
Therefore the compiler can optimize as if doesnt_modify did not modify X.
Version 2 is a bit more difficult for the compiler. 6.7.6.3/15 says that top-level qualifiers are not considered in prototype compatibility -- although they aren't ignored completely.
So although the prototype says:
void doesnt_modify2(const int *restrict p);
it could still be that the body of the function is declared as void doesnt_modify2(const int *p) and therefore might modify *p.
My conclusion is that if and only if the compiler can see the definition for doesnt_modify2 and confirm that p is declared restrict in the definition's parameter list then it would be able to perform the optimization.
Generally, restrict means that the pointer is not aliased (i.e. only it or a pointer derived from it can be used to access the pointed-to object).
With const, this means that the pointed-to object cannot be modified by well-formed code.
There is, however, nothing to stop the programmer breaking the rules using an explicit type conversion to remove the constness. Then the compiler (having been beaten into submission by the programmer) will permit an attempt to modify the pointed-to object without any complaint. This, strictly speaking, results in undefined behaviour so any result imaginable is then permitted including - possibly - modifying the pointed-to object.
If not, is there a (portable, if possible) way to tell the compiler doesnt_modify doesn’t modify what its argument points to?
No such way.
Compiler optimizers have difficulty optimizing when pointer and reference function parameters are involved. Because the implementation of that function can cast away constness compilers assume that T const* is as bad as T*.
Hence, in your example, after the call doesnt_modify(n) it must reload *n from memory.
See 2013 Keynote: Chandler Carruth: Optimizing the Emergent Structures of C++. It applies to C as well.
Adding restrict keyword here does not change the above.
Simultaneous use of a restrict qualifier on a pointer-type parameter and a const qualifier on its target type would invite a compiler to assume that no region of storage which is accessed during the lifetime of the pointer object via the pointer contained therein or any pointer derived from it, will be modified via any means during that pointer's lifetime. It generally says nothing whatsoever about regions of storage which are not accessed using the pointer in question.
The only situations where const restrict would have implications for an entire object would be those where pointer is declared using array syntax with a static bound. In that situation, behavior would only be defined in cases where the entire array object could be read (without invoking UB). Since reading any part of the array object which changes during function execution would invoke UB, code would be allowed to assume that no portion of the array can be changed in any fashion whatsoever.
Unfortunately, while a compiler that knew that a function's actual definition starts with:
void foo(int const thing[restrict static 1]);
would be entitled to assume that no part of *thing would be changed during the function's execution, even if the object might be one the function could otherwise access via pointer not derived from thing, the fact that a function's prototype includes such qualifiers would not compel its definition to do likewise.
In ISO C99, arrays that are not lvalues still decay to pointers, and may be subscripted, although they may not be modified or used after the next sequence point. (source)
I understand that this feature allows array indexing in cases where a function returning a structure containing an array, which is not allowed in C89 ( http://yarchive.net/comp/struct_return.html)
will you please help me understand why there is a restriction on using/modifying it after the next sequence point?
Note, the text OP quoted is from GCC documentation. Relevant text from C99 to back up that quote is:
C99 6.5.2.2/5
If an attempt is made to modify the result of a function call or to access it after the next
sequence point, the behavior is undefined.
and also from the list of changes in the Foreword:
conversion of array to pointer not limited to lvalues
I don't have the C89 text to compare, but the C99 description of array-to-pointer conversion (6.3.2.1/3) does not mention any restriction on the array being an lvalue. Also, the C99 section on subscripting (6.5.2.1/2) talks about the expression being subscripted as postfix expression, it does not mention lvalues either.
Consider this code:
struct foo
{
char buf[20];
};
struct foo foo(char const *p) { struct foo f; strcpy(f.buf, p); return f; }
int main()
{
char *hello = foo("hello").buf;
char *bye = foo("bye").buf;
// other stuff...
printf("%s\n", hello);
printf("%s\n", bye);
}
Where do the pointers hello and bye point to? The purpose of this clause is to say that the compiler does not have to keep all of the returned objects hanging around in memory somewhere in order to make those pointers remain valid indefinitely.
Instead, the hello is only valid up until the next ; in this case (or next sequence point in general). This leaves the compiler free to implement returning structs by value as a hidden pointer parameter, as Chris Torek describes in his excellent post, which can be "freed" at the end of the current statement.
NB. The C99 situation isn't quite as simple as described in Chris's post, as the following has to work:
printf("%s %s\n", foo("hello").buf, foo("bye").buf);
My install of gcc 4.8 seems to get it right though - that works with -std=c99, and segfaults with -std=c89.
The following simple code segfaults under gcc 4.4.4
#include<stdio.h>
typedef struct Foo Foo;
struct Foo {
char f[25];
};
Foo foo(){
Foo f = {"Hello, World!"};
return f;
}
int main(){
printf("%s\n", foo().f);
}
Changing the final line to
Foo f = foo(); printf("%s\n", f.f);
Works fine. Both versions work when compiled with -std=c99. Am I simply invoking undefined behavior, or has something in the standard changed, which permits the code to work under C99? Why does is crash under C89?
I believe the behavior is undefined both in C89/C90 and in C99.
foo().f is an expression of array type, specifically char[25]. C99 6.3.2.1p3 says:
Except when it is the operand of the sizeof operator or the unary
& operator, or is a string literal used to initialize an array, an
expression that has type "array of type" is converted to an
expression with type "pointer to type" that points to the initial
element of the array object and is not an lvalue. If the array object
has register storage class, the behavior is undefined.
The problem in this particular case (an array that's an element of a structure returned by a function) is that there is no "array object". Function results are returned by value, so the result of calling foo() is a value of type struct Foo, and foo().f is a value (not an lvalue) of type char[25].
This is, as far as I know, the only case in C (up to C99) where you can have a non-lvalue expression of array type. I'd say that the behavior of attempting to access it is undefined by omission, likely because the authors of the standard (understandably IMHO) didn't think of this case. You're likely to see different behaviors at different optimization settings.
The new 2011 C standard patches this corner case by inventing a new storage class. N1570 (the link is to a late pre-C11 draft) says in 6.2.4p8:
A non-lvalue expression with structure or union type, where the
structure or union contains a member with array type (including,
recursively, members of all contained structures and unions) refers to
an object with automatic storage duration and temporary lifetime.
Its lifetime begins when the expression is evaluated and its initial
value is the value of the expression. Its lifetime ends when the
evaluation of the containing full expression or full declarator ends.
Any attempt to modify an object with temporary lifetime results in
undefined behavior.
So the program's behavior is well defined in C11. Until you're able to get a C11-conforming compiler, though, your best bet is probably to store the result of the function in a local object (assuming your goal is working code rather than breaking compilers):
[...]
int main(void ) {
struct Foo temp = foo();
printf("%s\n", temp.f);
}
printf is a bit funny, because it's one of those functions that takes varargs. So let's break it down by writing a helper function bar. We'll return to printf later.
(I'm using "gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3")
void bar(const char *t) {
printf("bar: %s\n", t);
}
and calling that instead:
bar(foo().f); // error: invalid use of non-lvalue array
OK, that gives an error. In C and C++, you are not allowed to pass an array by value. You can work around this limitation by putting the array inside a struct, for example void bar2(Foo f) {...}
But we're not using that workaround - we're not allowed to pass in the array by value. Now, you might think it should decay to a char*, allowing you to pass the array by reference. But decay only works if the array has an address (i.e. is an lvalue). But temporaries, such as the return values from function, live in a magic land where they don't have an address. Therefore you can't take the address & of a temporary. In short, we're not allowed to take the address of a temporary, and hence it can't decay to a pointer. We are unable to pass it by value (because it's an array), nor by reference (because it's a temporary).
I found that the following code worked:
bar(&(foo().f[0]));
but to be honest I think that's suspect. Hasn't this broken the rules I just listed?
And just to be complete, this works perfectly as it should:
Foo f = foo();
bar(f.f);
The variable f is not a temporary and hence we can (implicitly, during decay) takes its address.
printf, 32-bit versus 64-bit, and weirdness
I promised to mention printf again. According to the above, it should refuse to pass foo().f to any function (including printf). But printf is funny because it's one of those vararg functions. gcc allowed itself to pass the array by value to the printf.
When I first compiled and ran the code, it was in 64-bit mode. I didn't see confirmation of my theory until I compiled in 32-bit (-m32 to gcc). Sure enough I got a segfault, as in the original question. (I had been getting some gibberish output, but no segfault, when in 64 bits).
I implemented my own my_printf (with the vararg nonsense) which printed the actual value of the char * before trying to print the letters pointed at by the char*. I called it like so:
my_printf("%s\n", f.f);
my_printf("%s\n", foo().f);
and this is the output I got (code on ideone):
arg = 0xffc14eb3 // my_printf("%s\n", f.f); // worked fine
string = Hello, World!
arg = 0x6c6c6548 // my_printf("%s\n", foo().f); // it's about to crash!
Segmentation fault
The first pointer value 0xffc14eb3 is correct (it points to the characters "Hello, world!"), but look at the second 0x6c6c6548. That's the ASCII codes for Hell (reverse order - little endianness or something like that). It has copied the array by value into printf and the first four bytes have been interpreted as a 32-bit pointer or integer. This pointer doesn't point anywhere sensible and hence the program crashes when it attempts to access that location.
I think this is in violation of the standard, simply by virtue of the fact that we're not supposed to be allowed to copy arrays by value.
On MacOS X 10.7.2, both GCC/LLVM 4.2.1 ('i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)') and GCC 4.6.1 (which I built) compile the code without warnings (under -Wall -Wextra), in both 32-bit and 64-bit modes. The programs all run without crashing. This is what I'd expect; the code looks fine to me.
Maybe the problem on Ubuntu is a bug in the specific version of GCC that has since been fixed?