Is there a way to force gcc to detect all undefined behavior? I want it to detect both things that can be discovered at compile time and runtime. I know that UB is useful both for making it simpler to create the compilers and to allow the compiler to optimize the code. The latter is not relevant when you're debugging, and the need of lightweight compilers is not as big as it was 1972. Furthermore, gcc is a very mature compiler at this point, and if this was possible, it would bake debugging so much easier.
I know that -Wformat will yield a warning for printf("%d", 42) and for uninitialized variables. The parameter -Warray-bounds might catch when you try to access memory outside an array, although I needed to put some work in constructing code that actually yielded a warning. I also know that some runtime errors can be detected with -fstack-protector-all.
So my question is simply this. Is there a way to guarantee that all UB gets detected, either at compilation if possible, but at the very latest when it happens in runtime?
This is impossible. Detecting undefined behavior can literally require solving the halting problem; for example, quoting C11 6.8.5:
6 An iteration statement whose controlling expression is not a constant expression, that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.
C is not designed to make error detection easy.
That's in principle impossible. Consider that some UB can depend on runtime data in very complex ways.
If you ask your user to input a value at runtime and then use that value as a pointer (or to compute a pointer) which you dereference and write through, how do you detect the write will cause UB or not? You can check the process image and see if the write will cause a segfault right away, but if it doesn't how do you detect that the write wasn't in a place that will cause a butterfly effect that will ultimately lead to a segfault or execution of unintended code?
It doesn't have to be about pointers either. You can parse faster if you assume all input is well formed (no error checking on malformed input), but if you then parse a malformed file, anything can happen just as with the pointer example.
Imagine this example (assume we have some arbitrary-precision BigInteger class and a function random_big_int that returns the positive integer n with probability 1/2^n)
void compute_collatz(BigInteger x) {
while (x != 1) {
if (x % 2) {
x = 3*x + 1;
} else {
x = x / 2;
}
}
std::cout << "Terminated successfully!" << std::endl;
}
int main() {
BigInteger x = random_big_int();
compute_collatz(x);
}
If the Collatz conjecture is false, this may enter a side-effect-free infinite loop (if a random integer is picked for which the conjecture is false), which is undefined behavior.
So, in order to tell whether this can invoke UB, the compiler would need to know whether the Collatz conjecture is true, which is an open problem in mathematics.
Related
The following code produces strange things on my system:
#include <stdio.h>
void f (int x) {
int y = x + x;
int v = !y;
if (x == (1 << 31))
printf ("y: %d, !y: %d\n", y, !y);
}
int main () {
f (1 << 31);
return 0;
}
Compiled with -O1, this prints y: 0, !y: 0.
Now beyond the puzzling fact that removing the int v or the if lines produces the expected result, I'm not comfortable with undefined behavior of overflows translating to logical inconsistency.
Should this be considered a bug, or is the GCC team philosophy that one unexpected behavior can cascade into logical contradiction?
When invoking undefined behavior, anything can happen. There's a reason why it's called undefined behavior, after all.
Should this be considered a bug, or is the GCC team philosophy that one unexpected behavior can cascade into logical contradiction?
It's not a bug. I don't know much about the philosophy of the GCC team, but in general undefined behavior is "useful" to compiler developers to implement certain optimizations: assuming something will never happen makes it easier to optimize code. The reason why anything can happen after UB is exactly because of this. The compiler makes a lot of assumptions and if any of them is broken then the emitted code cannot be trusted.
As I said in another answer of mine:
Undefined behavior means that anything can happen. There is no explanation as to why anything strange happens after invoking undefined behavior, nor there needs to be. The compiler could very well emit 16-bit Real Mode x86 assembly, produce a binary that deletes your entire home folder, emit the Apollo 11 Guidance Computer assembly code, or whatever else. It is not a bug. It's perfectly conforming to the standard.
The 2018 C standard defines, in clause 3.4.3, paragraph 1, “undefined behavior” to be:
behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this document imposes no requirements
That is quite simple. There are no requirements from the standard. So, no, the standard does not require the behavior to be “consistent.” There is no requirement.
Furthermore, compilers, operating systems, and other things involved in building and running a program generally do not impose any requirement of “consistency” in the sense asked about in this question.
Addendum
Note that answers that say “anything can happen” are incorrect. The C standard only says that it imposes no requirements when there is behavior that it deems “undefined.” It does not nullify other requirements and has no authority to nullify them. Any specifications of compilers, operating systems, machine architectures, or consumer product laws; or laws of physics; laws of logic; or other constraints still apply. One situation where this matters is simply linking to software libraries not written in C: The C standard does not define what happens, but what does happen is still constrained by the other programming language(s) used and the specifications of the libraries, as well as the linker, operating system, and so on.
Marco Bonelli has given the reasons why such behaviour is allowed; I'd like to attempt an explanation of why it might be practical.
Optimising compilers, by definition, are expected to do various stuff in order to make programs run faster. They are allowed to delete unused code, unwrap loops, rearrange operations and so on.
Taking your code, can the compiler be really expected to perform the !y operation strictly before the call to printf()? I'd say if you impose such rules, there'll be no place left for any optimisations. So, a compiler should be free to rewrite the code as
void f (int x) {
int y = x + x;
int notY = !(x + x);
if (x == (1 << 31))
printf ("y: %d, !y: %d\n", y, notY);
}
Now, it should be obvious that for any inputs which don't cause overflow the behaviour would be identical. However, in the case of overflow y and notY experience the effects of UB independently, and may both become 0 because why not.
For some reason, a myth has emerged that the authors of the Standard used the phrase "Undefined Behavior" to describe actions which earlier descriptions of the language by its inventor characterized as "machine dependent" was to allow compilers to infer that various things wouldn't happen. While it is true that the Standard doesn't require that implementations process such actions meaningfully even on platforms where there would be a natural "machine-dependent" behavior, the Standard also doesn't require that any implementation be capable of processing any useful programs meaningfully; an implementation could be conforming without being able to meaningfully process anything other than a single contrived and useless program. That's not a twisting of the Standard's intention: "While a deficient implementation could probably contrive
a program that meets this requirement, yet still succeed in being useless, the C89 Committee felt that such ingenuity would probably require more work than making something useful."
In discussing the decision to make short unsigned values promote to signed int, the authors of the Standard observed that most current implementations used quiet wraparound integer-overflow semantics, and having values promote to signed int would not adversely affect behavior if the value was used in overflow scenarios where the upper bits wouldn't matter.
From a practical perspective, guaranteeing clean wraparound semantics costs a little more than would allowing integer computations to behave as though performed on larger types at unspecified times. Even in the absence of "optimization", even straightforward code generation for an expression like long1 = int1*int2+long2; would on many platforms benefit from being able to use the result of a 16x16->32 or 32x32->64 multiply instruction directly, rather than having to sign-extend the lower half of the result. Further, allowing a compiler to evaluate x+1 as a type larger than x at its convenience would allow it to replace x+1 > y with x >= y--generally a useful and safe optimization.
Compilers like gcc go further, however. Even though the authors of the Standard observed that in the evaluation of something like:
unsigned mul(unsigned short x, unsigned short y) { return x*y; }
the Standard's decision to promote x and y to signed int wouldn't adversely affect behavior compared with using unsigned ("Both schemes give the same answer in the vast majority of cases, and both give the same effective result in even more cases in implementations with two’s-complement arithmetic and quiet wraparound on signed overflow—that is, in most current implementations."), gcc will sometimes use the above function to infer within calling code that x cannot possibly exceed INT_MAX/y. I've seen no evidence that the authors of the Standard anticipated such behavior, much less intended to encourage it. While the authors of gcc claim any code that would invoke overflow in such cases is "broken", I don't think the authors of the Standard would agree, since in discussing conformance, they note: "The goal is to give the programmer a fighting chance to make powerful C programs that are also highly portable, without seeming to demean perfectly useful C programs that happen not to be portable, thus the adverb strictly."
Because the authors of the Standard failed to forbid the authors of gcc from processing code nonsensically in case of integer overflow, even on quiet-wraparound platforms, they insist that they should jump the rails in such cases. No compiler writer who was trying to win paying customers would take such an attitude, but the authors of the Standard failed to realize that compiler writers might value cleverness over customer satisfaction.
Looking at this code:
static int global_var = 0;
int update_three(int val)
{
global_var = val;
return 3;
}
int main()
{
int arr[5];
arr[global_var] = update_three(2);
}
Which array entry gets updated? 0 or 2?
Is there a part in the specification of C that indicates the precedence of operation in this particular case?
Order of Left and Right Operands
To perform the assignment in arr[global_var] = update_three(2), the C implementation must evaluate the operands and, as a side effect, update the stored value of the left operand. C 2018 6.5.16 (which is about assignments) paragraph 3 tells us there is no sequencing in the left and right operands:
The evaluations of the operands are unsequenced.
This means the C implementation is free to compute the lvalue arr[global_var] first (by “computing the lvalue,” we mean figuring out what this expression refers to), then to evaluate update_three(2), and finally to assign the value of the latter to the former; or to evaluate update_three(2) first, then compute the lvalue, then assign the former to the latter; or to evaluate the lvalue and update_three(2) in some intermixed fashion and then assign the right value to the left lvalue.
In all cases, the assignment of the value to the lvalue must come last, because 6.5.16 3 also says:
… The side effect of updating the stored value of the left operand is sequenced after the value computations of the left and right operands…
Sequencing Violation
Some might ponder about undefined behavior due to both using global_var and separately updating it in violation of 6.5 2, which says:
If a side effect on a scalar object is unsequenced relative to either a different side effect on the same scalar object or a value computation using the value of the same scalar object, the behavior is undefined…
It is quite familiar to many C practitioners that the behavior of expressions such as x + x++ is not defined by the C standard because they both use the value of x and separately modify it in the same expression without sequencing. However, in this case, we have a function call, which provides some sequencing. global_var is used in arr[global_var] and is updated in the function call update_three(2).
6.5.2.2 10 tells us there is a sequence point before the function is called:
There is a sequence point after the evaluations of the function designator and the actual arguments but before the actual call…
Inside the function, global_var = val; is a full expression, and so is the 3 in return 3;, per 6.8 4:
A full expression is an expression that is not part of another expression, nor part of a declarator or abstract declarator…
Then there is a sequence point between these two expressions, again per 6.8 4:
… There is a sequence point between the evaluation of a full expression and the evaluation of the next full expression to be evaluated.
Thus, the C implementation may evaluate arr[global_var] first and then do the function call, in which case there is a sequence point between them because there is one before the function call, or it may evaluate global_var = val; in the function call and then arr[global_var], in which case there is a sequence point between them because there is one after the full expression. So the behavior is unspecified—either of those two things may be evaluated first—but it is not undefined.
The result here is unspecified.
While the order of operations in an expression, which dictate how subexpressions are grouped, is well defined, the order of evaluation is not specified. In this case it means that either global_var could be read first or the call to update_three could happen first, but there’s no way to know which.
There is not undefined behavior here because a function call introduces a sequence point, as does every statement in the function including the one that modifies global_var.
To clarify, the C standard defines undefined behavior in section 3.4.3 as:
undefined behavior
behavior, upon use of a nonportable or erroneous program construct or
of erroneous data,for which this International Standard imposes no
requirements
and defines unspecified behavior in section 3.4.4 as:
unspecified behavior
use of an unspecified value, or other behavior where this
International Standard provides two or more possibilities and imposes
no further requirements on which is chosen in any instance
The standard states that the evaluation order of function arguments is unspecified, which in this case means that either arr[0] gets set to 3 or arr[2] gets set to 3.
I tried and I got the entry 0 updated.
However according to this question: will right hand side of an expression always evaluated first
The order of evaluation is unspecified and unsequenced.
So I think a code like this should be avoided.
As it makes little sense to emit code for an assignment before you have a value to assign, most C compilers will first emit code that calls the function and save the result somewhere (register, stack, etc.), then they will emit code that writes this value to its final destination and therefore they will read the global variable after it has been changed. Let us call this the "natural order", not defined by any standard but by pure logic.
Yet in the process of optimization, compilers will try to eliminate the intermediate step of temporarily storing the value somewhere and try to write the function result as directly as possible to the final destination and in that case, they often will have to read the index first, e.g. to a register, to be able to directly move the function result to the array. This may cause the global variable to be read before it was changed.
So this is basically undefined behavior with the very bad property that its quite likely that the result will be different, depending on if optimization is performed and how aggressive this optimization is. It's your task as a developer to resolve that issue by either coding:
int idx = global_var;
arr[idx] = update_three(2);
or coding:
int temp = update_three(2);
arr[global_var] = temp;
As a good rule of the thumb: Unless global variables are const (or they are not but you know that no code will ever change them as a side effect), you should never use them directly in code, as in a multi-threaded environment, even this can be undefined:
int result = global_var + (2 * global_var);
// Is not guaranteed to be equal to `3 * global_var`!
Since the compiler may read it twice and another thread can change the value in between the two reads. Yet, again, optimization would definitely cause the code to only read it once, so you may again have different results that now also depend on the timing of another thread. Thus you will have a lot less headache if you store global variables to a temporary stack variable before usage. Keep in mind if the compiler thinks this is safe, it will most likely optimize even that away and instead use the global variable directly, so in the end, it may make no difference in performance or memory use.
(Just in case someone asks why would anyone do x + 2 * x instead of 3 * x - on some CPUs addition is ultra-fast and so is multiplication by a power two as the compiler will turn these into bit shifts (2 * x == x << 1), yet multiplication with arbitrary numbers can be very slow, thus instead of multiplying by 3, you get much faster code by bit shifting x by 1 and adding x to the result - and even that trick is performed by modern compilers if you multiply by 3 and turn on aggressive optimization unless it's a modern target CPU where multiplication is equally fast as addition since then the trick would slow down the calculation.)
Global edit: sorry guys, I got all fired up and wrote a lot of nonsense. Just an old geezer ranting.
I wanted to believe C had been spared, but alas since C11 it has been brought on par with C++. Apparently, knowing what the compiler will do with side effects in expressions requires now to solve a little maths riddle involving a partial ordering of code sequences based on a "is located before the synchronization point of".
I happen to have designed and implemented a few critical real-time embedded systems back in the K&R days (including the controller of an electric car that could send people crashing into the nearest wall if the engine was not kept in check, a 10 tons industrial robot that could squash people to a pulp if not properly commanded, and a system layer that, though harmless, would have a few dozen processors suck their data bus dry with less than 1% system overhead).
I might be too senile or stupid to get the difference between undefined and unspecified, but I think I still have a pretty good idea of what concurrent execution and data access mean. In my arguably informed opinion, this obsession of the C++ and now C guys with their pet languages taking over synchronization issues is a costly pipe dream. Either you know what concurrent execution is, and you don't need any of these gizmos, or you don't, and you would do the world at large a favour not trying to mess with it.
All this truckload of eye-watering memory barrier abstractions is simply due to a temporary set of limitations of the multi-CPU cache systems, all of which can be safely encapsulated in common OS synchronization objects like, for instance, the mutexes and condition variables C++ offers.
The cost of this encapsulation is but a minute drop in performances compared with what a use of fine grained specific CPU instructions could achieve is some cases.
The volatile keyword (or a #pragma dont-mess-with-that-variable for all I, as a system programmer, care) would have been quite enough to tell the compiler to stop reordering memory accesses.
Optimal code can easily be produced with direct asm directives to sprinkle low level driver and OS code with ad hoc CPU specific instructions. Without an intimate knowledge of how the underlying hardware (cache system or bus interface) works, you're bound to write useless, inefficient or faulty code anyway.
A minute adjustment of the volatile keyword and Bob would have been everybody but the most hardboiled low level programers' uncle.
Instead of that, the usual gang of C++ maths freaks had a field day designing yet another incomprehensible abstraction, yielding to their typical tendency to design solutions looking for non existent problems and mistaking the definition of a programming language with the specs of a compiler.
Only this time the change required to deface a fundamental aspect of C too, since these "barriers" had to be generated even in low level C code to work properly. That, among other things, wrought havoc in the definition of expressions, with no explanation or justification whatsoever.
As a conclusion, the fact that a compiler could produce a consistent machine code from this absurd piece of C is only a distant consequence of the way C++ guys coped with potential inconsistencies of the cache systems of the late 2000s.
It made a terrible mess of one fundamental aspect of C (expression definition), so that the vast majority of C programmers - who don't give a damn about cache systems, and rightly so - is now forced to rely on gurus to explain the difference between a = b() + c() and a = b + c.
Trying to guess what will become of this unfortunate array is a net loss of time and efforts anyway. Regardless of what the compiler will make of it, this code is pathologically wrong. The only responsible thing to do with it is send it to the bin.
Conceptually, side effects can always be moved out of expressions, with the trivial effort of explicitly letting the modification occur before or after the evaluation, in a separate statement.
This kind of shitty code might have been justified in the 80's, when you could not expect a compiler to optimize anything. But now that compilers have long become more clever than most programmers, all that remains is a piece of shitty code.
I also fail to understand the importance of this undefined / unspecified debate. Either you can rely on the compiler to generate code with a consistent behaviour or you can't. Whether you call that undefined or unspecified seems like a moot point.
In my arguably informed opinion, C is already dangerous enough in its K&R state. A useful evolution would be to add common sense safety measures. For instance, making use of this advanced code analysis tool the specs force the compiler to implement to at least generate warnings about bonkers code, instead of silently generating a code potentially unreliable to the extreme.
But instead the guys decided, for instance, to define a fixed evaluation order in C++17. Now every software imbecile is actively incited to put side effects in his/her code on purpose, basking in the certainty that the new compilers will eagerly handle the obfuscation in a deterministic way.
K&R was one of the true marvels of the computing world. For twenty bucks you got a comprehensive specification of the language (I've seen single individuals write complete compilers just using this book), an excellent reference manual (the table of contents would usually point you within a couple of pages of the answer to your question), and a textbook that would teach you to use the language in a sensible way. Complete with rationales, examples and wise words of warning about the numerous ways you could abuse the language to do very, very stupid things.
Destroying that heritage for so little gain seems like a cruel waste to me. But again I might very well fail to see the point completely.
Maybe some kind soul could point me in the direction of an example of new C code that takes a significant advantage of these side effects?
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I've read a lot of articles talking about undefined behavior (UB), but all do talk about theory. I am wondering what could happen in practice, because the programs containing UB may actually run.
My questions relates to unix-like systems, not embedded systems.
I know that one should not write code that relies on undefined behavior. Please do not send answers like this:
Everything could happen
Daemons can fly out of your nose
Computer could jump and catch fire
Especially for the first one, it is not true. You obviously cannot get root by doing a signed integer overflow. I'm asking this for educational purpose only.
Question A)
Source
implementation-defined behavior: unspecified behavior where each implementation documents how the choice is made
Is the implementation the compiler?
Question B)
*"abc" = '\0';
For something else than a segfault to happen, do I need my system to be broken? What could actually happen even if it is not predictable? Could the first byte be set to zero ? What else, and how?
Question C)
int i = 0;
foo(i++, i++, i++);
This is UB because the order in which parameters are evaluated is undefined. Right. But, when the program runs, who decides in what order the parameters are evaluated: is is the compiler, the OS, or something else?
Question D)
Source
$ cat test.c
int main (void)
{
printf ("%d\n", (INT_MAX+1) < 0);
return 0;
}
$ cc test.c -o test
$ ./test
Formatting root partition, chomp chomp
According to other SO users, this is possible. How could this happen? Do I need a broken compiler?
Question E)
Use the same code as above. What could actually happen, except of the expression (INT_MAX+1) yielding a random value ?
Question F)
Does the GCC -fwrapv option defines the behavior of a signed integer overflow, or does it only make GCC assume that it will wrap around but it could in fact not wrap around at runtime?
Question G)
This one concerns embedded systems. Of course, if the PC jumps to an unexpected place, two outputs could be wired together and create a short-circuit (for example).
But, when executing code similar to this:
*"abc" = '\0';
Wouldn't the PC be vectored to the general exception handler? Or what am I missing?
In practice, most compilers use undefined behavior in either of the following ways:
Print a warning at compile time, to inform the user that he probably made a mistake
Infer properties on the values of variables and use those to simplify code
Perform unsafe optimizations as long as they only break the expected semantic of undefined behavior
Compilers are usually not designed to be malicious. The main reason to exploit undefined behavior is usually to get some performance benefit from it. But sometimes that can involve total dead code elimination.
A) Yes. The compiler should document what behavior he chose. But usually that is hard to predict or explain the consequences of UB.
B) If the string is actually instantiated in memory and is in a writable page (by default it will be in a read-only page), then its first character might become a null character. Most probably, the entire expression will be thrown out as dead-code because it is a temporary value that disappears out of the expression.
C) Usually, the order of evaluation is decided by the compiler. Here it might decide to transform it into a i += 3 (or a i = undef if it is being silly). The CPU could reorder instructions at run-time but preserve the order chosen by the compiler if it breaks the semantic of its instruction set (the compiler usually cannot forward the C semantic further down). An incrementation of a register cannot commute or be executed in parallel to an other incrementation of that same register.
D) You need a silly compiler that print "Formatting root partition, chomp chomp" when it detects undefined behavior. Most probably, it will print a warning at compile time, replace the expression by a constant of his choice and produce a binary that simply perform the print with that constant.
E) It is a syntactically correct program, so the compiler will certainly produce a "working" binary. That binary could in theory have the same behavior as any binary you could download on the internet and that you run. Most probably, you get a binary that exit straight away, or that print the aforementioned message and exit straight away.
F) It tells GCC to assume the signed integers wrap around in the C semantic using 2's complement semantic. It must therefore produce a binary that wrap around at run-time. That is rather easy because most architecture have that semantic anyway. The reason for C to have that an UB is so that compilers can assume a + 1 > a which is critical to prove that loops terminate and/or predict branches. That's why using signed integer as loop induction variable can lead to faster code, even though it is mapped to the exact same instructions in hardware.
G) Undefined behavior is undefined behavior. The produced binary could indeed run any instructions, including a jump to an unspecified place... or cleanly trigger an interruption. Most probably, your compiler will get rid of that unnecessary operation.
You obviously cannot get root by doing a signed integer overflow.
Why not?
If you assume that signed integer overflow can only yield some particular value, then you're unlikely to get root that way. But the thing about undefined behavior is that an optimizing compiler can assume that it doesn't happen, and generate code based on that assumption.
Operating systems have bugs. Exploiting those bugs can, among other things, invoke privilege escalation.
Suppose you use signed integer arithmetic to compute an index into an array. If the computation overflows, you could accidentally clobber some arbitrary chunk of memory outside the intended array. That could cause your program to do arbitrarily bad things.
If a bug can be exploited deliberately (and the existence of malware clearly indicates that that's possible), it's at least possible that it could be exploited accidentally.
Also, consider this simple contrived program:
#include <stdio.h>
#include <limits.h>
int main(void) {
int x = INT_MAX;
if (x < x + 1) {
puts("Code that gets root");
}
else {
puts("Code that doesn't get root");
}
}
On my system, it prints
Code that doesn't get root
when compiled with gcc -O0 or gcc -O1, and
Code that gets root
with gcc -O2 or gcc -O3.
I don't have concrete examples of signed integer overflow triggering a security flaw (and I wouldn't post such an example if I had one), but it's clearly possible.
Undefined behavior can in principle make your program do accidentally anything that a program starting with the same privileges could do deliberately. Unless you're using a bug-free operating system, that could include privilege escalation, erasing your hard drive, or sending a nasty e-mail message to your boss.
To my mind, the worst thing that can happen in the face of undefined behavior is something different tomorrow.
I enjoy programming, but I also enjoy finishing a program, and going on to work on something else. I do not delight in continuously tinkering with my already-written programs, to keep them working in the face of bugs they spontaneously develop as hardware, compilers, or other circumstances keep changing.
So when I write a program, it is not enough for it to work. It has to work for the right reasons. I have to know that it works, and that it will keep working next week and next month and next year. It can't just seem to work, to have given apparently correct answers on the -- necessarily finite -- set of test cases I've run it on so far.
And that's why undefined behavior is so pernicious: it might do something perfectly fine today, and then do something completely different tomorrow, when I'm not around to defend it. The behavior might change because someone ran it on a slightly different machine, or with more or less memory, or on a very different set of inputs, or after recompiling it with a different compiler.
See also the third part of this other answer (the part starting with "And now, one more thing, if you're still with me").
It used to be that you could count on the compiler to do something "reasonable". More and more often, though, compilers are truly taking advantage of their license to do weird things when you write undefined code. In the name of efficiency, these compilers are introducing very strange optimizations, which don't do anything close to what you probably want.
Read these posts:
Linus Torvalds describes a kernel bug that was much worse than it could have been given that gcc took advantage of undefined behavior
LLVM blog post on undefined behavior (first of three parts, also two, three)
another great blog post by John Regehr (also first of three parts: two, three)
They say that when having UB, a program may do whatever it wants.
But if I have UB in one statement, such as
signed char a = 0x40;
a <<= 2;
or maybe even an unused(!) zero-size variable length array:
int l = 0;
char data[l];
is this in any way tolerable as only the result is undefined, or is this "bad" nevertheless?
I am especially interested in situations like these:
signed char a = 0x40;
a <<= 2;
switch (state) {
case X: return
case Y: do something with a; break;
case Z: do something else with a; break;
}
Assume that case X covers the case where the value of a is undefined, and the other cases make use of this case. It would simplify things if I was allowed to calculate this the way I want and make the distinction later.
Another situation is the one I talked about the other day:
void send_stuff()
{
char data[4 * !!flag1 + 2 * !!flag2];
uint8_t cursor = 0;
if (flag1) {
// fill 4 bytes of data into &data[cursor]
cursor += 4;
}
if (flag2) {
// fill 2 bytes of data into &data[cursor]
cursor += 2;
}
for (int i=0; i < cursor; i++) {
send_byte(data[i])
}
}
If both flags are unset, I have the "undefined" array data with length 0. But as I don't read from nor write to it, I don't see why and how it can possibly hurt...
Undefined behaviour means that it isn't defined by the C specification.
It may very well be defined (or partially defined) for a specific compiler.
Most compilers define a behavior for unsigned shift.
Most compilers define whether zero-length arrays are allowed.
Sometimes you can change the bahaviour with compiler flags, like --pedantic or flags that treat all warnings as errors.
So the answer to your question is:
That depends on the compiler. You need to check the documentation for your particular compiler.
Is it OK to rely on a specific result when you use something that is
UB according to the C standard?
That depends on what you are coding. If it is code for a specific embedded system where the likelyhood of ever porting to anywhere else is low, then by all means, rely on UB if it gives a big return. But best practice is to avoid UB when possible.
Edit:
is this in any way tolerable as only the result is undefined, or is this "bad" nevertheless?
Yes (only the result is undefined is true in practice, but in theory, the compiler manufacturer can terminate the program without breaking the C spec) and yes, it is bad nevertheless (because it requires additional tests to ensure that the behaviour remains the same after a change is made).
If the behaviour is unspecified, then you can observe what behaviour you get. Best is if you check the assembly code generated.
You need to be aware that the behaviour can change if you change anything, though. Changes that may change the behaviour include, but is not limited to, changes to the optimization level, and the application of compiler upgrades or patches.
The people who write the compilers are generally rational people which means that in most cases the program will behave in the way that was easiest for the compiler developer.
Best practice is still to avoid UB when possible.
You are confusing undefined and implementation defined behaviour.
Shifting a value by more bits than it has is implementation defined. It will have an effect and you need to read your compiler documentation. On some architectures, for instance, it it might have no effect, on others you'll be left with zero. Implementation defined behaviour isn't portable between architecture or compiler versions, requires no diagnostics, but will be consistent between runs.
However, declaring an array with a size of 0 is undefined behaviour. The compiler is free to make optimisations based on you not doing something like that, and produce code that doesn't work when you do. The compiler is free to do anything it likes if you invoke undefined behaviour, and it's possible your program will work today, and not tomorrow, or will work until you add another line somewhere else in the program or ....
Undefined means - not defined. There is no mileage in trying to work out how it's going to behave or depending on the results of said behavior.
In the context of your send_stuff function, the compiler is free to optimize your computation of cursor:
uint8_t cursor = flag1 ? 4 + 2 * !!flag2 : 2;
While this gives a different result for cursor when both flag1 and flag2 are 0, that's fine because that would result in undefined behavior anyway, so it's allowed to do whatever it wants in that case.
This is a completely machine-independent optimization, so even if you "know" that you'll always run on the same architecture with the same compiler, you can find that one day you make a seemingly unrelated change, the compiler gets tipped into a different decision about what optimal code looks like and your previously working code is suddenly behaving differently.
In C++11 is it Undefined Behavior, but is it the case in C that while(1); is Undefined Behavior?
It is well defined behavior. In C11 a new clause 6.8.5 ad 6 has been added
An iteration statement whose controlling expression is not a constant expression,156) that performs no input/output operations, does not access volatile objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.157)
157)This is intended to allow compiler transformations such as removal of empty loops even when termination cannot be proven.
Since the controlling expression of your loop is a constant, the compiler may not assume the loop terminates. This is intended for reactive programs that should run forever, like an operating system.
However for the following loop the behavior is unclear
a = 1; while(a);
In effect a compiler may or may not remove this loop, resulting in a program that may terminate or may not terminate. That is not really undefined, as it is not allowed to erase your hard disk, but it is a construction to avoid.
There is however another snag, consider the following code:
a = 1; while(a) while(1);
Now since the compiler may assume the outer loop terminates, the inner loop should also terminate, how else could the outer loop terminate. So if you have a really smart compiler then a while(1); loop that should not terminate has to have such non-terminating loops around it all the way up to main. If you really want the infinite loop, you'd better read or write some volatile variable in it.
Why this clause is not practical
It is very unlikely our compiler company is ever going to make use of this clause, mainly because it is a very syntactical property. In the intermediate representation (IR), the difference between the constant and the variable in the above examples is easily lost through constant propagation.
The intention of the clause is to allow compiler writers to apply desirable transformations like the following. Consider a not so uncommon loop:
int f(unsigned int n, int *a)
{ unsigned int i;
int s;
s = 0;
for (i = 10U; i <= n; i++)
{
s += a[i];
}
return s;
}
For architectural reasons (for example hardware loops) we would like to transform this code to:
int f(unsigned int n, int *a)
{ unsigned int i;
int s;
s = 0;
for (i = 0; i < n-9; i++)
{
s += a[i+10];
}
return s;
}
Without clause 6.8.5 ad 6 this is not possible, because if n equals UINT_MAX, the loop may not terminate. Nevertheless it is pretty clear to a human that this is not the intention of the writer of this code. Clause 6.8.5 ad 6 now allows this transformation. However the way this is achieved is not very practical for a compiler writer as the syntactical requirement of an infinite loop is hard to maintain on the IR.
Note that it is essential that n and i are unsigned as overflow on signed int gives undefined behavior and thus the transformation can be justified for this reason. Efficient code however benefits from using unsigned, apart from the bigger positive range.
An alternative approach
Our approach would be that the code writer has to express his intention by for example inserting an assert(n < UINT_MAX) before the loop or some Frama-C like guarantee. This way the compiler can "prove" termination and doesn't have to rely on clause 6.8.5 ad 6.
P.S: I'm looking at a draft of April 12, 2011 as paxdiablo is clearly looking at a different version, maybe his version is newer. In his quote the element of constant expression is not mentioned.
After checking in the draft C99 standard, I would say "no", it's not undefined. I can't find any language in the draft that mentions a requirement that iterations end.
The full text of the paragraph describing the semantics of the iterating statements is:
An iteration statement causes a statement called the loop body
to be executed repeatedly until the controlling expression compares equal to 0.
I would expect any limitation such as the one specififed for C++11 to appear there, if applicable. There is also a section named "Constraints", which also doesn't mention any such constraint.
Of course, the actual standard might say something else, although I doubt it.
The simplest answer involves a quote from §5.1.2.3p6, which states the minimal requirements of a conforming implementation:
The least requirements on a conforming implementation are:
— Accesses to volatile objects are evaluated strictly according to the
rules of the abstract machine.
— At program termination, all data written into files shall be
identical to the result that execution of the program according to the
abstract semantics would have produced.
— The input and output dynamics of interactive devices shall take
place as specified in 7.21.3. The intent of these requirements is that
unbuffered or line-buffered output appear as soon as possible, to
ensure that prompting messages actually appear prior to a program
waiting for input.
This is the observable behavior of the program.
If the machine code fails to produce the observable behaviour due to optimisations performed, then the compiler isn't a C compiler. What is the observable behaviour of a program that contains only such an infinite loop, at the point of termination? The only way such a loop could end is by a signal causing it to end prematurely. In the case of SIGTERM, the program terminates. This would cause no observable behaviour. Hence, the only valid optimisation of that program is the compiler pre-empting the system closing the program and generating a program that ends immediately.
/* unoptimised version */
int main() {
for (;;);
puts("The loop has ended");
}
/* optimised version */
int main() { }
One possibility is that a signal is raised and longjmp is called to cause execution to jump to a different location. It seems like the only place that could be jumped to is somewhere reached during execution prior to the loop, so providing the compiler is intelligent enough to notice that a signal is raised causing the execution to jump to somewhere else, it could potentially optimise the loop (and the signal raising) away in favour of jumping immediately.
When multiple threads enter the equation, a valid implementation might be able to transfer ownership of the program from the main thread to a different thread, and end the main thread. The observable behaviour of the program must still be observable, regardless of optimisations.
The following statement appears in C11 6.8.5 Iteration statements /6:
An iteration statement whose controlling expression is not a constant expression, that performs no input/output operations, does not access volatile
objects, and performs no synchronization or atomic operations in its body, controlling expression, or (in the case of a for statement) its expression-3, may be assumed by the implementation to terminate.
Since while(1); uses a constant expression, the implementation is not allowed to assume it will terminate.
A compiler is free to remove such a loop entirely is the expression is non-constant and all other conditions are similarly met, even if it cannot be proven conclusively that the loop would terminate.