Double precision strange behaviour. Need an explanation - c

Question

Here's the code:
#include <stdio.h>
#include <math.h>
static double const x = 665857;
static double const y = 470832;
int main(){
double z = x*x*x*x -(y*y*y*y*4+y*y*4);
printf("%f \n",z);
return 0;
}
Mysteriously (to me) this code prints "0.0" if compiled on 32 bits machines (or with the -m32 flag on 64 bits machines like in my case) with GCC 4.6. As far as I know about floating point operations, it is possible to overflow/underflow them or to lose precision with them, but... a 0? How?
Thanks in advance.

Answer 1

The problem is not that the numbers overflow. The problem is that doubles don't have enough precision to distinguish between the two operands of your subtraction.
The value of x*x*x*x is 196573006004558194713601.
The value of y*y*y*y*4+y*y*4 is 196573006004558194713600.
These numbers have 78 bits, and only the last bit is different. Double precision numbers only have 53 bits. Other numbers are rounded to only 53 bits.
In your case, the two operands are rounded to the same number, and so their difference is 0.
Even stranger things happen if you slightly rewrite your expression for z:
double z = x * x * x * x - ((y * y + 1) * y * y * 4);
With this change, you get 33554432! Why? Because the way intermediate results were rounded caused the last bit of the right operand to be different. The value of the last bit is 2^(78-53)=2^25.

Answer 2

Evaluating the expression with arbitrary precision integers:
Prelude> 665857^4 - 4*(470832^4 + 470832^2)
1
Since a double normally only has 53 bits of precision and the intermediate results have 78 bits, the precision isn't sufficient to calculate the result exactly, hence it is rounded, the last bits are forgotten at some point.

Answer 3

There is no floating-point overflow or underflow in your code. The two quantities are of the order of 1.96573006 × 10^23, and largely fit within a double. Your example simply illustrates catastrophic cancellation, where you subtract two close quantities and the relative precision of the result becomes horrible.
See http://en.wikipedia.org/wiki/Loss_of_significance

Answer 4

This is result of the way IEEE 754 represents floating point numbers in normalized form. float or double or whatever other IEEE 754 compliant representation is stored like:
1.xxxxxxxxxxxxxxxxxxx * 2^exp
where xxxxxxxxxxxxxxxxxxx is the fractional part of the mantissa so the mantissa itself is always in the range [1, 2). The integer part which is always 1 is not stored in the representation. The number of x bits defines the precision. It is 52 bits for the double. The exponent is stored in an offset form (one must subtract 1023 in order to obtain its value) but that is irrelevant now.
665857^4 in 64-bit IEEE 754 is:
0 10001001100 (1)0100110100000001111100111011101010000101110010100010
+ exponent mantissa
(the first bit is the sign bit: 0 = positive, 1 - negative; the bit in parentheses is not really stored)
In 80-bit x86 extended precision it is:
0 10001001100 (1)0100110100000001111100111011101010000101110010100010
0 100000001001100 1 010011010000000111110011101110101000010111001010000111000111011
(here the integer part is explicitly part of the representation - a deviation from IEEE 754; I've aligned the mantissas for clarity)
4*470832^4 in 64-bit IEEE 754 and 80-bit x86 extended precision is:
0 10001001100 (1)0100110100000001111100111011101001111111010101100111
0 100000001001100 1 010011010000000111110011101110100111111101010110011100100010000
4*470832^2 in 64-bit IEEE 754 and 80-bit x86 extended precision is:
0 10000100110 (1)1001110011101010100101010100100000000000000000000000
0 100000000100110 1 100111001110101010010101010010000000000000000000000000000000000
When you sum up the last two numbers, the procedure is the following: the smaller value has its exponent adjusted to match the larger value's exponent while the mantissa is shifted to the right in order to preserve the value. Since the two exponents differ by 38, the mantissa of the smaller number is shifted 38 bits to the right:
470832^2*4 in adjusted 64-bit IEEE 754 and 80-bit x86 extended precision:
this bit came from 1.xxxx ------------------------------v
0 10001001100 (0)0000000000000000000000000000000000000110011100111010|1010
0 100000001001100 0 0000000000000000000000000000000000000110011100111010101001010101
Now both quantities have the same exponents and their mantissas could be summed:
0 10001001100 (1)0100110100000001111100111011101001111111010101100111|0010
0 10001001100 (0)0000000000000000000000000000000000000110011100111010|1010
--------------------------------------------------------------------------
0 10001001100 (1)0100110100000001111100111011101010000101110010100001|1100
I kept some of the 80-bit precision bits on the right of the bar, because the summation internally is done in the greater precision of 80 bits.
Now let's perform the subtraction in 64-bit + some bits of the 80-bit rep:
minuend 0 10001001100 (1)0100110100000001111100111011101010000101110010100001|1100
subtrahend 0 10001001100 (1)0100110100000001111100111011101010000101110010100001|1100
-------------------------------------------------------------------------------------
difference 0 10001001100 (0)0000000000000000000000000000000000000000000000000000|0000
A pure 0! If you perform the calculations in full 80-bit, you would once again obtain a pure 0.
The real problem here is that 1.0 cannot be represented in 64-bit precision with an exponent of 2^77 - there are no 77 bits of precision in the mantissa. This is also true for the 80-bit precision - there are only 63 bits in the mantissa, 14 bits less than necessary to represent 1.0 given an exponent of 2^77.
So that's it! It's just the wonderful world of scientific computing where nothing works the way you were taught in the math classes...