Why these two snippets generate different value? - c

Question

When I run the following code:
#include <stdio.h>
int main()
{
int i = 0;
volatile long double sum = 0;
for (i = 1; i < 50; ++i) /* first snippet */
{
sum += (long double)1 / i;
}
printf("%.20Lf\n", sum);
sum = 0;
for (i = 49; i > 0; --i) /* second snippet */
{
sum += (long double)1 / i;
}
printf("%.20Lf", sum);
return 0;
}
The output is:
4.47920533832942346919
4.47920533832942524555
Shouldn't the two numbers be same?
And more interestingly, the following code:
#include <stdio.h>
int main()
{
int i = 0;
volatile long double sum = 0;
for (i = 1; i < 100; ++i) /* first snippet */
{
sum += (long double)1 / i;
}
printf("%.20Lf\n", sum);
sum = 0;
for (i = 99; i > 0; --i) /* second snippet */
{
sum += (long double)1 / i;
}
printf("%.20Lf", sum);
return 0;
}
produces:
5.17737751763962084084
5.17737751763962084084
So why are they different then and same now?

Answer 1

First, please correct your code. By C standard, %lf isn't principal for *printf ('l' is void, the data type remains double). To print long double, one should use %Lf. With your variant %lf, it's possible to get into a bug with improper format, cut-down value, etc. (You seem running 32-bit environment: in 64 bits, both Unix and Windows pass double in XMM registers, but long double otherwhere - stack for Unix, memory by pointer for Windows. On Windows/x86_64, you code will segfault because callee expects pointer. But, with Visual Studio, long double is AFAIK aliased to double, so you can remain ignorant of this change.)
Second, you can't be sure this code is not optimized by your C compiler to compile-time calculations (which can be done with more precision than default run-time one). To avoid such optimization, mark sum as volatile.
With these changes, your code shows:
At Linux/amd64, gcc4.8:
for 50:
4.47920533832942505776
4.47920533832942505820
for 100:
5.17737751763962026144
5.17737751763962025971
At FreeBSD/i386, gcc4.8, without precision setting or with explicit fpsetprec(FP_PD):
4.47920533832942346919
4.47920533832942524555
5.17737751763962084084
5.17737751763962084084
(the same as in your example);
but, the same test on FreeBSD with fpsetprec(FP_PE), which switches FPU to real long double operations:
4.47920533832942505776
4.47920533832942505820
5.17737751763962026144
5.17737751763962025971
identical to Linux case; so, in real long double, there is some real difference with 100 summands, and it is, in accordance with common sense, larger than for 50. But your platform defaults to rounding to double.
And, finally, in general, this is well-known effect of a finite precision and consequent rounding. For example, in this classical book, this misrounding of decreasing number series sum is explained in the very first chapters.
I am not really ready now to investigate source of results with 50 summands and rounding to double, why it shows such huge difference and why this difference is compensated with 100 summands. That needs much deeper investigation than I can afford now, but, I hope, this answer clearly shows you a next place to dig.
UPDATE: if it's Windows, you can manipulate FPU mode with _controlfp() and _controlfp_s(). In Linux, _FPU_SETCW does the same. This description elaborates some details and gives example code.
UPDATE2: using Kahan summation gives stable results in all cases. The following shows 4 values: ascending i, no KS; ascending i, KS; descending i, no KS; descending i, KS:
50 and FPU to double:
4.47920533832942346919 4.47920533832942524555
4.47920533832942524555 4.47920533832942524555
100 and FPU to double:
5.17737751763962084084 5.17737751763961995266
5.17737751763962084084 5.17737751763961995266
50 and FPU to long double:
4.47920533832942505776 4.47920533832942524555
4.47920533832942505820 4.47920533832942524555
100 and FPU to long double:
5.17737751763962026144 5.17737751763961995266
5.17737751763962025971 5.17737751763961995266
you can see difference disappeared, results are stable. I would assume this is nearly final point that can be added here :)