Round off error in C (Forward and Backward sum) - c

Question

#include <stdio.h>
#include <string.h>
#include <math.h>
#include <stdlib.h>
int main() {
/* Enter your code here. Read input from STDIN. Print output to STDOUT */
float sum=0.0;
for(int i=1;i<=1000000;i++)
sum+=(1.0)/i;
printf("Forward sum is %f ",sum);
sum=0.0;
for(int i=1000000;i>=1;i--)
sum+=(1.0)/i;
printf("Backward sum is %f ",sum);
return 0;
}
Output:-
Forward sum is :- 14.357358
Backward sum is :- 14.392652.
Why is there a difference in both sums ? I think that there is some precision error which is causing the difference in both the sums but I am not able to get a clear picture of why is this happening.

Answer 1

This is one of the surprising aspects of floating-point arithmetic: it actually matters what order you do things like addition in. (Formally, we say that floating-point addition is not commutative.)
It's pretty easy to see why this is the case, with a simpler, slightly artificial example. Let's say you have this addition problem:
1000000. + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1
But let's say that you're using a single-precision floating-point format that has only 7 digits of precision. So even though you might think that 1000000.0 + 0.1 would be 1000000.1, actually it would be rounded off to 1000000.. So 1000000.0 + 0.1 + 0.1 would also be 1000000., and adding in all 10 copies of 0.1 would still result in just 1000000., also.
But if instead you tried this:
0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 1000000.
Now, when you add 0.1 + 0.1, there's no problem with precision, so you get 0.2. So it you add 0.1 ten times, you get 1.0. So if you do the whole problem in that order, you'll get 1000001..
You can see this yourself. Try this program:
#include <stdio.h>
int main()
{
float f1 = 100000.0, f2 = 0.0;
int i;
for(i = 0; i < 10; i++) {
f1 += 0.1;
f2 += 0.1;
}
f2 += 100000.0;
printf("%.1f %.1f\n", f1, f2);
}
On my computer, this prints 100001.0 100001.0, as expected. But if I change the two big numbers to 10000000.0, then it prints 10000000.0 10000001.0. The two numbers are clearly unequal.

Answer 2

The first loop starts with adding relatively large parts to the sum, and decreasingly smaller parts when the sum gets larger. So while more bits are needed to represent the sum, less bits are available for the small parts.
In the second loop, small parts are added to the sum, and increasingly larger parts are added when the sum gets larger. So less bits are required to store the the newly added part relative to the current value of sum.
(Not a very scientific explanation, but i hope this verbal attempt makes the principle clear)
N.b.: it also means the second result is more accurate.
In an attempt to be more precise: in order to add two floating point numbers they need to be scaled to have the same number of bits for mantissa and exponent. When the sum gets larger, the item added to will be scaled so as not to loose significance of this sum. As a result, the least significant bits of the part to be added will be scaled out of the register before the addition. For example (hypothetical) adding 0.00000001 to 1,000,000,000 will result in adding zero to this large number.