I am experimenting with optimizing double->text conversion (trying to beat grissu, ryu etc...).
While doing so, I am comparing my results with sprintf outputs. Now I have encountered the above interesting case.
printf("%.15e", 1e23);
(e.g. glibc) prints
9.999999999999999e+22
while my routine prints
1.000000000000000e+23
Now both numbers are the same distance from the "true value" and converting both these values back (e.g. with atof) yields the same double.
However, I believe my result satisfies "round to even" rule (which is the reason it went this way).
Which result is more correct?
1e23 is typically not exactly represented as a double.
The 2 closest choices are:
// %a v %f
0x1.52d02c7e14af6p+76 99999999999999991611392.000000
0x1.52d02c7e14af7p+76 100000000000000008388608.000000
There are both 8388608.0 from 100000000000000000000000.0, one above, one below.
Typically the selected one in tie cases is the even one. (See last bit in hex format.)
99999999999999991611392.000000 is the winner and so an output of "9.999999999999999e+22" is expected.
When printing with more the DBL_DIG (15) significant digits, ("%.15e" prints 16 significant digits) this problem is possible as code is effectively doing a text-double-text round-trip and exceeding what double can round-trip.
I am experimenting with optimizing double->text conversion
I recommend to also use "%a" to gain deeper understanding.
Related
This came up during testing where I have to compare values between actual output and expected output.
Code:
float nf = 584227.4649743827f;
printf("Output: \t %.9f \n", nf);
Output:
Output: 584227.437500
I clearly have some gaps in my knowledge in C, so could someone explain to me this situation:
Why is there this deviation (0.027474382659420f) in the print ?
Is this only the limitation of print, or is it the float data type limitation?
which value is actually stored in the variable nf?
How should I work with values like this so I don't lose information like having a deviation of 0.027474382659420f during assignment.
Any other suggestion related to this kind of problem in testing would be also much appriciated.
Why is there this deviation (0.027474382659420f) in the print ?
Because float has an accuracy of about 7 digits, and your deviation starts at the seventh digit.
Is this only the limitation of print, or is it the float data type
limitation?
It's a limitation of floating point numbers, so that also includes double (although double has a higher precision). It also has to do with the conversion between binary and decimal numbers, so for instance 0.2 is a repeating decimal (well, binary) in binary representation, so it is suspect to rounding errors, too, and might actually turn into something like 0.200000000000000011.
which value is actually stored in the variable nf?
The one that you see printed. The 584227.4649743827f that you specified most likely won't even exist in the binary of your compiled program and is "translated" to the actually used value during compilation.
How should I work with values like this so I don't lose information
like having a deviation of 0.027474382659420f during assignment.
Use double, that has an accuracy of about 15-17 digits. Also, you need to remove the f from the 584227.4649743827f, turning it into a double constant instead. If that is not accurate enough, you may have to use external libraries instead for arbitrary precision numbers, such as GMP.
Your floating point numbers most likely adhere to the IEEE 754 standard, but there's no guarantee.
I'm comparing simple floats and doubles in C, specifically the value 8.7 for both of them. Now I assign 8.7 to each variable, when I print I get a result of 8.7000 for both values. Why has the compiler added these zeros. And the main question I wanted to ask was is there any further numbers that I'm not seeing, as in hidden after the trailing zeros. I read that I shouldn't do comparisons like this with float because of a lack of precision, but I thought with such a small value surely it can store 8.7 with a degree of accuracy needed to compare itself with another 8.7 value?
My only worry is that its actually being represented somewhere in memory as eg 8.70000003758 or something, which is throwing my comparisons off? I tried to printf with %.20f to see any further numbers that might be hiding but I think that just created numbers that were otherwise not there as the whole accuracy of the number changed to 8.6918734634834929 or something similar.
I'm comparing simple floats and doubles in C, specifically the value 8.7 for both of them.
Bad choice, since 8.7 has no exact binary representation.
Now I assign 8.7 to each variable, when I print I get a result of 8.7000 for both values. Why has the compiler added these zeros.
It hasn't, your print routine has.
And the main question I wanted to ask was is there any further numbers that I'm not seeing, as in hidden after the trailing zeros.
Definitely, since 8.7 has no exact binary representation. (Try to write it out as the sum of integer powers of 2, you can't do it.)
I read that I shouldn't do comparisons like this with float because of a lack of precision, but I thought with such a small value surely it can store 8.7 with a degree of accuracy needed to compare itself with another 8.7 value?
You thought wrong. 1/3 is small but has no exact decimal representation with a finite number of digits. Whether a value is big or small has nothing to do with whether it can be represented exactly with a finite number of digits in a particular base.
My only worry is that its actually being represented somewhere in memory as eg 8.70000003758 or something, which is throwing my comparisons off?
Exactly, just as representing 1/3 as 0.333333333 would do.
I tried to printf with %.20f to see any further numbers that might be hiding but I think that just created numbers that were otherwise not there as the whole accuracy of the number changed to 8.6918734634834929 or something similar.
That's probably just a bug. Show us that code. Perhaps you tried to output a double and left out the l.
Reading Here be dragons: advances in problems you didn’t even know you had I've noticed that they compare the new algorithm with the one used in glibc's printf:
Grisu3 is about 5 times faster than the algorithm used by printf in GNU libc
But at the same time I've failed to find any format specifier for printf which would automatically find the best number of decimal places to print. All I tried either have some strange defaults like 6 digits after decimal point for %f or 2 after point for %g or 6 after point for %e.
How do I actually make use of that algorithm implementation in glibc, mentioned in the article? Is there really any such implementation in glibc and is it even discussed in any way by the Standard?
This is the actual article. The blog post is referring to the results in section 7 (in other words, “they” are not comparing anything in the blog post, “they” are regurgitating the information from the actual article, omitting crucial details):
Implementations of Dragon4 or Grisu3 can be found in implementations of modern programming languages that specify this “minimal number of decimal digits” fashion (I recommend you avoid calling it “perfect”). Java uses this type of conversion to decimal in some contexts, as does Ruby. C is not one of the languages that specify “minimal number of decimal digits” conversion to decimal, so there is no reason for a compiler or for a libc to provide an implementation for Dragon4 or Grisu3.
There is no such thing as "best number of decimal places" because floating point numbers are not stored as decimal numbers. So you need to define what you mean by "best". If you want to print the numbers without any possible loss of information C11 gives you the format specifier %a (except for non-normalized floating point numbers where the behavior is unspecified).
The defaults from the C11 standard are 6 digits for %f and %e and for %g it is:
Let P equal the
precision if nonzero, 6 if the precision is omitted, or 1 if the precision is zero.
Then, if a conversion with style E would have an exponent of X:
— if P > X ≥ −4, the conversion is with style f (or F) and precision
P − (X + 1).
— otherwise, the conversion is with style e (or E) and precision P − 1.
If you want to use that algorithm, implement your own function for it. Or hope that glibc have implemented it in the past 5 years. Or just rethink if the performance of printing floating point numbers is really a problem you have.
In some situations, one generally uses a large enough integer value to represent infinity. I usually use the largest representable positive/negative integer. That usually yields more code, since you need to check if one of the operands is infinity before virtually all arithmetic operations in order to avoid overflows. Sometimes it would be desirable to have saturated integer arithmetic. For that reason, some people use smaller values for infinity, that can be added or multiplied several times without overflow. What intrigues me is the fact that it's extremely common to see (specially in programming competitions):
const int INF = 0x3f3f3f3f;
Why is that number special? It's binary representation is:
00111111001111110011111100111111
I don't see any specially interesting property here. I see it's easy to type, but if that was the reason, almost anything would do (0x3e3e3e3e, 0x2f2f2f2f, etc). It can be added once without overflow, which allows for:
a = min(INF, b + c);
But all the other constants would do, then. Googling only shows me a lot of code snippets that use that constant, but no explanations or comments.
Can anyone spot it?
I found some evidence about this here (original content in Chinese); the basic idea is that 0x7fffffff is problematic since it's already "the top" of the range of 4-byte signed ints; so, adding anything to it results in negative numbers; 0x3f3f3f3f, instead:
is still quite big (same order of magnitude of 0x7fffffff);
has a lot of headroom; if you say that the valid range of integers is limited to numbers below it, you can add any "valid positive number" to it and still get an infinite (i.e. something >=INF). Even INF+INF doesn't overflow. This allows to keep it always "under control":
a+=b;
if(a>INF)
a=INF;
is a repetition of equal bytes, which means you can easily memset stuff to INF;
also, as #Jörg W Mittag noticed above, it has a nice ASCII representation, that allows both to spot it on the fly looking at memory dumps, and to write it directly in memory.
I may or may not be one of the earliest discoverers of 0x3f3f3f3f. I published a Romanian article about it in 2004 (http://www.infoarena.ro/12-ponturi-pentru-programatorii-cc #9), but I've been using this value since 2002 at least for programming competitions.
There are two reasons for it:
0x3f3f3f3f + 0x3f3f3f3f doesn't overflow int32. For this some use 100000000 (one billion).
one can set an array of ints to infinity by doing memset(array, 0x3f, sizeof(array))
0x3f3f3f3f is the ASCII representation of the string ????.
Krugle finds 48 instances of that constant in its entire database. 46 of those instances are in a Java project, where it is used as a bitmask for some graphics manipulation.
1 project is an operating system, where it is used to represent an unknown ACPI device.
1 project is again a bitmask for Java graphics.
So, in all of the projects indexed by Krugle, it is used 47 times because of its bitpattern, once because of its ASCII interpretation, and not a single time as a representation of infinity.
in Redis (http://code.google.com/p/redis) there are scores associated to elements, in order to take this elements sorted. This scores are doubles, even if many users actually sort by integers (for instance unix times).
When the database is saved we need to write this doubles ok disk. This is what is used currently:
snprintf((char*)buf+1,sizeof(buf)-1,"%.17g",val);
Additionally infinity and not-a-number conditions are checked in order to also represent this in the final database file.
Unfortunately converting a double into the string representation is pretty slow. While we have a function in Redis that converts an integer into a string representation in a much faster way. So my idea was to check if a double could be casted into an integer without lost of data, and then using the function to turn the integer into a string if this is true.
For this to provide a good speedup of course the test for integer "equivalence" must be fast. So I used a trick that is probably undefined behavior but that worked very well in practice. Something like that:
double x = ... some value ...
if (x == (double)((long long)x))
use_the_fast_integer_function((long long)x);
else
use_the_slow_snprintf(x);
In my reasoning the double casting above converts the double into a long, and then back into an integer. If the range fits, and there is no decimal part, the number will survive the conversion and will be exactly the same as the initial number.
As I wanted to make sure this will not break things in some system, I joined #c on freenode and I got a lot of insults ;) So I'm now trying here.
Is there a standard way to do what I'm trying to do without going outside ANSI C? Otherwise, is the above code supposed to work in all the Posix systems that currently Redis targets? That is, archs where Linux / Mac OS X / *BSD / Solaris are running nowaday?
What I can add in order to make the code saner is an explicit check for the range of the double before trying the cast at all.
Thank you for any help.
Perhaps some old fashion fixed point math could help you out. If you converted your double to a fixed point value, you still get decimal precision and converting to a string is as easy as with ints with the addition of a single shift function.
Another thought would be to roll your own snprintf() function. Doing the conversion from double to int is natively supported by many FPU units so that should be lightning fast. Converting that to a string is simple as well.
Just a few random ideas for you.
The problem with doing that is that the comparisons won't work out the way you'd expect. Just because one floating point value is less than another doesn't mean that its representation as an integer will be less than the other's. Also, I see you comparing one of the (former) double values for equality. Due to rounding and representation errors in the low-order bits, you almost never want to do that.
If you are just looking for some kind of key to do something like hashing on, it would probably work out fine. If you actually care about which values really have greater or lesser value, its a bad idea.
I don't see a problem with the casts, as long as x is within the range of long long. Maybe you should check out the modf() function which separates a double into its integral and fractional part. You can then add checks against (double)LLONG_MIN and (double)LLONG_MAX for the integral part to make sure. Though there may be difficulties with the precision of double.
But before doing anything of this, have you made sure it actually is a bottleneck by measuring its performance? And is the percentage of integer values high enough that it would really make a difference?
Your test is perfectly fine (assuming you have already separately handled infinities and NANs by this point) - and it's probably one of the very few occaisions when you really do want to compare floats for equality. It doesn't invoke undefined behaviour - even if x is outside of the range of long long, you'll just get an "implementation-defined result", which is OK here.
The only fly in the ointment is that negative zero will end up as positive zero (because negative zero compares equal to positive zero).