I don't want to introduce floating point when an inexact value would be a distaster, so I have a couple of questions about when you actually can use them safely.
Are they exact for integers as long as you don't overflow the number of significant digit? Are these two tests always true:
double d = 2.0;
if (d + 3.0 == 5.0) ...
if (d * 3.0 == 6.0) ...
What math functions can you rely on? Are these tests always true:
#include <math.h>
double d = 100.0;
if (log10(d) == 2.0) ...
if (pow(d, 2.0) == 10000.0) ...
if (sqrt(d) == 10.0) ...
How about this:
int v = ...;
if (log2((double) v) > 16.0) ... /* gonna need more than 16 bits to store v */
if (log((double) v) / log(2.0) > 16.0) ... /* C89 */
I guess you can summarize this question as: 1) Can floating point types hold the exact value of all integers up to the number of their significant digits in float.h? 2) Do all floating point operators and functions guarantee that the result is the closest to the actual mathematical result?
I too find incorrect results distasteful.
On common hardware, you can rely on +, -, *, /, and sqrt working and delivering the correctly-rounded result. That is, they deliver the floating-point number closest to the sum, difference, product, quotient, or square root of their argument or arguments.
Some library functions, notably log2 and log10 and exp2 and exp10, traditionally have terrible implementations that are not even faithfully-rounded. Faithfully-rounded means that a function delivers one of the two floating-point numbers bracketing the exact result. Most modern pow implementations have similar issues. Lots of these functions will even blow exact cases like log10(10000) and pow(7, 2). Thus equality comparisons involving these functions, even in exact cases, are asking for trouble.
sin, cos, tan, atan, exp, and log have faithfully-rounded implementations on every platform I've recently encountered. In the bad old days, on processors using the x87 FPU to evaluate sin, cos, and tan, you would get horribly wrong outputs for largish inputs and you'd get the input back for larger inputs. CRlibm has correctly-rounded implementations; these are not mainstream because, I'm told, they've got rather nastier worst cases than the traditional faithfully-rounded implementations.
Things like copysign and nextafter and isfinite all work correctly. ceil and floor and rint and friends always deliver the exact result. fmod and friends do too. frexp and friends work. fmin and fmax work.
Someone thought it would be a brilliant idea to make fma(x,y,z) compute x*y+z by computing x*y rounded to a double, then adding z and rounding the result to a double. You can find this behaviour on modern platforms. It's stupid and I hate it.
I have no experience with the hyperbolic trig, gamma, or Bessel functions in my C library.
I should also mention that popular compilers targeting 32-bit x86 play by a different, broken, set of rules. Since the x87 is the only supported floating-point instruction set and all x87 arithmetic is done with an extended exponent, computations that would induce an underflow or overflow in double precision may fail to underflow or overflow. Furthermore, since the x87 also by default uses an extended significand, you may not get the results you're looking for. Worse still, compilers will sometimes spill intermediate results to variables of lower precision, so you can't even rely on your calculations with doubles being done in extended precision. (Java has a trick for doing 64-bit math with 80-bit registers, but it is quite expensive.)
I would recommend sticking to arithmetic on long doubles if you're targeting 32-bit x86. Compilers are supposed to set FLT_EVAL_METHOD to an appropriate value, but I do not know if this is done universally.
Can floating point types hold the exact value of all integers up to the number of their significant digits in float.h?
Well, they can store the integers which fit in their mantissa (significand). So [-2^53, 2^53] for double. For more on this, see: Which is the first integer that an IEEE 754 float is incapable of representing exactly?
Do all floating point operators and functions guarantee that the result is the closest to the actual mathematical result?
They at least guarantee that the result is immediately on either side of the actual mathematical result. That is, you won't get a result which has a valid floating point value between itself and the "actual" result. But beware, because repeated operations may accumulate an error which seems counter to this, while it is not (because all intermediate values are subject to the same constraints, not just the inputs and output of a compound expression).
Related
I am using the GMP and MPFR libraries to work with large numbers and I need to calculate the power of a number quickly. The result of the potentiation will always be an integer, but the potency may or may not be a floating number. the GMP library calculates powers very quickly but does not accept floating powers (Using the mpz_pow_ui function), the MPFR library accepts floating powers but is extremely slow as it requires high precision to calculate integers correctly (using the function mpfr_pow).
Is there any solution for this? How can GMP accept floating powers, or MPFR calculate whole numbers quickly (and correctly)?
//Ex:
mpz_pow_ui(mpz_power, base, 4790) //Fast
// power = 4790.60
mpfr_pow(mpfr_power, base, power, MPFR_RNDN) //Slow
The mpz_pow_ui function is fast since the computation can be done with multiplications. Note that mpfr_pow_ui with MPFR_RNDN should be faster if you don't need the exact integer result (which will be huge if the exponent is large), as the multiplications can be done with a smaller precision and rounded.
Unless the exponent is a not-too-large integer, mpfr_pow will be slower because it needs to compute a logarithm and an exponential, which are more complex than multiplications. I don't think that you can avoid it, even if you know that the result will be an integer. But if you know that the result will be an integer, you can compute it with an error less than 1/2. Thus, with an error analysis, you may be able to reduce the precision of the input and output variables, so that mpfr_pow will be faster.
Note: You are saying
the MPFR library accepts floating powers but is extremely slow as it requires high precision to calculate integers correctly (using the function mpfr_pow)
This is not true. MPFR will carefully chose the intermediate precisions to provide an answer with the required accuracy (though it may sometimes do wrong choices).
However, if the exact result is very close to a "machine number", there may be an issue due to the fact that MPFR needs to return the sign of the error (faithful rounding MPFR_RNDF instead of MPFR_RNDN could help, but it is not optimized yet for mpfr_pow): the Table Maker's Dilemma will occur, requiring internal computations in a higher precision. But if you have carefully chosen the precisions of the inputs, this is unlikely to occur in your case; indeed, reducing the precisions of the input will tend to add some error to the exact result, and this will move the exact result away from the expected integer (by exact result, I mean the exact result with the approximate inputs). You can also use some tricks. For instance, if you know that your integer is not a perfect square, then instead of computing xy, you can compute xy/2 (which does not correspond to a machine number, thus should not have an issue due to the Table Maker's Dilemma), then square the result.
It all depends on the order of magnitude of the numbers you are calculating with. The following assumes both the base and the exponent are positive non-complex numbers.
First note that exponentiation of an integer b by a decimal number like 4790.60 can be rewritten like this (just can't beleive I can't write Latex-style math equations in StackOverflow):
b ^ 4790.60 = b ^ (4790 + 0.60) = (b ^ 4790) * (b ^ 0.60)
Then, the first term (b ^ 4790) can clearly be calculated with GMP and results in an integer value.
The second term has an exponent smaller than 1, so its value will be smaller than b. If b is not a huge integer (say << FLINTMAX, the largest integer that can be represented consecutively in a floating point value, is 2^53 for double), then you can use the native double pow function to calculate it and safely round it to integer as desired, and then multiply by the first term. If b is a huge integer, you have too options: convert it to a double if it's within the double range and then use the double pow function (and perhaps loose some precision in the resulting integer with a speed gain), or you can use the mpfr_pow function to calculate this second term, but noting that it will be an smaller number than b, and so you can adjust MPFR precision in that calculation according to the exponent b: if b is closer to zero use a small precision, if b is close to 1 use a precision not too smaller than that of b.
On the end, all of it is a tradeoff between accuracy and speed, given the machine resource limits you are bound to.
When I calculate tan(PI/2) I get -22877332 in c, but tan(Pi/2) is infinity.
and google giving it as 3060023.30695 Why i am getting different answer
I tried on mingw compiler and in google both are giving different answer
float32 Tan_f32 (float32 ValValue )
{
float32 Result_Val;
Result_Val= (tanf(ValValue));
return Result_Val;
}
in mingw compiler it gives -22877332
and in google 3060023.30695
It is impossible to pass π/2 to tan or tanf because π is irrational, so any floating-point number, no matter how precise, will be at least slightly different from π/2. Therefore, tanf(ValValue) returns the tangent of some value close to π/2, and that tangent is large but not infinite.
In the common format used for float, IEEE-754 basic 32-bit binary floating-point, the closest representable number to π/2 is 1.57079637050628662109375. The tangent of that number is approximately −22877332.4289, and the closest value representable in float is −22877332, which is the result you got. So your tanf is giving you the best possible result for the input number you gave it.
The C standard, or indeed the common but by no means ubiquitous IEEE754 floating point standard, give no guarantee of the accuracy of tan (Cf sqrt). An implementation will make a compromise in getting a good result out in a reasonable number of clock cycles.
In particular, the behaviour of the trigonometric function near an asymptote is particularly unpredictable; and that's the case here.
Accepting that the fault is not due to your value of pi (worth a check although note that because pi is transcendental it can't be represented exactly in any floating point system), if you want a well-behaved tan function across the whole domain, you'll be better off using a third party mathematics library.
Finally, note that under IEEE754, you might get more consistent behaviour around an asyptote if you let floating point division deal with the pole, and use
double c = cos(x); tan(x) = sqrt(1 / c / c - 1);
This might be more numerically stable, as IEEE754 defines a division by zero.
Regarding minimising the error in floating-point operations, if I have an operation such as the following in C:
float a = 123.456;
float b = 456.789;
float r = 0.12345;
a = a - (r * b);
Will the result of the calculation change if I split the multiplication and subtraction steps out, i.e.:
float c = r * b;
a = a - c;
I am wondering whether a CPU would then treat these calculations differently and thereby the error may be smaller in one case?
If not, which I presume anyway, are there any good rules-of-thumb to mitigate against floating-point error? Can I massage data in a way that will help?
Please don't just say "use higher precision" - that's not what I'm after.
EDIT
For information about the data, in the general sense errors seem to be worse when the operation results in a very large number like 123456789. Small numbers, such as 1.23456789, seem to yield more accurate results after operations. Am I imagining this, or would scaling larger numbers help accuracy?
Note: this answer starts with a lengthy discussion of the distinction between a = a - (r * b); and float c = r * b; a = a - c; with a c99-compliant compiler. The part of the question about the goal of improving accuracy while avoiding extended precision is covered at the end.
Extended floating-point precision for intermediate results
If your C99 compiler defines FLT_EVAL_METHOD as 0, then the two computations can be expected to produce exactly the same result. If the compiler defines FLT_EVAL_METHOD to 1 or 2, then a = a - (r * b); will be more precise for some values of a, r and b, because all intermediate computations will be done at an extended precision (double for the value 1 and long double for the value 2).
The program cannot set FLT_EVAL_METHOD, but you can use commandline options to change the way your compiler computes with floating-point, and that will make it change its definition accordingly.
Contraction of some intermediate results
Depending whether you use #pragma fp_contract in your program and on your compiler's default value for this pragma, some compound floating-point expressions can be contracted into single instructions that behave as if the intermediate result was computed with infinite precision. This happens to be a possibility for your example when targeting a modern processor, as the fused-multiply-add instruction will compute a directly and as accurately as allowed by the floating-point type.
However, you should bear in mind that the contraction only take place at the compiler's option, without any guarantees. The compiler uses the FMA instruction to optimize speed, not accuracy, so the transformation may not take place at lower optimization levels. Sometimes several transformations are possible (e.g. a * b + c * d can be computed either as fmaf(c, d, a*b) or as fmaf(a, b, c*d)) and the compiler may choose one or the other.
In short, the contraction of floating-point computations is not intended to help you achieve accuracy. You might as well make sure it is disabled if you like reproducible results.
However, in the particular case of the fused-multiply-add compound operation, you can use the C99 standard function fmaf() to tell the compiler to compute the multiplication and addition in a single step with a single rounding. If you do this, then the compiler will not be allowed to produce anything else than the best result for a.
float fmaf(float x, float y, float z);
DESCRIPTION
The fma() functions compute (x*y)+z, rounded as one ternary operation:
they compute the value (as if) to infinite precision and round once to
the result format, according to the current rounding mode.
Note that if the FMA instruction is not available, your compiler's implementation of the function fmaf() will at best just use higher precision, and if this happens on your compilation platform, your might just as well use the type double for the accumulator: it will be faster and more accurate than using fmaf(). In the worst case, a flawed implementation of fmaf() will be provided.
Improving accuracy while only using single-precision
Use Kahan summation if your computation involves a long chain of additions. Some accuracy can be gained by simply summing the r*b terms computed as single-precision products, assuming there are many of them. If you wish to gain more accuracy, you might want to compute r*b itself exactly as the sum of two single-precision numbers, but if you do this you might as well switch to double-single arithmetics entirely. Double-single arithmetics would be the same as the double-double technique succinctly described here, but with single-precision numbers instead.
Yesterday I asked a floating point question, and I have another one. I am doing some computations where I use the results of the math.h (C language) sine, cosine and tangent functions.
One of the developers muttered that you have to be careful of the return values of these functions and I should not make assumptions on the return values of the gcc math functions. I am not trying to start a discussion but I really want to know what I need to watch out for when doing computations with the standard math functions.
x
You should not assume that the values returned will be consistent to high degrees of precision between different compiler/stdlib versions.
That's about it.
You should not expect sin(PI/6) to be equal to cos(PI/3), for example. Nor should you expect asin(sin(x)) to be equal to x, even if x is in the domain for sin. They will be close, but might not be equal.
Floating point is straightforward. Just always remember that there is an uncertainty component to all floating point operations and functions. It is usually modelled as being random, even though it usually isn't, but if you treat it as random, you'll succeed in understanding your own code. For instance:
a=a/3*3;
This should be treated as if it was:
a=(a/3+error1)*3+error2;
If you want an estimate of the size of the errors, you need to dig into each operation/function to find out. Different compilers, parameter choice etc. will yield different values. For instance, 0.09-0.089999 on a system with 5 digits precision will yield an error somewhere between -0.000001 and 0.000001. this error is comparable in size with the actual result.
If you want to learn how to do floating point as precise as posible, then it's a study by it's own.
The problem isn't with the standard math functions, so much as the nature of floating point arithmetic.
Very short version: don't compare two floating point numbers for equality, even with obvious, trivial identities like 10 == 10 / 3.0 * 3.0 or tan(x) == sin(x) / cos(x).
you should take care about precision:
Structure of a floating-point number
are you on 32bits, 64 bits Platform ?
you should read IEEE Standard for Binary Floating-Point Arithmetic
there are some intersting libraries such GMP, or MPFR.
you should learn how Comparing floating-point numbers
etc ...
Agreed with all of the responses that say you should not compare for equality. What you can do, however, is check if the numbers are close enough, like so:
if (abs(numberA - numberB) < CLOSE_ENOUGH)
{
// Equal for all intents and purposes
}
Where CLOSE_ENOUGH is some suitably small floating-point value.
This started suddenly today morning.
Original lines were this
float angle = (x+90)*(M_PI/180.0);
float xx = cosf(angle);
float yy = sinf(angle);
After putting a breakpoint and hovering cursor.. I get the correct answer for yy as 1. but xx is NOT zero.
I tried with cosf(M_PI_2); still no luck.. it was working fine till yesterday.. I did not change any compiler setting etc..
I am using Xcode latest version as of todays date
The first thing to notice is that you're using floats. These are inherently inaccurate, and for most calculations give you only a close approximation of the mathematically-correct answer. Assuming that x in your code has value 0, angle will have a close approximation to π/2. xx will therefore have an approximation to cos(π/2). However, this is unlikely to be exactly zero due to approximation and rounding issues.
If you were able to change your code to us doubles rather than floats you're likely to get more accuracy, and an answer nearer zero. However, if it is important for your code to produce a value of exactly zero at this point, you're going to have to rethink how you're doing the calculations.
If this doesn't answer your particular problem, give us some more details and we'll have another think.
Contrary to what others have said, this is not an x87 co-processor issue. XCode uses SSE for floating-point computation on Intel by default (except for long double arithmetic).
The "problem" is: when you write cosf(M_PI_2), you are actually telling the XCode compiler (gcc or llvm-gcc or clang) to do the following:
Look up the expansion of M_PI_2 in <math.h>. Per the POSIX standard, it is a double precision literal that converts to the correctly rounded value of π/2.
Round the converted double precision value to single precision.
Call the math library function cosf on the single precision value.
Note that, throughout this process, you are not operating on the actual value of π/2. You are instead operating on that value rounded to a representable floating-point number. While cos(π/2) is exactly zero, you are not telling the compiler to do that computation. You are instead telling the compiler to do cos(π/2 + tiny), where tiny is the difference between the rounded value (float)M_PI_2 and the (unrepresentable) exact value of π/2. If cos is computed with no error at all, the result of cos(π/2 + tiny) is approximately -tiny. If it returned zero, that would be an error.
edit: a step-by-step expansion of the computation on an Intel mac with the current XCode compiler:
M_PI_2 is defined to be
1.57079632679489661923132169163975144
but that's not actually a representable double precision number. When the compiler converts it to a double precision value it becomes exactly
1.5707963267948965579989817342720925807952880859375
This is the closest double-precision number to π/2, but it differs from the actual mathematical value of π/2 by about 6.12*10^(-17).
Step (2) rounds this number to single-precision, which changes the value to exactly
1.57079637050628662109375
Which is approximately π/2 + 4.37*10^(-8). When we compute cosf of this number then, we get:
-0.00000004371138828673792886547744274139404296875
which is very nearly the exact value of cosine evaluated at that point:
-0.00000004371139000186241438857289400265215231661...
In fact, it is the correctly rounded result; there is no value that the computation could have returned that would be more accurate. The only error here is that the computation that you asked the compiler to perform is different from the computation that you thought you were asking it to do.
I suspect the answer is as near as damnit to 0 as not to be worth worrying about.
If i run the same thing through I get the answer "-4.3711388e-008" which can also be written as "-0.000000043711388". Which is pretty damned close to 0. Definitely near enough to not worry about it being out at the 8th decimal place.
Edit: Further to what LiraLuna is saying I wrote the following piece of x87 assembler under visual studio
float fRes;
_asm
{
fld1
fld1
fadd st, st(1)
fldpi
fdiv st, st(1)
fcos
fstp [fRes]
}
char str[16];
sprintf( str, "%f", fRes );
Basically this uses the x87's fcos instruction to do a cosine of pi/2. the value held in str is "0.000000"
This, however, is not actually what fcos returned. It ACTUALLY returned 6.1230318e-017. This implies that the error occurs at the 17th decimal place and, lets be honest, thats far less significant than the standard debug cosf above.
As SSE3 has no specific cosine instruction I suspect (though i cannot confirm without seeing the assembler generated) that it is either using its own taylor series expansion or it is using the fcos instruction anyway. Either way you are still unlikely to get better precision than the error occurring at the 17th decimal place, in my opinion.
The only thing I can think of is a malicious macro substituion i.e. M_PI_2 is no longer 1.57079632679489661923.
Try calling cosf( 1.57079632679489661923 ) to test this.
The real thing you should be careful about is the sign of cosine. Make sure it is the same as you expected. E.g. if you operate with angles between 0 and pi/2. make sure that what you use as PI_2 is less that actual value of pi/2!
And the difference between 0.000001 and 0.0 is less than you think.
The reason
What you are experiencing is the infamous x87 math co-processor float truncate 'bug' - or rather - a feature. IEEE floats have an amazing range of numbers, but at a cost. They sacrifice precession for high range.
They are not inaccurate as you think, though - this is a semi-myth generate by Intel's x87 chip design, that internally uses 80bit internal representation for floats - they have far superior precession though a bit slower.
When you perform a float comparison, x87 caches the float as an 80bit float, then when it's stack is full, it saves the 32bit representation in RAM, decreasing accuracy by a large degree.
The solution
x87 is old, really old. It's replacement is SSE. SSE computes 32bit floats and 64bit floats natively, leading to minimal precession lost on math. Please note that precession issues with floats still exist, but printf("%f\n", cosf(M_PI_2)); should be zero. Heck - even float comparison with SSE is accurate again! (unlike x87).
Since latest Xcode is actually GCC 4.2.1, use the compiler switch -msse3 -mfpmath=sse and see how you get a perfectly round 0.00000 (Note: if you get -0.00000, do not worry, it's perfectly fine and still equals 0.00000 under the IEEE spec (read more at this wikipedia article)).
All Intel macs are guaranteed to have SSE3 support (OSx86 Macs excluded, if you want to support those, use -msse2).