Round functions has two behaviors: With the value cours is equal to "3.1235", round(cours, 3) = 3.123. Although, when we replace cours by its value (3.1235) in this round formula, round(3.1235, 3) = 3.1240.
You shouldn't be using the float datatype for specific decimal values like this as it's not designed for that purpose. Would need to see more of your code to get a better context of what you're trying to do, but if it needs to be a float initially, potentially you could cast #cours as decimal?
round(cast(#cours as decimal(5,4)), 3)
Your FLOAT does not really contain 3.1235, that is only what is printed or shown in a grid. Internally the FLOAT is 3.1234999999999999, which is obviously rounded down to 3.123.
The literal 3.1235 becomes a NUMERIC with enough precision to be totally exact, and so it is rounded up to 3.124, as one would expect.
Proof:
SELECT CAST('3.1235' as FLOAT),
CAST( 3.1235 as FLOAT)
-- misleading output: both print 3.1235
SELECT CAST(CAST('3.1235' as FLOAT) as NUMERIC(24,23)),
CAST(CAST( 3.1235 as FLOAT) as NUMERIC(24,23))
-- both print 3.12349999999999990000000
SELECT CAST('3.1235' as NUMERIC(24,23)),
CAST( 3.1235 as NUMERIC(24,23))
-- both print 3.12350000000000000000000
Related
I'm dividing two floats, multiplying it by 100 and then subtracting it by 100. I'm returning a percentage.
My question is: why is the final result a float that isn't rounded when the right part of the subtraction returns a float of 2 digits?
These is one sequence:
/* 1 */
-- Returns 0.956521739130435, which is correct.
select cast(198 as float)/(cast(198 as float) + cast(9 as float)) -- correct
/* 2 */
-- Returns 95.6521739130435, which is correct.
select 100*(cast(198 as float)/(cast(198 as float) + cast(9 as float))) --correct
/* 3 */
-- It's the same as previous one, but with a ROUND
-- Returns 95.65, which is correct.
select round(100*(cast(198 as float)/(cast(198 as float) + cast(9 as float))),2)
/* 4 */
-- Returns 4.34999999999999, should be 100-95.65, but it's not. ROUND is ignored. Why?
select 100-round(100*(cast(198 as float)/(cast(198 as float) + cast(9 as float))),2)
|-------------- This returns 95.65 --------------------------------|
Another sequence:
/* 1 */
-- Returns 0.956521739130435, which is correct.
select cast(198 as float)/(cast(198 as float) + cast(9 as float))
/* 2 */
-- Returns 0.9565, which is correct.
select round(cast(198 as float)/(cast(198 as float) + cast(9 as float)), 4)
/* 3 */
-- Returns 95.65, which is correct.
select 100*round(cast(198 as float)/(cast(198 as float) + cast(9 as float)), 4)
/* 4 */
-- Returns 4.34999999999999, should be 100-95.65, but it's not. ROUND is ignored. Why?
select 100-(100*round(cast(198 as float)/(cast(198 as float) + cast(9 as float)), 4))
|-------------------- This returns 95.65 --------------------------------|
I'm just curious as to why this happens, although it can easily be fixed with one ROUND at the beginning:
select round(100-(100*(cast(198 as float)/(cast(198 as float) + cast(9 as float)))), 2)
The reason I ask is because it's not something that can be easily reproduced. I tried reproducing it, and out of 2,000 times, it only occurred 12 times. That's less than 1%, but with floats with repetitive numbers after the 2nd decimal (ie. 3.47999999999), which makes sense:
declare #rand int = 1
While(#rand <= 2000)
begin
select 100-round(100*(cast(abs(checksum(NewId()) % 1500) as float)/(cast(abs(checksum(NewId()) % 1500) as float) + cast(abs(checksum(NewId()) % 1500) as float))),2)
set #rand = #rand + 1
end
I guess my other question is: what type is the sql editor returning when it returns 95.65 with select round(100*(cast(198 as float)/(cast(198 as float) + cast(9 as float))),2)?
To expand on Jeroen's comment:
SQL Server's FLOAT type is a double-precision floating-point value. As with (most) floating point formats, the value is stored in binary. Just as the number 1/3 cannot be represented with a finite number of digits after the decimal, the number 95.65 cannot be represented with a finite number of bits. The closest value to 95.65 that can be stored in a FLOAT has the exact value:
95.650000000000005684341886080801486968994140625
If you subtract that number from 100, you get exactly:
4.349999999999994315658113919198513031005859375
When displayed, this is rounded to 15 significant digits, and the value printed is:
4.34999999999999
As discussed, you can solve this problem by using DECIMAL type instead of FLOAT.
There are many resources available on StackOverflow and elsewhere if you'd like to learn more about floating-point math.
-- EDIT --
I'm going to use parenthesis notation for repeating decimals. When I write
0.(3)
that means
0.333333333333333333333333333... and so on forever.
Let's start at the beginning. 168 can be stored in a float. 168+9 is 177. That can be stored in a float. If you divide 168 by 177 the mathematically correct answer is:
0.95(6521739130434782608695)
But this value cannot be stored in a float. The closest value that can be stored in a float is:
0.9565217391304348115710354250040836632251739501953125
Take that number and multiply by 100 , the mathematically correct answer is:
95.65217391304348115710354250040836632251739501953125
Since you multiplied a float by 100, you get a float, and that number cannot be stored in a float, so the closest possible value is:
95.6521739130434838216388016007840633392333984375
You ask that this float be rounded to 2 digits after the decimal. The mathematically correct answer is:
95.65
But since you asked to round a float, the answer is also a float, and that value cannot be stored in a float. The closest possible value is:
95.650000000000005684341886080801486968994140625
You asked to subtract that from 100. The mathematically correct value is:
4.349999999999994315658113919198513031005859375
As it happens, that value can be stored in a float. So that's the value that's being selected.
When converting this number to a string, SQL Server rounds the result to 15 significant digits. So that number, when printed, appears as:
4.34999999999999
When you ran the same calculation on your Java console, the exact same calculations were performed, but when the value was printed, Java rounded to 16 significant digits:
4.349999999999994
-- Another EDIT --
Why can't 96.65 be stored exactly in a float? The float type stores numbers in binary format. If you want to express 96.65 in binary, the mathematically exact value is:
1011111.1010011001100110011001100110011001100110011001(1001)
You can see the pattern. Just as 1/3 is represented as an infinite repeating value in decimal, this value has an infinite repeating value in binary. You can see the pattern (1001) being repeated over and over.
A float can only hold 53 significant bits. And so this is rounded to:
1011111.1010011001100110011001100110011001100110011010
If you convert that number back to decimal, you get the exact value:
95.650000000000005684341886080801486968994140625
-- Yet Another Edit --
You ask what happens when you call Round again on the result.
We started with the number:
4.349999999999994315658113919198513031005859375
You ask that this be rounded to 2 places. The mathematically correct answer is:
4.35
Since you are rounding a float, this result must also be a float. Express this value in binary. The mathematically correct answer is:
100.0101100110011001100110011001100110011001100110011001(1001)
Again, this is a repeating binary value. But float can't store an infinite number of bits. The value is rounded to 53 significant bits. The result is:
100.0101100110011001100110011001100110011001100110011
If you convert this to decimal, the exact value is:
4.3499999999999996447286321199499070644378662109375
That is the value you selected. Now SQL Server needs to print that on the screen. As before, it is rounded to 15 significant digits. The result is:
4.35000000000000
It removes the trailing zeros, and the result you see on the screen is:
4.35
The last round did nothing magic. The answer is still stored as a float, and the answer is still not an exact value. As it happens SQL Server chooses to round values to 15 significant digits when printing a float. In this case, that rounded value happened to match the exact value you were expecting.
If values were rounded to 14 places when printing them, the original query would have appeared to have the value you expected.
If values were rounded to 16 places, then the result of the final round would be shown as
4.3499999999999996
I have a piece of code:
IF OBJECT_ID(N'dbo.rounding_testing') IS NOT NULL
DROP FUNCTION dbo.rounding_testing;
GO
CREATE FUNCTION dbo.rounding_testing
(
#value FLOAT,
#digit INT
)
RETURNS FLOAT
BEGIN
DECLARE
#factor FLOAT,
#result FLOAT;
SELECT #factor = POWER(10, #digit);
SELECT #result = FLOOR(#value * #factor + 0.4);
RETURN #result;
END;
GO
SELECT dbo.rounding_testing(5.7456, 3);
SELECT FLOOR(5.7456 * 1000 + 0.4);
The results are:
5745
5746
I'm expecting two 5746. I tried to debug the function and found some interesting behavior. Below are some testing I did in the Immediate Window when debugging.
#factor
1.000000000000000e+003
#result
5.745000000000000e+003
#value
5.745600000000000e+000
#value*#factor
5745.6
#value*#factor+0.4
5746
floor(#value*#factor+0.4)
5745
floor(5746)
5746
Can anyone help to explain the result? Especially these three lines:
#value*#factor+0.4
5746
floor(#value*#factor+0.4)
5745
floor(5746)
5746
In the expression FLOOR(5.7456 * 1000 + 0.4);, the part between parentheses is evaluated first. For constants the data types are inferred based on the notation; for 5.7456 that is decimal(5,4); 1000 is an int; and 0.4 is decimal(1,1). The inferred data type for 5.7456 * 1000 is then decimal(10,4); and for the full expression it is decimal(11,4). These are all exact numeric data types so you will not experience any rounding; the end result is 5746.0000 exactly. The FLOOR function trims the fraction and converts to decimal(11,0), returning 5746.
In the user-defined function, you store input parameters and intermediate results in float data type (floating point data). This data type is intended to be used for approximate data, such as measurements, where the data you read from the intstrument is already an approximation. I have learned in high school to read as many digits as I can, but treat the last one as insignificant - I had to keep it in all computations, but round the final result to the number of significant digits based on the accuracy of my measurements. The rounding ensures that inaccuracies in the last digits will not affect the end result.
Floating point data types should be treated in the same way.
Internally, floating point digits are represented in a base-2 number system. This means that there are numbers that have an exact representation in our commonly used base-10 system (such as 5.7456), but a never ending fractional part in base-2. (Similar to how for instance one third, which can be represented exactly in base-3, has a never ending fractional part in base-10: 0.33333333333(etc)). The number of base-2 digits used for storage of a float number is finite, so it has to be cut off at the end - which results in it being rounded either up or down by a tiny fraction. You can see this if you run:
DECLARE #a float = 5.7456;
SELECT CAST(#a AS decimal(19,16));
In this case, the effect of cutting off after a lot of base-2 digits is that the value stored is 0.0000000000000004 less than the decimal value you put in. That small difference turns into a huge effect because of the FLOOR function, which does exactly what it should do: round down to the nearest integer.
(I've seen a lot of people call this an error. It is not. It is intended and documented behavior. And the precision loss here is neither worse nor better than the precision loss you get when you store a third in a DECIMAL(7,6); it is just a bit less obvious because we have all grown up being used to working in base-10)
You issue can be fixed by changing float to real
IF OBJECT_ID(N'dbo.rounding_testing') IS NOT NULL
DROP FUNCTION dbo.rounding_testing;
GO
CREATE FUNCTION dbo.rounding_testing
(
#value REAL,
#digit INT
)
RETURNS REAL
BEGIN
DECLARE
#factor REAL,
#result REAL;
SELECT #factor = POWER(10, #digit);
SELECT #result = FLOOR(#value * #factor + 0.4);
RETURN #result;
END;
GO
SELECT dbo.rounding_testing(5.7456, 3);
SELECT FLOOR(5.7456 * 1000 + 0.4);
sum(convert(int,<your column) * .01) as 'Decimal Amount'
Convert column to integer, then multiply by .01. Sum converted field, if desired.
I have a number with two digits after the decimal point; I try to cast this number using CAST for example:
Select SUM(CAST((123345.56) as decimal(28, 2))) * 100 AS AMOUNT
I get the following result
12334556.00
but I want get only 12334556 without displaying two digits after the decimal point.
Thank you.
It is not clear what you really want to do, but
SELECT SUM(CAST((123345.56 * 100.0) AS DECIMAL(28,0))) AS AMOUNT
gives 12334556.
When you use a number in the format 1234.56 it is a decimal literal in SQL Server: see "decimal constants" in Constants (Transact-SQL).
Integer data type is whole numbers, so cast your calculated value as int.
Select SUM(CAST((123345.56 * 100) as int)) AS AMOUNT
I was trying to round some fields. When I have 59 days, I want to change it to 60.
The problem is that when I use this code, the 59 is changed to 30 because the round it is 1.
select round(1.9666,0)*30, round(59/30,0)*3'
The result of that query is 60 for the first field and 30 for the second one. The problem is that when I've tried:
select 59/30
The result is 1 and I need the entire answer that is 1.9666...
How can I make it?
Because the number you are dividing by is an INT (the data type of the left side is irrelevant), SQL Server will return an INT as the answer.
If you want a number with a decimal place as your result, you'll need to divide by one.
Don't cast to a FLOAT as the answer is probably not what you want (floats are generally not accurate and are 'approximations'):
SELECT 59 / CAST(30 AS FLOAT) -- = 1.96666666666667
CAST the right-hand side of the division to a DECIMAL:
SELECT 59 / CAST(30 AS DECIMAL(10, 2)) -- = 1.96666
SELECT cast(59 AS FLOAT) / cast(30 AS FLOAT)
Because the original figures are whole numbers, SQL presumes you want a whole number output.
To ensure you get one with the decimal places, you need to first change the data type from an integer int to a floating point float.
This is what the CAST command does.
EDIT: Commenter suggests you cast to DECIMAL instead. The principle is the same, but you need to supply more arguments. To cast to a decimal use something like:
cast(59 as DECIMAL(18, 3))
The first argument (the 18) is the total number of figures you want to permit in the decimal. The second argument (the 3) is the number you want after the decimal point.
The suggestion that it's more accurate is correct - as you'll see if you run the SELECT statements in this answer one after the other. But in this particular case, it only makes a tiny difference.
Using T-SQL and Microsoft SQL Server I would like to specify the number of decimal digits when I do a division between 2 integer numbers like:
select 1/3
That currently returns 0. I would like it to return 0,33.
Something like:
select round(1/3, -2)
But that doesn't work. How can I achieve the desired result?
The suggestions from stb and xiowl are fine if you're looking for a constant. If you need to use existing fields or parameters which are integers, you can cast them to be floats first:
SELECT CAST(1 AS float) / CAST(3 AS float)
or
SELECT CAST(MyIntField1 AS float) / CAST(MyIntField2 AS float)
Because SQL Server performs integer division. Try this:
select 1 * 1.0 / 3
This is helpful when you pass integers as params.
select x * 1.0 / y
It's not necessary to cast both of them. Result datatype for a division is always the one with the higher data type precedence. Thus the solution must be:
SELECT CAST(1 AS float) / 3
or
SELECT 1 / CAST(3 AS float)
use
select 1/3.0
This will do the job.
I understand that CASTing to FLOAT is not allowed in MySQL and will raise an error when you attempt to CAST(1 AS float) as stated at MySQL dev.
The workaround to this is a simple one. Just do
(1 + 0.0)
Then use ROUND to achieve a specific number of decimal places like
ROUND((1+0.0)/(2+0.0), 3)
The above SQL divides 1 by 2 and returns a float to 3 decimal places, as in it would be 0.500.
One can CAST to the following types: binary, char, date, datetime, decimal, json, nchar, signed, time, and unsigned.
Looks like this trick works in SQL Server and is shorter (based in previous answers)
SELECT 1.0*MyInt1/MyInt2
Or:
SELECT (1.0*MyInt1)/MyInt2
Use this
select cast((1*1.00)/3 AS DECIMAL(16,2)) as Result
Here in this sql first convert to float or multiply by 1.00 .Which output will be a float number.Here i consider 2 decimal places. You can choose what you need.
If you came here (just like me) to find the solution for integer value, here is the answer:
CAST(9/2 AS UNSIGNED)
returns 5
I was surprised to see select 0.7/0.9 returning 0.8 in Teradata given they're already as floats/decimal numbers! I had to do cast(0.7 as float) to get the output that I was after.
When using literals, the best way is to "tell" SQL
which type you mean.
if you want a decimal result, add decimal point ".0" to your numbers:
SELECT 1.0 / 3.0
Result
0.333333
if you want a float (real) result, add "e0" to your numbers:
SELECT 1e0 / 3e0
Result
0.333333333333333