Why doesn't this
select round(17 * .235, 2), round(17 * (1-.235), 2)
return
4.00 and 13.00
instead of
4.00 and 13.01
Also,
select CONVERT(Decimal(19,2), 17 * .235), CONVERT(Decimal(19,2), 17 * (1-.235))
returns
4.00 and 13.01
How can I force the two calculations to add up to the starting number (17.00)?
select cast(17 * .235 as float)
,cast(17 * (1-.235) as float)
,cast(17 * .235 as float)+cast(17 * (1-.235) as float)
Returns
(No column name) (No column name) (No column name)
3.995 13.005 17
If you don't like float, try decimal (some folks hate float, me not so much)
select cast(17 * .235 as decimal(18,9))
,cast(17 * (1-.235) as decimal(18,9))
,cast(17 * .235 as decimal(18,9))+cast(17 * (1-.235) as decimal(18,9))
I think the answer is that SQL rounds AWAY FROM ZERO. Which is different from VB's rounding (ROUND TOWARDS EVEN). At least that's the way I read this:
http://blogs.lessthandot.com/index.php/DataMgmt/DataDesign/sql-server-rounding-methods/
Related
I have a rounding discrepancy in SQL that I could do with a hand resolving.
I have 2 SQL calculations, the first one equals 1.1 and the second 5.65 (see below)
round((sum((monthly_markup)+100) / 100) / sum(monthly_qty),2) as timesby, --equals 1.1
sum(monthly_buy)/sum(monthly_qty) as buy, -- equals 5.65
If I then take those calculations and do calc1 x calc2 it equals 6.21
cast (round(sum(monthly_buy)/sum(monthly_qty) * (sum((monthly_markup)+100) / 100) / sum(monthly_qty),2) as decimal (30,2)), -- equals 6.21 !!
But I am expecting 6.22, as per the below calculation
cast (round((5.65 * 1.1),2) as decimal (30,2)) -- equals 6.22
How can I get my calculation to return 6.22?
Thanks
In case it helps anyone else, I resolved by casting the sum of buy * qty to decimal first.
round((cast(sum(monthly_buy)/sum(monthly_qty) as decimal (5,3)) * round((sum((monthly_markup)+100) / 100) / sum(monthly_qty),2)),2)
When I run this simple operation in SQL server:
Select 800.0 /30.0
I get the value 26.666666, where even if it rounds for 6 digits it should be 26.666667.
How can I get the calculation to be accurate? I tried to search about it online and I found a solution where I cast each operand to a high precision decimal before the operation, but this will not be convenient for me because I have many long complex calculations. think there must be a better solution.
When a using division, in SQL Server, any digits after the resulting scale are truncated, not rounded. For your expression you have a decimal(4,1) and a decimal(3,1), which results in a decimal(10,6):
Precision = p1 - s1 + s2 + max(6, s1 + p2 + 1)
Scale = max(6, s1 + p2 + 1)
As a result, 26.66666666666666~ is truncated to 26.666666.
You can get around this by can increasing the size of the precision and scale, and then CONVERT back to your required precision and scale. For example, increase the precision and scale of the decimal(3,1) to decimal(5,2) and convert back to a decimal(10,6):
SELECT CONVERT(decimal(10,6),800.0 / CONVERT(decimal(5,3),30.0));
This returns 26.666667.
This might helpful:
Use ROUND (Transact-SQL)
SELECT ROUND(800.0 /30.0, 5) AS RoundValue;
Result:
RoundValue
26.666670
I believe it's because SQL Server takes your numbers as decimal values (which are exact e.g., 6.6666 and 6.6667 means exactly those values, not 6 and two-thirds) rather than float values (which can work with approximate numbers).
If you explicity cast/convert it to a float at the start, you should get your calculations running smoothly.
Here's some examples to demonstrate the difference between int, decimal, and float calculations
Dividing 20 by 3
Dividing 20 by 3, then multiplying by 3 again (which mathematically should be 20).
SELECT (20/3) AS int_calc,
(20/3) * 3 AS int_calc_x3,
(CAST(20 AS decimal(10,3)) /3) AS dec_calc,
(CAST(20 AS decimal(10,3)) /3) * 3 AS dec_calc_x3,
(CAST(20 AS float) /3) AS float_calc,
(CAST(20 AS float) /3) * 3 AS float_calc_x3
with the following results
int_calc int_calc_x3 dec_calc dec_calc_x3 float_calc float_calc_x3
6 18 6.666666 19.999998 6.66666666666667 20
In your case, you can use
Select CAST(800.0 AS float) /30.0
which results in 26.6666666666667
Note if you then multiply back by 30, it gets the correct result e.g.,
Select (CAST(800.0 AS float) /30.0) * 30
results in 800. Solutions dealing with decimals will not have this.
Note also that once you have it as a float, then it should stay a float until converted back to a decimal or an int somehow (e.g., saved in a table as an int). So...
SELECT A.Num / 30
FROM (Select ((CAST(800.0 AS float) /30.0) * 30) AS Num) AS A
will still result in 26.6666666666667
This will hopefully help you in your long complex calculations.
Sample data:
1) 0.02500 = 25
2) 1.0000=100
I tried below code but it's converting 0.25 to 30
Select Try_cast(workcountry as decimal (10,2)*100
Multiply first, then cast. This should give you your an expected behavior.
SELECT TRY_CAST(0.02500*100 AS decimal(10,2))
Your problem is the rounding, 0.025 with 2 decimal places rounds to 0.03. You then multiply that by 100 to get 3. Add an extra decimal place and you'll be fine
select try_cast(0.02500 as decimal (10,3)) * 100
You can try this:
declare #workcountry float = 0.025;
select try_cast(#workcountry * 100. as decimal (10,2))
I have a query:
use [Some_Database];
go
select toip 10 IDEVENT,
COUNT(*)
from [AL_PROTOCOL]
group by IDEVENT
order by Quanity desc
go
In output I get results as
IDEVENT Quanity
664 4,037787E+07
566 2,124254E+07
438 1,248467E+07
294 9926404
564 9777449
436 5784661
310 5709771
428 5161083
432 5154893
434 5150308
So, then I try to calculate second (*60 / 1,000,000) column using CONVERT (real (100), COUNT(*)*60/1000000, 2) as 'Size, Mb'
next way:
use [Some_Database];
go
select top 10 IDEVENT,
COUNT(*),
CONVERT (real (100), COUNT(*)*60/1000000, 2) as 'Size, Mb'
from [AL_PROTOCOL]
group by IDEVENT
order by Quanity desc
go
After this, I get this error
8115 'Arithmetic overflow error converting expression to int type
numeric'.
Technet explains, that this kind of errors occurs when a value in the column is overgrowing the maximal size of integer type. But, why doesn't it convert in REAL type?
Also, this question solved by replacing '*60/1,000,000' on '/1,000,000*60', but my interest is not satisfied.
What am I doing wrong? thanks
Look carefully at the order of operations you have asked the DBMS to perform. CONVERT (real (100), COUNT(*)*60/1000000, 2) means:
COUNT(*)
result * 60
result / 1000000
CONVERT (real (100), result, 2)
Until step 4, the value is still an integer, so you get an overflow error if COUNT(*) * 60 is higher than the maximum representable integer.
Because * and / have the same precedence, the workaround you found is CONVERT (real (100), COUNT(*)/1000000*60, 2), which means:
COUNT(*)
result / 1000000
result * 60
CONVERT (real (100), result, 2)
We're still doing maths on integers, but now we never overflow because we divide before multiplying.
But what you actually wanted was to do all the maths on the floating point value:
COUNT(*)
CONVERT (real (100), result, 2)
result * 60
result / 1000000
For that, you just need to swap the nesting around so that you convert the COUNT(*) result directly, and then apply the maths: CONVERT (real (100), COUNT(*), 2) * 60 / 1000000
If you multiply a really large number by 60, you're going to get an even bigger number, increasing the chances of an arithmetic overflow. The fact that you divide it by 1,000,000 after is too late, you've already tried to generate a number that is too big to proceed with.
However, when you divide it by 1,000,000 first before multiplying by 60, then you are always going to get a smaller number, so you should never get an arithmetic overflow.
Also, with your CONVERT, you are doing it on the entire result, and all of the numbers in the calculation are of INT types, so it will generate an INT before the conversion.
COUNT(*)*60/1000000 -- COUNT(*) is an INT, as well as the other numbers
If you change the ordering of the CONVERT \ CAST it should work.
DECLARE #number AS INT
SET #number = 123456789
-- this doesn't work
SELECT CAST((#number * 60) AS REAL(100))
-- this does
SELECT CAST(#number AS REAL(100)) * 60
Using T-SQL and Microsoft SQL Server I would like to specify the number of decimal digits when I do a division between 2 integer numbers like:
select 1/3
That currently returns 0. I would like it to return 0,33.
Something like:
select round(1/3, -2)
But that doesn't work. How can I achieve the desired result?
The suggestions from stb and xiowl are fine if you're looking for a constant. If you need to use existing fields or parameters which are integers, you can cast them to be floats first:
SELECT CAST(1 AS float) / CAST(3 AS float)
or
SELECT CAST(MyIntField1 AS float) / CAST(MyIntField2 AS float)
Because SQL Server performs integer division. Try this:
select 1 * 1.0 / 3
This is helpful when you pass integers as params.
select x * 1.0 / y
It's not necessary to cast both of them. Result datatype for a division is always the one with the higher data type precedence. Thus the solution must be:
SELECT CAST(1 AS float) / 3
or
SELECT 1 / CAST(3 AS float)
use
select 1/3.0
This will do the job.
I understand that CASTing to FLOAT is not allowed in MySQL and will raise an error when you attempt to CAST(1 AS float) as stated at MySQL dev.
The workaround to this is a simple one. Just do
(1 + 0.0)
Then use ROUND to achieve a specific number of decimal places like
ROUND((1+0.0)/(2+0.0), 3)
The above SQL divides 1 by 2 and returns a float to 3 decimal places, as in it would be 0.500.
One can CAST to the following types: binary, char, date, datetime, decimal, json, nchar, signed, time, and unsigned.
Looks like this trick works in SQL Server and is shorter (based in previous answers)
SELECT 1.0*MyInt1/MyInt2
Or:
SELECT (1.0*MyInt1)/MyInt2
Use this
select cast((1*1.00)/3 AS DECIMAL(16,2)) as Result
Here in this sql first convert to float or multiply by 1.00 .Which output will be a float number.Here i consider 2 decimal places. You can choose what you need.
If you came here (just like me) to find the solution for integer value, here is the answer:
CAST(9/2 AS UNSIGNED)
returns 5
I was surprised to see select 0.7/0.9 returning 0.8 in Teradata given they're already as floats/decimal numbers! I had to do cast(0.7 as float) to get the output that I was after.
When using literals, the best way is to "tell" SQL
which type you mean.
if you want a decimal result, add decimal point ".0" to your numbers:
SELECT 1.0 / 3.0
Result
0.333333
if you want a float (real) result, add "e0" to your numbers:
SELECT 1e0 / 3e0
Result
0.333333333333333