Trying to calculate this in SQL Server.
Where c=85, z=4, d=6, and e=4
I tried this
select 85 / 4 * (6 * 4)
but looks like this is wrong any suggest please
Use POWER to handle the exponent:
SELECT 85.0 / (4 * POWER(6, 4))
or
SELECT CAST(85 AS DECIMAL(10, 4)) / (4 * POWER(6, 4))
Or in variable format:
SELECT c / (z * POWER(d, e))
select 85.0 / 4 * power(6,4)
slightly ambiguous formula, does it mean C divided by z then multiplied by d^e - anyway - use SQL x^y = POWER(x,y) and also put 85.0 to make sure floats are used
Related
I have a rounding discrepancy in SQL that I could do with a hand resolving.
I have 2 SQL calculations, the first one equals 1.1 and the second 5.65 (see below)
round((sum((monthly_markup)+100) / 100) / sum(monthly_qty),2) as timesby, --equals 1.1
sum(monthly_buy)/sum(monthly_qty) as buy, -- equals 5.65
If I then take those calculations and do calc1 x calc2 it equals 6.21
cast (round(sum(monthly_buy)/sum(monthly_qty) * (sum((monthly_markup)+100) / 100) / sum(monthly_qty),2) as decimal (30,2)), -- equals 6.21 !!
But I am expecting 6.22, as per the below calculation
cast (round((5.65 * 1.1),2) as decimal (30,2)) -- equals 6.22
How can I get my calculation to return 6.22?
Thanks
In case it helps anyone else, I resolved by casting the sum of buy * qty to decimal first.
round((cast(sum(monthly_buy)/sum(monthly_qty) as decimal (5,3)) * round((sum((monthly_markup)+100) / 100) / sum(monthly_qty),2)),2)
Following batch statement if i execute in SSMS giving me Result like given below
DECLARE #M_TUBE_VOLUME NUMERIC(38,10),
#M_TUBE_OD NUMERIC(38,10)=12.50000,
#M_TUBE_ID NUMERIC(38,10)=12.50000,
#M_TUBE_LEN NUMERIC(38,10)=4000.00000,
#M_TUBE_COUNT NUMERIC(38,10)=212.4215000,
#M_S_TUBE_LEN NUMERIC(38,10)=0.0000,
#M_S_TUBE_COUNT NUMERIC(38,10)=3587.000
SET #M_TUBE_VOLUME=(SELECT 3.141592 / 4 * #M_TUBE_OD * #M_TUBE_ID * ((#M_TUBE_LEN * #M_TUBE_COUNT) + (#M_S_TUBE_LEN * #M_S_TUBE_COUNT)));
SELECT #M_TUBE_VOLUME
RESULT -- 104272138.7104680000
and if i execute same thing in SSMS using select statement
SELECT 3.141592 / 4 * 12.50000 * 12.50000 * ((4000.00000 * 212.4215000) + (0.0000 * 3587.000))
RESULT -- 104272138.285625000000000000000
Why two result are different any reason please help me
it has something to do with rounding. As your computation involved numbers with decimal places.
Please refer to documentation Precision, scale, and Length (Transact-SQL)
I have extracted the relevant portion here
Operation Result precision Result scale *
e1 * e2 p1 + p2 + 1 s1 + s2
e1 / e2 p1 - s1 + s2 + max(6, s1 + p2 + 1) max(6, s1 + p2 + 1)
The result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, it's reduced to 38, and the
corresponding scale is reduced to try to prevent truncating the
integral part of a result. In some cases such as multiplication or
division, scale factor won't be reduced, to maintain decimal
precision, although the overflow error can be raised.
In your second query, numbers like 3.141592 is taking as numeric(7,6) and 12.50000 is numeric(7,5) and in your first query, all those are numeric(38,10)
The rounding occurs on the first 4 expression.
I use SELECT INTO a temp table and then query the data type of the column in the temp table
SELECT f1 = 3.141592 / 4 * #M_TUBE_OD * #M_TUBE_OD,
f2 = 3.141592 / 4 * 12.50000 * 12.50000
INTO #t
select c.name, t.name, c.precision, c.scale
from tempdb.sys.columns c
inner join master.sys.systypes t on c.system_type_id = t.xtype
where object_id = object_id('tempdb..#t')
name name percision scale
f1 numeric 38 6
f2 numeric 25 18
You can see that for your first query, the result scale is reduced to only 6. So for first query, the result of the first 4 expression is 122.718438 (rounded to 38) rather than 122.7184375.
You over define the precision of your numeric. Reduce it and try
In the second version, you haven't explicitly defined your data type and it has defaulted to FLOAT.
You can verify this by changing your query to
DECLARE #M_TUBE_VOLUME FLOAT,
#M_TUBE_OD FLOAT=12.50000,
#M_TUBE_ID FLOAT=12.50000,
#M_TUBE_LEN FLOAT=4000.00000,
#M_TUBE_COUNT FLOAT=212.4215000,
#M_S_TUBE_LEN FLOAT=0.0000,
#M_S_TUBE_COUNT FLOAT=3587.000
SET #M_TUBE_VOLUME=(SELECT 3.141592 / 4 * #M_TUBE_OD * #M_TUBE_ID * ((#M_TUBE_LEN * #M_TUBE_COUNT) + (#M_S_TUBE_LEN * #M_S_TUBE_COUNT)));
SELECT #M_TUBE_VOLUME
Why doesn't this
select round(17 * .235, 2), round(17 * (1-.235), 2)
return
4.00 and 13.00
instead of
4.00 and 13.01
Also,
select CONVERT(Decimal(19,2), 17 * .235), CONVERT(Decimal(19,2), 17 * (1-.235))
returns
4.00 and 13.01
How can I force the two calculations to add up to the starting number (17.00)?
select cast(17 * .235 as float)
,cast(17 * (1-.235) as float)
,cast(17 * .235 as float)+cast(17 * (1-.235) as float)
Returns
(No column name) (No column name) (No column name)
3.995 13.005 17
If you don't like float, try decimal (some folks hate float, me not so much)
select cast(17 * .235 as decimal(18,9))
,cast(17 * (1-.235) as decimal(18,9))
,cast(17 * .235 as decimal(18,9))+cast(17 * (1-.235) as decimal(18,9))
I think the answer is that SQL rounds AWAY FROM ZERO. Which is different from VB's rounding (ROUND TOWARDS EVEN). At least that's the way I read this:
http://blogs.lessthandot.com/index.php/DataMgmt/DataDesign/sql-server-rounding-methods/
Let's have a look at the picture.
The result is different even though the expression is the same.
Why does this happen?
I have to follow excel result, what should I have do with sql server?
No matter whatever the software is 1+1 will always yeild 2 and if its not you should check you calculation again. see below
SELECT ((4972000.0000) * (1.0000 - 4.4000/100.0000))
/ ((1.0000 + ((36.0000/365.0000)) * (13.0000 / 100.0000)))
RESULT: 4693057.996104
To get the result on upto four decimal places Use ROUND() function.
SELECT ROUND(((4972000.0000) * (1.0000 - 4.4000/100.0000))
/ ((1.0000 + ((36.0000/365.0000)) * (13.0000 / 100.0000))), 4)
RESULT: 4693057.996100
I am trying to do the following operation in SQL to calculate the weighted rating :
SELECT CTE_3.idProduct,(CTE_3.vote_count / (CTE_3.vote_count + #minimumVotesRequired)) * CTE_3.vote_mean + (#minimumVotesRequired / (CTE_3.vote_count+ #minimumVotesRequired)) * ((SUM(CTE_3.vote_mean)/COUNT(CTE_3.IdProduct))) AS WeightedRating
FROM CTE_3
GROUP BY CTE_3.IdProduct,
CTE_3.vote_count,
CTE_3.vote_mean
ORDER BY idProduct;
But the problem I am facing is that the result is ALWAYS 0.. I tried using Convert(FLOAT,operation) AS WeightedRating but still I am getting a result of 0.
When I manually perform this on a calculator it returns 2.5416666..so I am quite sure that SQL Server is not being able to manage the values I feed to the operation.
Should I do something else than cast?
The values are :
vote_count is 2
vote_mean is 2.5
#minimumVotesRequired is 1
EDIT :
Now the only value after casting everything to float is 2.5 from CTE_3.vote_mean
SELECT CTE_3.idProduct,(CONVERT(FLOAT,CTE_3.vote_count) / (CONVERT(FLOAT,CTE_3.vote_count) + #minimumVotesRequired))
* CONVERT(FLOAT,CTE_3.vote_mean) +
(#minimumVotesRequired / (CONVERT(FLOAT,CTE_3.vote_count)+ #minimumVotesRequired))
* (SUM(CONVERT(FLOAT,CTE_3.vote_mean)))/COUNT(CTE_3.IdProduct)) AS WeightedRating
FROM CTE_3
GROUP BY CTE_3.IdProduct,
CTE_3.vote_count,
CTE_3.vote_mean
ORDER BY idProduct;
Any suggestion in what am I missing?
If CTE_3.vote_count type is int or #minimumVotesRequired type is int, then you are getting the truncated int value. Make sure those are floats, or cast them as floats before doing your division.
Also, don't forget that COUNT is an integer function. You will want to cast the result of your COUNT as a float as well.