Similar questions have been asked before, but none have addressed why the AVG function in MSSQL produces a value different from explicit SUMand-divide when no NULL values are present in the data.
I would like to calculate the score for the data in this (simplified) TestTable.
CorrectCount IncorrectCount
5.0 0.0
3.0 2.0
5.0 0.0
4.0 0.0
3.0 0.0
5.0 0.0
2.0 1.0
5.0 0.0
5.0 0.0
2.0 2.0
The score is calculated as score = sum(CorrectCount) / sum(CorrectCount+IncorrectCount)
The following query
SELECT
AVG(CorrectCount / (CorrectCount+IncorrectCount)) as ScoreAverage,
SUM(CorrectCount) / SUM(CorrectCount+IncorrectCount) as ScoreSumDivide
FROM TestTable
produces this output:
ScoreAverage ScoreSumDivide
0.876666666666667 0.886363636363636
Where does this difference come from? What does AVG do different than the SUMand-divide? I am looking for an explanation why the result of the AVG is different from the explicit SUM(CorrectCount) / SUM(CorrectCount+IncorrectCount). I expect it is due to precision or rounding internal to the AVG function.
Probably my comment was not understood, so I am expanding it here. Assume you have Correct/Incorrect counts as:
5/2
3/1
Averaging Correct/(Correct+Incorrect) means (5/7 + 3/4)/2 = 41/56
However Sum( Correct) /sum(Correct+Incorrect) means ( 5+3 )/(7+4) = 8/11
41/56 != 8/11
If you did : Sum(Correct /(Correct+Incorrect))/Count it would be: (5/7 + 3/4)/2 = 41/56 which is equal to avg.
It is simply how the math works out. Your score average takes an average of the individual percentages. If C1, C2 … Cn is your correct scores and I1, I2 … In is your incorrect scores and “N” is the number of records then the math will look as follow:
C1/(C1+ I1) + C2 /(C2 + I2) + ... + Cn /(Cn + In)
-------------------------------------------------
N
Your sum average first sum all of your correct scores and sum all your total scores and then calculate the percentage ratio. The math of this look as follow:
C1 + C2 + ... + Cn
----------------------------------------------
(C1+ I1) + (C2 + I2) + ... + (Cn + In)
Both numbers are meaningful but the second will better reflect what the percentage of correct counts where for the entire data set.
I think you are simply calculating different stuff there. The aquivalent for the AVG should be SUM(CorrectCount / (CorrectCount + IncorrectCount)) / COUNT(*).
SELECT
AVG(CorrectCount / (CorrectCount+IncorrectCount)) as ScoreAverage,
SUM(CorrectCount / (CorrectCount + IncorrectCount)) / COUNT(*) ScoreSumDivide
Your average calculation wants modification;
,AVG(CorrectCount) / (AVG(CorrectCount)+AVG(IncorrectCount)) as ScoreAverage
This returns the correct value of 0.886363 (39 / 44) rather than what looks like a rounding issue without it.
Related
I have a rounding discrepancy in SQL that I could do with a hand resolving.
I have 2 SQL calculations, the first one equals 1.1 and the second 5.65 (see below)
round((sum((monthly_markup)+100) / 100) / sum(monthly_qty),2) as timesby, --equals 1.1
sum(monthly_buy)/sum(monthly_qty) as buy, -- equals 5.65
If I then take those calculations and do calc1 x calc2 it equals 6.21
cast (round(sum(monthly_buy)/sum(monthly_qty) * (sum((monthly_markup)+100) / 100) / sum(monthly_qty),2) as decimal (30,2)), -- equals 6.21 !!
But I am expecting 6.22, as per the below calculation
cast (round((5.65 * 1.1),2) as decimal (30,2)) -- equals 6.22
How can I get my calculation to return 6.22?
Thanks
In case it helps anyone else, I resolved by casting the sum of buy * qty to decimal first.
round((cast(sum(monthly_buy)/sum(monthly_qty) as decimal (5,3)) * round((sum((monthly_markup)+100) / 100) / sum(monthly_qty),2)),2)
I have the following query:
DECLARE #A as numeric(36,14) = 480
DECLARE #B as numeric(36,14) = 1
select #B/#A
select cast(#B as decimal)/cast(#A as decimal)
Why does the first calculation returns 0.002083 and the second one returns 0.00208333333333333?
Isn´t numeric(36,14) good enough to have a good precision (just as the second query)?
If I use only numeric, instead of numeric(36,14), I have a good precision again:
select cast(#B as numeric)/cast(#A as numeric)
You can calculate precision and scale by yourself using this documentation from SQL Server Books online.
I tried to calculate precision and scale for your case (operation=division, p=36, s=14) and I got a pretty strange results...
precision of the result: [p1 - s1 + s2 + max(6, s1 + p2 + 1)] -> 36-14+14+max(6,14+36+1)=36+51=87
scale of the result : [max(6, s1 + p2 + 1)] -> max(6,14+36+1)=51
In this situation precision is greater than 38 and in this case (as stated in the documentation)
*The result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, the corresponding scale is
reduced to prevent the integral part of a result from being truncated.
scale must be reduced by (87-38=) 49, that is (51-49=) 2 ...
I think that minimum scale length is 6 (because of expression scale=[max(6, s1 + p2 + 1)]) and it can't be reduced lower than 6 - that we have as a result (0.002083).
Just contributing for the understanding of the problem (going deeper on #Andrey answer), the things could be tricky, depending on the order of calculations.
Consider the variables:^
DECLARE #A as NUMERIC(36,19) = 100
DECLARE #B as NUMERIC(36,19) = 480
DECLARE #C as NUMERIC(36,19) = 100
Calculating A/B*C
If you want to calculate A/B*C, using the formulaes, we have:
A/B is of type NUMERIC(38,6) --> as calculated by #Andrey
The result will be 0.208333 (with scale of 6)
Multiplying by 100, we will get 20.833300
Calculating A*C/B
The result of A*C is 10000 of type NUMERIC(38,6). Diving by C, the result will be 20.833333 of type NUMERIC(38,6)
Then, the result may vary depending on the order of calculation (the same problem was pointed in https://dba.stackexchange.com/questions/77664/how-does-sql-server-determine-precision-scale).
I am trying to do the following operation in SQL to calculate the weighted rating :
SELECT CTE_3.idProduct,(CTE_3.vote_count / (CTE_3.vote_count + #minimumVotesRequired)) * CTE_3.vote_mean + (#minimumVotesRequired / (CTE_3.vote_count+ #minimumVotesRequired)) * ((SUM(CTE_3.vote_mean)/COUNT(CTE_3.IdProduct))) AS WeightedRating
FROM CTE_3
GROUP BY CTE_3.IdProduct,
CTE_3.vote_count,
CTE_3.vote_mean
ORDER BY idProduct;
But the problem I am facing is that the result is ALWAYS 0.. I tried using Convert(FLOAT,operation) AS WeightedRating but still I am getting a result of 0.
When I manually perform this on a calculator it returns 2.5416666..so I am quite sure that SQL Server is not being able to manage the values I feed to the operation.
Should I do something else than cast?
The values are :
vote_count is 2
vote_mean is 2.5
#minimumVotesRequired is 1
EDIT :
Now the only value after casting everything to float is 2.5 from CTE_3.vote_mean
SELECT CTE_3.idProduct,(CONVERT(FLOAT,CTE_3.vote_count) / (CONVERT(FLOAT,CTE_3.vote_count) + #minimumVotesRequired))
* CONVERT(FLOAT,CTE_3.vote_mean) +
(#minimumVotesRequired / (CONVERT(FLOAT,CTE_3.vote_count)+ #minimumVotesRequired))
* (SUM(CONVERT(FLOAT,CTE_3.vote_mean)))/COUNT(CTE_3.IdProduct)) AS WeightedRating
FROM CTE_3
GROUP BY CTE_3.IdProduct,
CTE_3.vote_count,
CTE_3.vote_mean
ORDER BY idProduct;
Any suggestion in what am I missing?
If CTE_3.vote_count type is int or #minimumVotesRequired type is int, then you are getting the truncated int value. Make sure those are floats, or cast them as floats before doing your division.
Also, don't forget that COUNT is an integer function. You will want to cast the result of your COUNT as a float as well.
I am trying to calculate the monthly payment of a loan and it always comes out wrong.
The formula is as follows where i is interest
((1 + i)^months /
(1 + i)^months - 1)
* principal * i
Assuming that annual interest rate and principal is an invisible floating point, can you tell me what's wrong with my formula?
double calculatePaymentAmount(int annualInterestRate,
int loanSize,
int numberOfPayments;
{
double monthlyInterest = annualInterestRate / 1200.0;
return
(
pow(1 + monthlyInterest, numberOfPayments) /
(pow(1 + monthlyInterest, numberOfPayments) - 1)
)
* (loanSize / 100)
* monthlyInterest;
}
For example: an interest rate of 1.25 and a loan size of 250 for 12 months gives 22.27 instead of 20.97.
Thank you in advance.
Edit 1: Changed monthly interest to annualInterestRate / 1200
Assuming annualInterestRate is in percent, then you should calculate monthlyInterest like this:
double monthlyInterest = pow(1+(double)annualInterestRate / 100, 1/12.0) - 1.0;
converting
double monthlyInterest = (double)annualInterestRate /
1200 / 100;
to
double monthlyInterest = (double)annualInterestRate / 12.0;
would do the trick.
you may read more about operator precedence in c from http://en.cppreference.com/w/c/language/operator_precedence
I found what was wrong. monthlyInterest = annualInterestRate / 1200.0 / 100
In C , how do I make 1200 / 500 = 3.
I'm doing a homework assignment.
Shipping Calculator: Speedy Shipping company will ship your package based on how much it weighs and how far you are sending the package. They will only ship small packages up to 10 pounds. You need to have a program that will help you determine how much they will charge. The charges are based on each 500 miles shipped. They are not pro-rated, i.e., 600 miles is the same charge as 900 miles.
Here is the table they gave you:
Package Weight--------------------------Rate per 500 miles shipped
2 pounds or less------------------------$1.50
More than 2 but not more than 6---------$3.70
More than 6 but not more than 10--------$5.25
Here is one test case.
Test Case Data:
Weight: 5.6 pounds
Miles: 1200 miles
Expected results:
Your shipping charge is $11.10
My answer keeps coming out to 7.40
Are you trying to round up? Before dividing, you could add 499 to the number that is being divided.
(0 + 499) / 500 -> 0
(1 + 499) / 500 -> 1
(1200 + 499) / 500 -> 3
This will round up.
Say you want to get a ceiling division a by b (in your example a = 1200 b = 500).
You can do it in integer arithmetic like this.
result = (a + b - 1) / b;
Or you could use floating point numbers and do it like this (probably a bad idea)
result = (int) ceil( (double) a / b );
The thing is that as this is a homework, you could just make it up in small steps:
if( a % b == 0 ) {
result = a / b;
} else {
result = a / b + 1;
}
Another advantage of this code is that it actually doesn't overflow for too big as, but this is not relevant in this case, I guess.
I'd suggest using the mod and truncate functions. If mod comes out zero, it's fine, otherwise truncate and add 1.
You have to use the ceiling of the division. This will round the quotient up to the next integer.
So when you are trying to find the number of 500-mile increments, you have to round the quotient up to the next integer.
Alternatively, (and inefficiently), you could increment the number of miles by 1, until it is divisible by 500...that is, while ( (q = x_miles++%500) != 0 ) {} . Then multipy q by the rate to get your answer (That is also assuming you will have an integer number of miles).
You could also use the stdlib div function. This might be nice if you only wanted integer math and specifically wanted to avoid floating point math.
http://www.cplusplus.com/reference/clibrary/cstdlib/div/
#include <stdlib.h>
int foo(void)
{
div_t result = div(1200, 500);
return result.quot + (0 < result.rem);
}
[EDIT1]
From your code you would implement this part as follows:
if ( weight <= 5.6 )
{
int multiplier = (int) miles / 500;
if( ((int)miles % 500) > 0)
multiplier++;
rate370 = (double)multiplier * 3.7;
printf("Your total cost : %.2lf\n", rate370);
}
[ORIGINAL]
In "integer land" 1200 / 3 should equal to 2.
for what it "seems" you want try this:
int multFiveHundreds = (int)totalWeight / 500;
if(multFiveHundreds % 500 > 0)
multFiveHundreds++;