In using the T-SQL ROUND function I noticed what seems like weird behavior. It looks like the ROUND function only looks at the first digit to the right of the digit to be rounded. If I round -6.146 to one decimal I get -6.1. I would have thought it would start at the right and round each digit as it works its way to the left, like this: -6.146 -> -6.15 -> -6.2
I've observed the same behavior with Excel’s round function too.
The query below illustrates what I am describing. I may simply use the nested ROUND functions as shown below but I'm curious if there’s a better way and which approach is considered mathematically correct.
DECLARE #Num AS FLOAT
SET #Num = -6.1463
SELECT #Num [OriginalVal], ROUND(#Num, 1, 0) [SingleRound]
, ROUND(ROUND(ROUND(#Num, 3, 0), 2, 0), 1, 0) [NestedRound]
Results
OriginalVal | SingleRound | NestedRound
-6.1463 | -6.1 | -6.2
I think the basic rule of thumb is, in rounding, you look at the 1 digit immediately to the right of the place you are rounding to. You do not extend it all the way to the very end of the right of the decimal.
http://math.about.com/od/arithmetic/a/Rounding.htm
Related
Consider the overly simplistic example: POINT(0 0) and LINESTRING (1 -10, 1 10)
The closest point on the line to the POINT would be 1, 0.
How would one determine this in TSQL? My simple, not entirely accurate, approach was to make a linestring (POINT POINT) and extend out the X coord of one coords until the two linestrings intersected.
So:
linestring (0 0, 0.25 0) (no intersect)
linestring (0 0, 0.5 0) (no intersect)
linestring (0 0, 0.75 0) (no intersect)
linestring (0 0, 1 0) (intersection - so 1 0 is the point closest to POINT
This quasi worked, but doesn't seem to the most bestest/more performant way of accomplishing this.
For example, one inefficiency is that I move it one direction (positive increments), and if there was no match (after x attempts), then I would start over, but with negative increments.
To optimize, I tried moving in larger steps, then when intersected (probably went past the point), I backed off 1 increment and started from there with a smaller increment. I did this a couple of times - instead of going in tiny tiny increments so as not to overshoot by too much.
One acceptable assumption based on my processing that the POINT will be next to (left/right) of the LINESTRING.
Another acceptable assumption is that the LINESTRING will be fairly "perpendicular" to the POINT.
I think you can do this mathematically rather than with a brute-force iterative algorithm.
There is a post to get closest point to a line that describes the method.
I converted this method to SQL which returns the correct value (1,0). Your 'trivial' example is actually a bit of an edge case (vertical line with infinite slope) so it seems robust.
I also tested the source code with this example: https://www.desmos.com/calculator/iz07az84f5 and using the input for the line of (-1,2) (3,0) and a point at (2,2) got the correct answer (1.4, 0.8).
SQL code (also in SQL Fiddle at http://sqlfiddle.com/#!6/d87aa/15)
DECLARE #x int, #y int, #x1 int, #y1 int, #x2 int, #y2 int
DECLARE #atb2 float, #atp_dot_atb float
DECLARE #t float
--SELECT #x=0, #y=0
--SELECT #x1=1, #y1=10, #x2=1, #y2=-10
SELECT #x=2, #y=2
SELECT #x1=-1, #y1=2, #x2=3, #y2=0
SELECT #atb2 = SQUARE(#x2-#x1) + SQUARE(#y2-#y1) -- Basically finding the squared magnitude of a_to_b
SELECT #atp_dot_atb = (#x-#x1)*(#x2-#x1) + (#y-#y1)*(#y2-#y1) -- The dot product of a_to_p and a_to_b
SELECT #t = #atp_dot_atb / #atb2 -- The normalized "distance" from a to your closest point
SELECT #x1 + (#x2-#x1)*#t, #y1 + (#y2-#y1)*#t --Add the distance to A, moving towards B
I am looking for a division result that is extremely accurate.
This SQL returns the following results:
SELECT (CAST(297282.26 AS DECIMAL(38, 30)) / CAST(495470.44 AS DECIMAL(38, 30))) AS ResultDecimal
SELECT (CAST(297282.26 AS FLOAT) / CAST(495470.44 AS FLOAT)) AS ResultFloat
Here is the accurate result from WolframAlpha:
http://www.wolframalpha.com/input/?i=297282.26%2F495470.44
I was under the impression that DECIMAL would be more accurate than FLOAT:
"Because of the approximate nature of the float and real data types, do not use these data types when exact numeric behavior is required, such as in financial applications, in operations involving rounding, or in equality checks. Instead, use the integer, decimal, money, or smallmoney data types."
https://technet.microsoft.com/en-us/library/ms187912(v=sql.105).aspx
Why does the FLOAT calculation give me a result more accurate than when using DECIMAL?
I found the best precision to be when you use:
SELECT (CAST(297282.26 AS DECIMAL(15, 9)) / CAST(495470.44 AS DECIMAL(24, 2))) AS ResultDecimal
This gives a result of
0.599999991926864496699338915153
I think the actual value (to 100 digits) is:
0.5999999919268644966993389151530412187657451370862810705720405842980259326873264124495499670979362562...
Please bear in mind SQL Server defines the maximum precision and scale for division as:
max precision = (p1 - s1 + s2) + MAX(6, s1 + p2 + 1) -- up to 38
max scale = MAX(6, s1 + p2 + 1)
Where p1 & p2 are the precision of the two numbers and s1 & s2 are the scale of the numbers.
In this case the maximum precision is (15-9+2) + MAX(6, 9+24+1) = 8 + 34 = 42.
However SQL Server only allows a maximum precision of 38.
The maximum scale = MAX(6, 9+24+1) = 34
Hopefully you already understand that just because the FLOAT version presents more numbers after the decimal point, doesn't necessarily mean that those are the true numbers. This is about precision, not accuracy.
It is the CAST function itself that causes this loss of precision, not the difference between the FLOAT and DECIMAL data types.
To demonstrate this, compare your previous results to the result of this:
SELECT 297282.26 / 495470.44 AS ResultNoCast
In my version of the query, the presence of a decimal point in the literal numbers tells SQL Server to treat the values as DECIMAL datatype, with appropriate length and precision as determined by the server. The result is more precise than when you CAST explicitly to DECIMAL.
A clue to the reason for this can be found hidden in the official documentation of the CAST function, under Truncating and Rounding Results:
When you convert data types that differ in decimal places, sometimes the result value is truncated and at other times it is rounded. The following table shows the behavior.
From | To | Behavior
numeric | numeric | Round
So the fact that each separate literal value is treated as a NUMERIC (same thing as DECIMAL) on the way in, and is being casted to NUMERIC, causes rounding.
Anticipating your next question a little, if you want a more precise result from the NUMERIC/DECIMAL datatype, you just need to tell SQL Server that each component of the calculation is more precise:
SELECT 297282.26000000 / 495470.44000000 AS ResultSuperPrecise
This appears (from experimentation) to be the most precise I can get: either adding or removing a 0 from either the numerator or denominator makes the result less precise. I'm at a loss to explain why that is, because the result is only 23 digits to the right of the decimal point.
It doesn't give you a more accurate result. I say that because the value is an approximate and not all values will be available to stored in a float. On the other side of that coin though is that float has the possibility of a lot more precision. The maximum precision of a decimal/numeric is 38. https://msdn.microsoft.com/en-us/library/ms187746.aspx
When you look at float though the maximum precision is 53. https://msdn.microsoft.com/en-us/library/ms173773.aspx
Okay, here is what I think is going on.
#philosophicles - I think you are right in that the CAST is causing the problem, but not because I am trying to "convert data types that differ in decimal places".
When I execute the following statement
SELECT CAST((297282.26 / 495470.44) AS DECIMAL(38, 30)) AS ResultDecimal
The accurate result for the calculation is
This has way more than 30 digits after the decimal point, and my data type has scale set to 30. So the CAST rounds the value, then just adds zeros to the end until there are 30 digits. We end up with this:
So the interesting thing is how does the CAST determine up to how many decimals to round or truncate the output? I am not sure, but as #philosophicles pointed out, the scale of the input effects the rounding applied on the output.
SELECT CAST(((297282.26/10000) / (495470.44/10000)) AS DECIMAL(38, 30)) AS ResultDecimal
Thoughts?
Also interesting:
However, in simple terms, precision is lost when the input scales are
high because the result scales need to be dropped to 38 with a
matching precision drop.
https://dba.stackexchange.com/questions/41743/automatic-decimal-rounding-issue
The precision and scale of the numeric data types besides decimal are fixed.
https://dba.stackexchange.com/questions/41743/automatic-decimal-rounding-issue
I was trying to round some fields. When I have 59 days, I want to change it to 60.
The problem is that when I use this code, the 59 is changed to 30 because the round it is 1.
select round(1.9666,0)*30, round(59/30,0)*3'
The result of that query is 60 for the first field and 30 for the second one. The problem is that when I've tried:
select 59/30
The result is 1 and I need the entire answer that is 1.9666...
How can I make it?
Because the number you are dividing by is an INT (the data type of the left side is irrelevant), SQL Server will return an INT as the answer.
If you want a number with a decimal place as your result, you'll need to divide by one.
Don't cast to a FLOAT as the answer is probably not what you want (floats are generally not accurate and are 'approximations'):
SELECT 59 / CAST(30 AS FLOAT) -- = 1.96666666666667
CAST the right-hand side of the division to a DECIMAL:
SELECT 59 / CAST(30 AS DECIMAL(10, 2)) -- = 1.96666
SELECT cast(59 AS FLOAT) / cast(30 AS FLOAT)
Because the original figures are whole numbers, SQL presumes you want a whole number output.
To ensure you get one with the decimal places, you need to first change the data type from an integer int to a floating point float.
This is what the CAST command does.
EDIT: Commenter suggests you cast to DECIMAL instead. The principle is the same, but you need to supply more arguments. To cast to a decimal use something like:
cast(59 as DECIMAL(18, 3))
The first argument (the 18) is the total number of figures you want to permit in the decimal. The second argument (the 3) is the number you want after the decimal point.
The suggestion that it's more accurate is correct - as you'll see if you run the SELECT statements in this answer one after the other. But in this particular case, it only makes a tiny difference.
I have really simply question about DECIMAL (and maybe NUMERIC) type in SQL Server 2008 R2.
MSDN said:
(scale)
The maximum number of decimal digits that can be stored to the right of the decimal point. Scale must be a value from 0 through p.
I understand this following way:
if I have DECIMAL(10, 5) - I am able to store 12345.12345 or 12345678.91.
if I have DECIMAL(5, 5) - I can have 12345 or 1234.5 or 1.2345, etc...
Is it clear?
But I got this error message:
SELECT CAST(2.8514 AS DECIMAL(5,5))
Arithmetic overflow error converting numeric to data type numeric.
I thought 5,5 means I can have up to 5 digits and up to 5 CAN BE right of the decimal point.
As I tried:
SELECT CAST(12.851 AS DECIMAL(6,5)) - overflows too
however
SELECT CAST(1.23456 AS DECIMAL(6,5)) - is OK.
So what's the truth?
DECIMAL(a,b) says that I can have up to a digits and JUST b of them are right to the decimal point (and there rest a-b to the left to the dec. point)?
I'm really confused about statement in doc which is copied everywhere. Please take a while and explain me this simple thing.
Lot of thanks!
The easiest way to think of it (for me) is that precision is the total number of digits, of which scale is the number of digits to the right of the decimal point. So DECIMAL(p,s) means p-s digits to the left of the point, and s digits to the right of the point.
That explains all the conversion errors you're seeing: the 2.8514 cannot be decimal(5,5) because p-s = 0; 12.851 cannot be decimal(6,5) because p-s = 1 and so on.
I use SQL Server 2005 and need to test whether values in a column that's metadata has been specified as DECIMAL(18.3) actually contains data that has values to the right of the Decimal point, and if so, what these values are.
I've read a few articles that only discuss how to drop off the decimal places or how to round the values, but not how to ONLY display what is stored to the right of the decimal point.
Your help would be greatly appreciated.
Kind Regards,
Ignacio.
Try:
SELECT a - FLOOR(a)
FROM ...
SELECT decimalnumber - FLOOR(decimalnumber) AS decimalpart
FROM mytable
WHERE decimalnumber - FLOOR(decimalnumber) > 0
This may not always work the way you expect it to. The problem occurs when you have negative numbers. You can think of FLOOR as a type of rounding, where it always rounds down to the next whole number. Floor(3.14) = 3, and Floor(-3.14) = -4.
To get the value of a number after the decimal point, you can use the ParseName function, which will work for positive and negative numbers.
Select ParseName(-3.9876, 1)
Select ParseName(-3.1234, 1)
Select ParseName(3.9876, 1)
Select ParseName(3.1234, 1)