I would like to understand the following T-SQL statement:
DECLARE #s NVARCHAR(16) = N'1.377532E-39';
SELECT #s AS [orig]
, CONVERT(REAL, #s) AS [real]
, CONVERT(FLOAT, #s) AS [float]
-- , CONVERT(NUMERIC(2, 0), #s) AS [numeric direct]
/* does not work: Msg 8114, Level 16, State 5 */
, CONVERT(NUMERIC(2, 0), CONVERT(FLOAT, #s)) AS [numeric via float];
My desired output is a value with NUMERIC(2,0) precision (in this case 0). Direct conversion does not work; I have to convert from NVARCHAR to FLOAT first and then from FLOAT to NUMERIC(2,0).
Direct conversion works if the string does not contain a scientific notation number. I would like to understand why? Thanks for any hint.
From what I can tell, it is simply a matter of the Float datatype being able to convert a string representing scientific notation to a number, whereas the Numeric datatype cannot. Once Float has converted a number out of scientific notation into a number (as opposed to a string that represents a number), then Numeric can handle the conversion.
In the documentation for the Float and Numeric datatypes, there is reference in both to the use of scientific notation, but this is to define the minimum and maximum values accepted by the datatype. In the Float documentation, it is stated that:
Conversion of float values that use scientific notation to decimal or
numeric is restricted to values of precision 17 digits only. Any value
with precision higher than 17 rounds to zero.
This indicates that Float can interpret scientific notation whereas Decimal and Numeric datatypes can only handle a number with a maximum precision of 17 resulting from a number specified using scientific notation.
I hope that makes sense. Leave a comment if you have any questions.
Related
I am experiencing a very strange behavior here with Microsoft SQL Server 2016 (SP2-CU15):
select convert(datetime, max(TS) + 1.0/24) as A
from table;
yields 2021-01-16 11:59:00.000
while
select convert(datetime, max(TS) + 1.0/24) as A
, dateadd(hour, 1, max(TS)) as B
from table;
gives me 2021-01-16 11:58:59.943 for A (and 2021-01-16 11:59:00.000 for B). So, it seems to me that adding the second column changes the result for the first?!
I can force the two-column version to work by casting 1.0 to real, btw: convert(datetime, max(TS) + cast(1.0 as real)/24), but I can not force the one-column version to fail by writing convert(datetime, max(TS) + cast(1.0 as float)/24).
Any ideas what's happening here?
Thanks!
Hendrik.
Update: As requested, here is a minimal example:
CREATE TABLE TestTS (TS FLOAT);
INSERT INTO TestTS (TS) VALUES (44210.4993055556);
SELECT convert(datetime, max(TS) + 1.0/24) as A
, dateadd(hour, 1, max(TS)) as B
from TestTS
As described, if you comment out the B-column, the value of A changes.
There's nothing wrong with DATEADD. The problem is the rest of the question.
First, there's a critical bug. Dates are stored as floats. An appropriate type should be used instead, eg datetime2, datetime or datetimeoffset. The best options are datetime2(0) or datetimeoffset(0), assuming no millisecond precision is needed.
datetime is essentially a legacy type, whose internal storage format is ... a float in the OADate format. That doesn't mean floats should be instead of the correct type though, no more than varbinary should be used instead of int or bigint.
Then, there's an attempt to add one hour to the OADate value, by calculating the floating point value of 1 hour in that format, 1/24. That's an irrational number though (0.04166666666....) which means that rounding errors always result in an inaccurate value.
Solution
The real solution is to use the correct type and DATEADD, eg :
CREATE TABLE TestTS (TS datetime2(0));
INSERT INTO TestTS (TS) VALUES ('2021-01-16 10:59:00.000');
SELECT dateadd(hour, 1, max(TS)) as B
from TestTS
If you want millisecond precision, use datetime2(3).
Getting the hack to work.
If you used datetime you wouldn't need to convert to datetime in the end, but the result would still be imprecise. This :
declare #TestTS table (TS datetime);
INSERT INTO #TestTS (TS) VALUES ('2021-01-16 10:59:00.000');
SELECT max(ts)+ (1.0/24)
from #TestTS
Produces 2021-01-16 11:58:59.943. The only reason the hack looked to be working in the first place was probably due to rounding errors during conversion.
The only way to get a correct result by adding floating point numbers is to increase precision to 8 fractional digits :
declare #TestTS table (TS datetime);
INSERT INTO #TestTS (TS) VALUES ('2021-01-16 10:59:00.000');
SELECT max(ts)+ (1.00000/24)--, dateadd(hour, 1, max(TS)) as B
from #TestTS
That produces 2021-01-16 11:59:00.000.
1.0 is a decimal(2,1). T-SQL calculates the fractional digits of decimal division based on the functional digits of the operands. If the operands have up to 4 fractional digits, the result will have 6 fractional digits, which isn't enough. 1 digit is added for any fractional digit above 4. 1.00000 results in 8 fractional digits 0.04166666
Don't do this though.
Cause
Thanks to #MartinSmith for the clue.
The cause is query auto-parameterization and the data types being chosen to store values.
Query 1 is auto-parameterized:
StatementText="SELECT CONVERT([datetime],MAX([TS])+#1/#2)
....
<ColumnReference Column="#2" ParameterCompiledValue="(24)" ParameterRuntimeValue="(24)" />
<ColumnReference Column="#1" ParameterCompiledValue="(1.0)" ParameterRuntimeValue="(1.0)" />
Query 2 is not auto-parameterized:
StatementText="SELECT convert(datetime, max(TS) + 1.0/24) as A...."
Why it happens is the first query and not the second query is a bit of a black magic.
From SQL Server data types page:
When you use the +, -, *, /, or % arithmetic operators to perform
implicit or explicit conversion of int, smallint, tinyint, or bigint
constant values to the float, real, decimal or numeric data types, the
rules that SQL Server applies when it calculates the data type and
precision of the expression results differ depending on whether the
query is autoparameterized or not.
Therefore, similar expressions in queries can sometimes produce
different results. When a query is not autoparameterized, the constant
value is first converted to numeric, whose precision is just large
enough to hold the value of the constant, before converting to the
specified data type. For example, the constant value 1 is converted to
numeric (1, 0), and the constant value 250 is converted to numeric (3, 0).
When a query is autoparameterized, the constant value is always
converted to numeric (10, 0) before converting to the final data
type. When the / operator is involved, not only can the result type's
precision differ among similar queries, but the result value can
differ also. For example, the result value of an autoparameterized
query that includes the expression SELECT CAST (1.0 / 7 AS float)
will differ from the result value of the same query that is not
autoparameterized, because the results of the autoparameterized query
will be truncated to fit into the numeric (10, 0) data type.
Effect
Based on the above, the following data types are used (refer to See: Precision, scale, and Length (Transact-SQL) for explanation of how result types are calculated):
Query 1 gives higher precision:
NUMERIC( 2, 1 ) / NUMERIC( 10, 0 ) = NUMERIC( 13, 12 )
Query 2:
NUMERIC( 2, 1 ) / NUMERIC( 2, 0 ) = NUMERIC( 7, 6 )
Solution
Cast your literals and / or intermediate results to the desired type to avoid surprises.
In your specific case, best solution is not to use number arithmetic to manipulate dates as Panagiotis Kanavos explains in his answer.
Alternatively, forcing float data types (per Dan Guzman comment) convert(datetime, max(TS) + 1e/24) would do the trick as well.
This question deals with the same issue.
I found weird or strange behavior of Round function in MSSQL for real column type. I have tested this issue in Azure SQL DB and SQL Server 2012
Why #number=201604.125 Return 201604.1 ?
Why round(1.12345,10) Return 1.1234500408 ?
-- For Float column it working as expected
-- Declare #number as float,#number1 as float;
Declare #number as real,#number1 as real;
set #number=201604.125;
set #number1=1.12345;
select #number as Realcolumn_Original
,round(#number,2) as Realcolumn_ROUND_2
,round(#number,3) as Realcolumn_ROUND_3
, #number1 as Realcolumn1_Original
,round(#number1,6) as Realcolumn1_ROUND_6
,round(#number1,7) as Realcolumn1_ROUND_7
,round(#number1,8) as Realcolumn1_ROUND_8
,round(#number1,9) as Realcolumn1_ROUND_9
,round(#number1,10) as Realcolumn1_ROUND_10
Output for real column type
I suspect what you are asking here is why does:
DECLARE #n real = 201604.125;
SELECT #n;
Return 201604.1?
First point of call for things like this should be the documentation: Let's start with float and real (Transact-SQL). Firstly we note that:
The ISO synonym for real is float(24).
If we then look further down:
float [ (n) ] Where n is the number of bits that are used to store the
mantissa of the float number in scientific notation and, therefore,
dictates the precision and storage size. If n is specified, it must be
a value between 1 and 53. The default value of n is 53. n value
Precision Storage size
1-24 7 digits 4 bytes
So, now we know that a real (aka a float(24)) has precision of 7. 201604.125 has a precision of 9, that's 2 too many; so off come that 2 and 5 in the return value.
Now, ROUND (Transact-SQL). That states:
Returns a numeric value, rounded to the specified length or precision.
When using real/float those digits aren't actually lost, as such, due to the floating point. When you use ROUND, you are specifically stating "I want this many decimal places". This is why you can then see the .13 and the .125, as you have specifically asked for those. When you just returned the value of #number it had a precision of 7, due to being a real, so 201604.1 was the value returned.
Round functions has two behaviors: With the value cours is equal to "3.1235", round(cours, 3) = 3.123. Although, when we replace cours by its value (3.1235) in this round formula, round(3.1235, 3) = 3.1240.
You shouldn't be using the float datatype for specific decimal values like this as it's not designed for that purpose. Would need to see more of your code to get a better context of what you're trying to do, but if it needs to be a float initially, potentially you could cast #cours as decimal?
round(cast(#cours as decimal(5,4)), 3)
Your FLOAT does not really contain 3.1235, that is only what is printed or shown in a grid. Internally the FLOAT is 3.1234999999999999, which is obviously rounded down to 3.123.
The literal 3.1235 becomes a NUMERIC with enough precision to be totally exact, and so it is rounded up to 3.124, as one would expect.
Proof:
SELECT CAST('3.1235' as FLOAT),
CAST( 3.1235 as FLOAT)
-- misleading output: both print 3.1235
SELECT CAST(CAST('3.1235' as FLOAT) as NUMERIC(24,23)),
CAST(CAST( 3.1235 as FLOAT) as NUMERIC(24,23))
-- both print 3.12349999999999990000000
SELECT CAST('3.1235' as NUMERIC(24,23)),
CAST( 3.1235 as NUMERIC(24,23))
-- both print 3.12350000000000000000000
I have the following doubt related to Microsoft SQL Server. If a bigint column has a value as 0E-9, does it mean that this cell can contain value with 9 decimal digits or what?
BIGINT: -9,223,372,036,854,775,808 TO 9,223,372,036,854,775,807
INT: -2,147,483,648 TO 2,147,483,647
SMALLINT: -32,768 TO 32,767
TININT: 0 TO 255
These are for storing non-decimal values. You need to use DECIMAL or NUMERIC to store values shuch as 112.455. When maximum precision is used, valid values are from - 10^38 +1 through 10^38 - 1.
OE-9 isn't NUMERICor INTEGER value. It's a VARCHAR unless you are meaning something else like scientific notation.
https://msdn.microsoft.com/en-us/library/ms187746.aspx
https://msdn.microsoft.com/en-us/library/ms187745.aspx
No, the value would be stored as an integer. Any decimal amounts will be dropped off and only values to the left of the decimal will be kept (no rounding).
More specifically, bigint stores whole numbers between -2^63 and 2^63-1.
It is really strange how auto convert between numeric data behaves in T-Sql
Declare #fdays as float(12)
Declare #mAmount As Money
Declare #fDaysi as float(12)
Set #fdays =3
Set #fdaysi =1
Set #mAmount=527228.52
Set #mAmount = #fdaysi * #mAmount/#fDays
Select #mAmount, 527228.52/3
The result of this computation is
175742.8281 175742.840000
Does this occur because money and float are not actually the same kind of numeric data? Float is Approximate Numeric and Money is Exact Numeric
Money and Decimal are fixed numeric datatypes while Float is an
approximate numeric datatype. Results of mathematical operations on
floating point numbers can seem unpredictable, especially when
rounding is involved. Be sure you understand the significance of the
difference before you use Float!
Also, Money doesn't provide any advantages over Decimal. If fractional
units up to 5 decimal places are not valid in your currency or
database schema, just use Decimal with the appropriate precision and
scale.
ref link : http://www.sqlservercentral.com/Forums/Topic1408159-391-1.aspx
Should you choose the MONEY or DECIMAL(x,y) datatypes in SQL Server?
https://dba.stackexchange.com/questions/12916/datatypes-in-sql-server-difference-between-similar-dataypes-numeric-money
float [ (n) ]
Where n is the number of bits that are used to store the mantissa of the float number in scientific notation and, therefore, dictates the precision and storage size. If n is specified, it must be a value between 1 and 53. The default value of n is 53.
When n in 1-24 then precision is 7 digits.
When n in 25-53 then precision is 15 digits.
So in your example precision is 7 digits, thus first part #fdaysi * #mAmount
rounds result to 7 digits 527228.5. The second part returns 527228.5/3=175742.828 and casting 175742.828 to Money results in 175742.8281. So FLOAT and REAL are approximate data types and sometimes you get such surprises.
DECLARE #f AS FLOAT = '29545428.022495';
SELECT CAST(#f AS NUMERIC(28, 14)) AS value;
The result of this is 29545428.02249500200000 with just a casting.