Why precision is decreasing when multiply sum to other number - sql-server

I have encountered with following bug (or feature) in SQL Server.
When I use SUM (*column*) where column has a numeric(18, 8) type and multiply it to any other number (integer or decimal) the result precision is reducing to numeric(18, 6).
Here is the example script to demonstrate.
CREATE TABLE #temp (Qnty numeric(18,8))
INSERT INTO #temp (Qnty) VALUES (0.00000001)
INSERT INTO #temp (Qnty) VALUES (0.00000002)
INSERT INTO #temp (Qnty) VALUES (0.00000003)
SELECT Qnty, 1*Qnty
FROM #temp
SELECT (-1)*SUM(Qnty), SUM(Qnty), -SUM(Qnty), SUM(Qnty) * CAST(2.234 as numeric(18,8))
FROM #temp
DROP TABLE #temp
The result of second SELECT query
0.000000 0.00000006 -0.00000006 0.000000
As you can see then I multiply SUM the result is 0.000000
Could anyone explain the strange behavior?
UPD. I executed this query in SQL Management Studio on 2000, 2005 and 2008 SQL Server.

Aggregating a numeric(18, 8) with SUM results in the datatype numeric(38, 8).
How the resulting datatype is calculated when multiplying something with numeric can be found here: Precision, Scale, and Length (Transact-SQL)
The datatype for your constant -1 is numeric(1, 0)
Precision is p1 + p2 + 1 = 40
Scale is s1 + s2 = 8
Max precision is 38 and that leaves you with numeric(38, 6).
Read more about why it is numeric(38, 6) here: Multiplication and Division with Numerics

If you read SUM's reference page, you'll see that on a decimal column it yields a type of NUMERIC(38,6). You need to cast the result of the SUM to NUMERIC(18,8) for it to work the way you want.
Executing SELECT CAST(SUM(Qnty) as numeric(18,8)) * 2.234
FROM #temp yields 0.00000013404 as you'd expect.

a technical explanation can be found at http://social.msdn.microsoft.com/Forums/en/transactsql/thread/233f7380-3f19-4836-b224-9f665b852406

Related

ORA-01440 error from setting precision of 38 on a NULL precision column with data?

In an Oracle database I work with, many NUMBER columns were originally created with NULL precision. My understanding is that these columns have implicit maximum precision. (Per Oracle: "If a precision is not specified, the column stores values as given.") They currently contain data.
The maximum precision I can explicitly specify for a new column is 38. However, when I attempt to modify these existing columns to set this explicit precision, Oracle throws an error:
ALTER TABLE test_table
MODIFY column NUMBER(38);
ORA-01440: column to be modified must be empty to decrease precision or scale
If 38 is the maximum, how am I decreasing precision by setting that number?
The alternative approaches to changing this type (e.g., creating a new column, copying data, dropping/re-adding constraints, etc.) are quite laborious relative to the modify statement.
You are right in quoting Oracle that
If a precision is not specified, the column stores values as given
and
The maximum precision I can explicitly specify for a new column is 38
But that doesn't mean that the maximal precision that Oracle can store is 38.
Here a simple example - assume that your column is defined without precisionand with scale zero:
create table MyTable
(num1 NUMBER(*,0)
);
You may verify this in the table disctionary
select COLUMN_NAME, DATA_PRECISION, DATA_SCALE
from user_tab_columns where table_name = 'MYTABLE';
COLUMN_NAME DATA_PRECISION DATA_SCALE
------------------------------ -------------- ----------
NUM1 0
Now let's fill some data
insert into MyTable(num1) values(sqrt(2)*power(10,38) );
Actually you got a precision of 39 (I'm on Windows this behaviour may be OS dependent, but Oracle must count with the possibility)
select num1 from MyTable;
141421356237309504880168872420969807857
select length(to_char(num1)) from MyTable;
39
So you see you can't simple overwrite the precision with the maximal allowed precision of 38 - this could mean a real decrease.
Similar situation is if the column is defined without scale and precision
create table MyTable
(num1 NUMBER
);
select COLUMN_NAME, DATA_PRECISION, DATA_SCALE
from user_tab_columns where table_name = 'MYTABLE';
COLUMN_NAME DATA_PRECISION DATA_SCALE
------------------------------ -------------- ----------
NUM1
Contrary to popular belief and the Oracle documentation in this case doesn't hold:
If no scale is specified, the scale is zero.
On the contrary the scale may be higher that 38 as you see in the example below:
insert into MyTable(num1) values(sqrt(2)/10);
select num1 from MyTable;
0,141421356237309504880168872420969807857
There are 39 decimal digits in the figure.
Summary
You can't set the precision without a reorganizing the table.
The maximum precision I can explicitly specify for a new column is 38
...
If 38 is the maximum, how am I decreasing precision by setting that number?
That is the maximum you can declare, as the p constraint in NUMBER(p, s). It is often thought that means that Oracle doesn't allow numbers of more than 38 digits even if unconstrained, but that isn't quite true. Even if you only constrain the scale you can have more; e.g. with a column of data type INTEGER, which has no precision constraint but does have a scale of 0.
The documentation for the NUMBER data type (12cR2) includes:
Oracle guarantees the portability of numbers with precision of up to 20 base-100 digits, which is equivalent to 39 or 40 decimal digits
With a column with unconstrained precision you can have more than 38 significant digits. Some sample tables to demonstrate against:
create table t1 (num number);
create table t2 (num integer);
create table t3 (num number(38));
create table t4 (num number(38,0));
select table_name, column_name, data_type, data_length, data_precision, data_scale
from user_tab_columns
where table_name in ('T1', 'T2', 'T3', 'T4');
TABLE_NAME COLUMN_NAME DATA_TYPE DATA_LENGTH DATA_PRECISION DATA_SCALE
---------- ------------ --------- ----------- -------------- ----------
T1 NUM NUMBER 22
T2 NUM NUMBER 22 0
T3 NUM NUMBER 22 38 0
T4 NUM NUMBER 22 38 0
then this is allowed:
insert into t1 values (power(10, 39)+1);
as is, in fact, this:
insert into t1 values (power(10, 40)+1);
1 row inserted.
When you look at the data, as reported in a SQL Developer worksheet with numwidth set to 42, you see:
select num, length(to_char(num)) from t1;
NUM LENGTH(TO_CHAR(NUM))
------------------------------------------ ------------------------------------------
1000000000000000000000000000000000000001 40
10000000000000000000000000000000000000000 40
The first row has 40 significant digits and the exact value is preserved. (The second row has lost the +1 from the insert, as it's now too large for internal precision in the quote above. The length is still reported as 40 without an explicit format mask, which is mildly interesting. Otherwise that row isn't helpful...)
An INTEGER column, which is NUMBER(,0), allows the same:
insert into t2 values (power(10, 39)+1);
1 row inserted.
select num, length(to_char(num)) from t2;
NUM LENGTH(TO_CHAR(NUM))
------------------------------------------ ------------------------------------------
1000000000000000000000000000000000000001 40
But once you constrain the precision that is no longer allowed:
insert into t3 values (power(10, 39)+1);
ORA-01438: value larger than specified precision allowed for this column
insert into t3 values (power(10, 38)+1);
ORA-01438: value larger than specified precision allowed for this column
insert into t3 values (power(10, 37)+1);
1 row inserted.
select num, length(to_char(num)) from t3;
NUM LENGTH(TO_CHAR(NUM))
------------------------------------------ ------------------------------------------
10000000000000000000000000000000000001 38
If you constraint the scale as well, to zero anyway, it behaves the same; inserting and querying t4 gets the same results.
So the point is that a plain NUMBER column is not the same as a NUMBER(38, 0) column, and your alter statement is actually reducing the theoretical maximum precision of the column. Oracle doesn't check the actual values in a column before throwing ORA-01440, so it doesn't matter than none of the existing values actually exceed the precision you are trying to specify - it just knows that they could.

Transact SQL: case when 1=1 then 0.5 else ceiling(sh) end /* returns 1 (!) why? */

-- Transact SQL: case when 1=1 then 0.5 else ceiling(sh) end /* returns 1 (!) why? */
declare #T table (h decimal(2,1))
insert #T (h) values (1.0)
select
case when 1=1 then 0.5 else ceiling(sh) end /* returns 1 (!) why? */
from #T T1
join (select sum(h) as sh from #T )T2 on 1 = 1
The answer is nothing to do with the int datatype
The literal 0.5 has a dataype of decimal(1,1)
Applying CEILING on a decimal(p,s) returns a result of type
decimal(p,0)
Applying SUM on a decimal(p,s) returns a result of
type decimal(38,s)
With a mixed CASE expression that can return
decimal(p1,s1) or decimal(p2,s2) the result will use the same rules as when UNION-ing these data types and have precision (*)
of max(s1, s2) + max(p1-s1, p2-s2) and scale of max(s1, s2)
* The result precision and scale have an absolute maximum of 38. When a
result precision is greater than 38, it is reduced to 38, and the
corresponding scale is reduced to try to prevent the integral part of
a result from being truncated. (source)
So your column h has datatype of decimal(2,1), the datatype when SUM is applied is decimal(38,1), the datatype of CEILING applied to that is decimal(38,0). Then you use that in a CASE expression with decimal(1,1)
max(s1, s2) + max(p1-s1, p2-s2)
max( 0, 1) + max( 38, 0) = 1 + 38 = 39
And
max(s1, s2) = max(0, 1) = 1
So the desired result datatype would be decimal(39,1). This is larger than 38 so you get the scale reduction described above and end up with decimal(38,0) - 0.5 is rounded to 1 when cast to that datatype.
If you would rather keep the precision of the final result you can use
case when 1=1 then 0.5 else CAST(ceiling(sh) AS decimal(38,1)) end
There is a miniscule additional risk of overflow with this but to be hit by it the sum would need to add up to one of the following values
9999999999999999999999999999999999999.5
9999999999999999999999999999999999999.6
9999999999999999999999999999999999999.7
9999999999999999999999999999999999999.8
9999999999999999999999999999999999999.9
such that the SUM itself fits into 38,1 but CEILING doesn't.
More explanation:
select CEILING(170.00/6.00),CEILING(170/6),170.00/6.00,170/6
Result:
29 28 28.333333 28
As explain in the documentation
When an operator combines two expressions of different data types, the rules for data type precedence specify that the data type with the lower precedence is converted to the data type with the higher precedence. If the conversion is not a supported implicit conversion, an error is returned. When both operand expressions have the same data type, the result of the operation has that data type.
In your case is converted to int because has the bigger precedence.
sql demo
declare #T table (h decimal(2,1))
insert #T (h) values (1.0)
select
case when 1=1 then 0.5 else ceiling(sh)*1.0 end -- << convert to float
from #T T1
join (select sum(h) as sh from #T )T2 on 1 = 1

CAST or convert Numeric to Money without rounding in SQL server

I want to convert numeric value to Money but without Rounding value.W.r.t. to
Link : https://technet.microsoft.com/en-us/library/ms187928(v=sql.105).aspx It is rounding numeric to Money while casting.
But is it possible to give value upto 4 digit after decimal.
NUMERIC VALUE : 123456789.3333
MONEY VALUE OUTPUT required : 123,456,789.3333
May be you are looking for something like this
SELECT FORMAT(CONVERT(MONEY, CAST(123456789.3333 AS NUMERIC(18,4))), '###,###.####')
Result
123,456,789.3333
I guess you mean numerics where you have more than 4 digits, then you could use ROUND:
SELECT CAST(ROUND(123456789.33339, 4, 1) AS MONEY)
-- 123456789,3333
vs.
SELECT CAST(123456789.33339 AS MONEY)
-- 123456789,3334
Rextester Demo
if you wanna split number as 3 digit , you can use this code in your select command
Select LEFT(CONVERT(VARCHAR, CAST(YourPrice AS MONEY), 1), LEN(CONVERT(VARCHAR, CAST(UnitPrice AS MONEY), 1)) - 3 )as UnitPrice

How to display two digits after decimal point in SQL Server

I have table which has a column of float data type in SQL Server
I want to return my float datatype column value with 2 decimal places.
for ex: if i insert 12.3,it should return 12.30
if i insert 12,it should return 12.00
select cast(your_float_column as decimal(10,2))
from your_table
decimal(10,2) means you can have a decimal number with a maximal total precision of 10 digits. 2 of them after the decimal point and 8 before.
The biggest possible number would be 99999999.99
You can also do something much shorter:
SELECT FORMAT(2.3332232,'N2')
You can also use below code which helps me:
select convert(numeric(10,2), column_name) as Total from TABLE_NAME
where Total is alias of the field you want.
You can also Make use of the Following if you want to Cast and Round as well. That may help you or someone else.
SELECT CAST(ROUND(Column_Name, 2) AS DECIMAL(10,2), Name FROM Table_Name
select cast(56.66823 as decimal(10,2))
This returns 56.67.

Returning the result of divided number with a Scale in a Select Statement

I am trying to run this query, but it seems like it's not formatting the numbers properly after doing a mathematical calculation. The scale should be at 2, but it won't display as it should.
SELECT 30 / 60 as Diff FROM Table
This returns as 0
SELECT Convert( Numeric(8,2), 30 / 60 ) as Diff FROM Table
This returns as 0.00
How do I get this to return 0.50 as needed?
You can try the following and there are few ways to achieve it. Using a variable, or just performing CAST, CONVERT on the fields itself.
SELECT (CAST(30 AS DECIMAL(8,2)) / CAST(60 AS DECIMAL(8,2))) as Diff FROM Table
SELECT (CAST(30/60) AS DECIMAL(8,2)) as Diff FROM Table
SELECT (30/60.0) as Diff FROM Table;
SELECT CONVERT(DECIMAL(8,2), 30/60.0) as Diff FROM Table
Please take a look at this MSDN article for further reference:
Convert to numeric before doing the division or divide real numbers instead of integers. When doing integer arithmetic any decimal fraction is dropped. Converting after the fraction has been dropped doesn't do you any good.
SELECT CONVERT(NUMERIC(8,2),30.0/60.0) AS DIFF FROM TABLE
or if selecting columns instead of using numbers
SELECT CONVERT(NUMERIC(8,2),columnA)/CONVERT(NUMERIC(8,2),columnB) AS DIFF FROM TABLE
(converting both just to be sure the division is done according to the rules we want)

Resources