I've noticed that the nzsql and 'nzunload' just truncates double precision column's mantissa values. Here is the issue:
select tot_amt from table1;
tot_amt
~~~~~~~
123.124
567.678
while when I use other clients like Aginity for Data analytics - the output I get is
tot_amt
~~~~~~~
123.1240535
567.6780122
Also I've found the 'truncation' happens when netezza encounters 0 after 3 mantissa digits.
We are trying to migrate this db to oracle and due to this issue the entire project is messed and the client doesn't trust our migration scripts. Has anyone encountered this issue? The only workaround, even frmo IBM engineer is to cast it TO_CHAR( '999,999.999', col ) This will kill the unload scripts if I have to do it for billions of rows.
I can reproduce this issue where I have a table created with column as FLOAT(6) such as:
USERDB.USER(USER)=> create table ZZ (
USERDB.USER(USER)(> YY FLOAT(6)
USERDB.USER(USER)(> );
CREATE TABLE
USERDB.USER(USER)=> insert into ZZ (yy) values (123.123456789);
INSERT 0 1
USERDB.USER(USER)=> insert into ZZ (yy) values (12.123456789);
INSERT 0 1
USERDB.USER(USER)=> select * from ZZ;
YY
---------
123.123
12.1234
(2 rows)
USERDB.USER(USER)=> select CAST ( YY as FLOAT(15) ) from ZZ;
?COLUMN?
----------------
123.1234588623
12.123399734497
(2 rows)
USERDB.USER(USER)=>
I can cast the column values to a wider type, however the problem I see is that the value I inserted is not the same as the value returned. And the same is true if I use Aginity also to query, the values are incorrect.
Check the precision (and scale) of the 'tot_amt' column in table1, I guess the data type used to store values is quite small (FLOAT(6) maybe?), and NZSQL is telling you the correct values as enforced by the data type.
Related
A table of financials has been provided to me with the following datatypes:
billed varchar 9
allowed varchar 9
paid varchar 7
with the following columns and values:
billed = 2555 allowed = 1051 paid = 951
I want to convert the varchar values (the whole column) to money or to some format where I'll have a $ sign and the number will have 2 decimal points instead of rounding up. I need the SUM to remain because I'm adding up values throughout the columns based on the date.
My Expected Results are:
BILLED
$2,554.67
ALLOWED
$1,050.75
PAID
$950.75
I have code that I've used, but I can't seem to format it correctly to be viewable in the post.`
Cast the values as numeric, do math with a function like sum, format as money and concatenate the $ symbol at the beginning.
Simplified example doing conversion:
select ('$' + FORMAT(CONVERT(MONEY, cast([allowed] as numeric(38,2))), '###,###.####')) as AllowedConversionExample
from dbo.payments
Simplified example with math using sum()
select
'$' + FORMAT(CONVERT(MONEY, sum(cast(V.Val as numeric(38,2)))), '###,###.####')
from (
select cast('1050.75' as varchar(9)) Val
union select cast('950.75' as varchar(9))
) V
I have table P in a SQL Server database.
In table P, I have a value 234.6 in column A and 234.595 in column B.
A query
select A-B
from P
yields the result
0.004999999999545
It ideally should be
0.005
What should I do to get result as 0.005?
If your data types are not appropriate, you can get rounding anomalies
Consider these 3 queries
select cast(234.6 as decimal(12,6)) - cast(234.595 as decimal(12,6))
select cast(234.6 as decimal(12,3)) - cast(234.595 as decimal(12,3))
select cast(234.6 as float) - cast(234.595 as float)
The results are:
0.005000
0.005
0.00499999999999545
Cast your columns as required
In SQL Server, I have a query that returns a value of 177200. I need this value represented as 1772.00 as the last 2 digits are past the decimal. The query below is adding .00 to the end of the full value. I have no experience in this type of SQL statement. Any help would be appreciated.
SELECT
STR(SUM(ActualPrice), 10, 2) AS Total, Department
FROM
#DepartmentSalesData
GROUP BY
Department
The data type you're looking for is called numeric
SELECT CAST(SUM(ActualPrice) / 100.0 AS numeric(18, 2)) AS Total, ...
FROM ...
You're passing in a precision (18 in my example) and a scale 2 in my example, as requested by you.
I 've noticed that when converting TEXT to VARCHAR the converted value is silently clipped at 30 characters.
CREATE TABLE foo (x TEXT)
-- insert a string that's 50 characters long
INSERT INTO foo(x) VALUES('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
SELECT CHAR_LENGTH(CONVERT(VARCHAR, x)) FROM foo -- retuns 30
SELECT CHAR_LENGTH(CONVERT(VARCHAR(3000), x)) FROM foo -- returns 50
My questions are:
where is that limit documented / originate from?
what's an idiomatic way to make the conversion without having to add an arbitrarily high value? (as in the second SELECT statement above)
You can better always specify the varchar-length and the maximum length of a varchar is in Sybase ASE 15.7 and 16.0 16384.
If you try to create a longer varchar, you'll get following error:
Length or precision specification 16385 is not within the range of 1 to 16384.
Tim
According to MSDN, the range for REAL values is - 3.40E + 38 to -1.18E - 38, 0 and 1.18E - 38 to 3.40E + 38. However, I have quite a few values beyond that range in my table.
The following query returns lots of very small values and no very large ones:
SELECT MyColumn ,
*
FROM data.MyTable
WHERE MyColumn <> 0
AND ( MyColumn < CONVERT(REAL, 1.18E-38)
OR MyColumn > CONVERT(REAL, 3.40E+38)
)
AND ( MyColumn < CONVERT(REAL, -3.40E+38)
OR MyColumn > CONVERT(REAL, -1.18E-38)
)
It is easy to show how these values end up in the table. I cannot insert them directly:
CREATE TABLE a(r REAL NULL);
GO
INSERT INTO a(r) VALUES(4.330473E-39);
GO
SELECT r FROM a
GO
DROP TABLE a;
----
0.0
But I can divide two columns and get and outside of range value:
CREATE TABLE a
(
r1 REAL NULL ,
r2 REAL NULL ,
r3 REAL NULL
) ;
GO
INSERT INTO a
( r1, r2 )
VALUES ( 4.330473E-38, 1000 ) ;
GO
UPDATE a
SET r3 = r1 / r2 ;
SELECT r1 ,
r2 ,
r3
FROM a
r1 r2 r3
------------- ------------- -------------
4.330473E-38 1000 4.330433E-41
So I guess MSDN gives wrong ranges of valid data, correct?
Am I missing anything?
Several people suggested that this is a bug.
What part of this behavior exactly is a bug. is it:
Wrong constants documented in MSDN and used in DBCC, as well as wrong threshold for rounding down.
Update being able to save wrong values
Books Online documents only the normal range for single- and double-precision floating point numbers. The IEEE 754 rules also specify floating-point numbers closer to zero than the smallest non-zero normal value, known variously as denormalized, denormal, and subnormal numbers. From that last link:
Denormal numbers provide the guarantee that addition and subtraction of floating-point numbers never underflows; two nearby floating-point
numbers always have a representable non-zero difference. Without
gradual underflow, the subtraction a−b can underflow and produce zero
even though the values are not equal. This can, in turn, lead to
division by zero errors that cannot occur when gradual underflow is
used.
SQL Server is following the rules for single-precision floating point calculations in the examples posted. The bug may be that DBCC checks only for normal values, and throws an incorrect error message when it encounters a stored denormal value.
Example producing a denormal single-precision value:
DECLARE
#v1 real = 14e-39,
#v2 real = 1e+07;
-- 1.4013e-045
SELECT #v1 / #v2;
Example showing a stored float denormal passes DBCC checks:
CREATE TABLE dbo.b (v1 float PRIMARY KEY);
INSERT b VALUES (POWER(2e0, -1075));
SELECT v1 FROM b; -- 4.94065645841247E-324
DBCC CHECKTABLE(b) WITH DATA_PURITY; -- No errors or warnings
DROP TABLE dbo.b;
This is a bug in SQL Server. The last script you post is a nice repro. Add one line to it at the end:
DBCC CHECKDB WITH data_purity
This fails with:
Msg 2570, Level 16, State 3, Line 1 Page (1:313), slot 0 in object ID
357576312, index ID 0, partition ID 1801439851932155904, alloc unit ID
2017612634169999360 (type "In-row data"). Column "r3" value is out of
range for data type "real". Update column to a legal value.
This proves it is a bug. I suggest you file a bug with Microsoft Connect for SQL Server.