I'm trying to extract monetary sums stored in some poorly formated xml columns (there is no schema defined for the XML column which I guess is part of the problem). I'm getting a conversion error whenever I encounter a node with 0 as its value.
Example:
select xml.value('sum(/List/value)', 'numeric') sum
from (select cast('<List><value>1</value><value>2</value></List>' as xml) xml) a
gives the sum 3 while:
select xml.value('sum(/List/value)', 'numeric') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
raises the error: "Error converting data type nvarchar to numeric."
Any idea how I can make my query return 0 when summing up a list of zero-valued nodes?
Your comment suggests an answer to your problem.
Instead of converting to numeric, convert to float. Scientific notation will convert to float.
you can also use and if statement like this:
#x.value('if (sum(/List/value) = 0) then 0 else sum(/List/value)', 'numeric')
I stumbled across this question, and ultimately an answer, while looking at a similar issue with integers. Despite the delay since the last answer, I'm adding here in case it helps anyone else in future.
First your basic answer:
select xml.value('xs:decimal(sum(/List/value))', 'numeric') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
In XQuery you can cast the value to a standard XML Schema type, which will then be handled correctly by SQL Server.
PLEASE NOTE: the default "numeric" in SQL Server does not have any decimal places(scale of "0")! You probably intended to do something more like:
select xml.value('xs:decimal(sum(/List/value))', 'numeric(20,5))') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
(you cannot get SQL Server to infer the precision or scale from the value returned from the Xml, you must explicitly specify it)
Finally, the actual issue that I personally needed to address was almost exactly the same, except I was dealing with integers, which also struggle with the xml representation of "0" double values:
select xml.value('xs:int(sum(/List/value))', 'int') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
UPDATE: The problem with the decimal-handling solution I posted above (converting to decimal in XQuery before SQL gets to parsing the value) is that the aggregation actually happens with the (assumed/inferred) floating point (double) data type. If the values you have stored in your Xml require a high degree of precision, this may actually be the wrong thing to do - the floating-point aggregation may actually result in a loss of data. EG here we lose the last digit of the number we are summing:
select xml.value('xs:decimal(sum(/List/value))', 'numeric(28, 0)') sum
from (select cast('<List>
<value>1000000000000000000000000001</value>
<value>1000000000000000000000000001</value>
</List>' as xml) xml) a
(comes out to "2000000000000000000000000000", which is wrong)
This issue equally applies to other approaches offered here, such as explicitly reading the value as "float" in T-SQL.
To avoid this, here's a final option using an XQuery FLWOR expression to set the data type before the aggregation operation. In this case, the aggregation occurs correctly, and we have the correct summed value (while also handling "0" values if/when they occur):
select xml.value('sum(for $r in /List/value return xs:decimal($r))', 'numeric(28, 0)') sum
from (select cast('<List>
<value>1000000000000000000000000001</value>
<value>1000000000000000000000000001</value>
</List>' as xml) xml) a
(comes out to "2000000000000000000000000002", the correct value)
Related
I have a table which has values in the r_version_label column like:
*CURRENT*, *LATEST*, 0.1, 0.2, 0.3, *0.8.5,* 1.0, 1.1
The CURRENT, LATEST and legacy version numbers such as 0.8.5 I can ignore.
I am writing SQL as below:
WITH cte_version_label AS
(
SELECT DISTINCT r_version_label
FROM pharma_document_rp
WHERE r_version_label LIKE '%.%'
AND r_version_label NOT LIKE '%.%.%'
)
SELECT *
FROM cte_version_label
WHERE CAST(r_version_label AS float) = 0.1
But I am getting:
Msg 8114, Level 16, State 5, Line 1
Error converting data type nvarchar to float.
I can however do this:
WITH cte_version_label AS
(
SELECT DISTINCT r_version_label
FROM pharma_document_rp
WHERE r_version_label LIKE '%.%'
AND r_version_label NOT LIKE '%.%.%'
)
SELECT CAST(r_version_label AS float)
FROM cte_version_label
Which returns all the right values without error.
So why can't I cast in the WHERE clause, but can in the SELECT clause? Obviously there is not really a CAST issue as I am removing the offending items, otherwise the SELECT CAST would not work.
The issue is, I need to run a python script reading in version numbers from Excel and then look these up in the table. Excel converts 1.0 into 1. So I need the whole query to operate using "floats" not the string type version stored in the database.
John's suggestion to use try_convert is definitely a better option.
But in response to WHY the second query works and the first doesn't, have a look at the execution plans.
On my instance (SQL 2017 Enterprise) this is the Estimated execution plan of the first query (can't use the actual because the query errors out).
Have a look at the predicate used in the first node. It's trying to do the CAST (internally using CONVERT) in the first operation on your whole table. When that hits something like 0.8.5 it bails.
Now let's look at the execution plan for your second query that works (this one is the Actual execution plan).
Notice the predicate in the first node - it's just your string filter. The CAST does not happen until later down the execution chain, in the Compute Scalar node, AFTER values that offend the CAST have already been filtered out.
select sum(cast(mmax as float)
from table
mmax is of datatype nvarchar and the value is
string,int,decimal, value
I trying to sum of like value 17.50,35.00.
I am avoiding string value in where clause
But not solved this problem
Error is thrown
String/Varchar values with commas such as "10,000" pass the IsNumeric() test but do not cast/convert into numeric types without raising an error.
You can replace the commas and perform the cast and sum operation:
select sum(cast(replace(mmax,',','') as float))
from tbl
where isnumeric(maxx)>0
One of the values cannot be converted to a float. You may have a million values that can convert if one (has a letter O instead of a 0 for example) you will get that message.
I am trying to write a simple query in order to change some of our stage data. I have a varchar $ column (unfortunately) that needs to be summed. My issue is that because of commas, I cannot change the datatype.
So, I can use REPLACE(AMT,',','') to remove the commas but It still wont let me cast it as a decimal and I get
Error converting data type varchar to numeric.
I am trying the following below with no luck. Any ideas? Can this be done or am I using the wrong syntax here?
Select SUM(Cast(REPLACE(Amt,',','') as Decimal (18,2)) )
I was able to resolve this with #HABO suggestion. I used Cast(Ltrim(rtrim(table.AMT)) as Money) for all instances of the varchar amount. This removed white space and removed the commas from the numbers.
This should work... including an example.
Edit: if you are on SQL Server 2012+, you may be able to shorten your task by using Try_Convert
DECLARE #SomeTable AS TABLE (Amt Varchar(100));
INSERT INTO #Sometable (Amt) VALUES ('abc123,456.01'),(' 123,456.78 '),(Null),('asdad'),('');
With NumericsOnly AS
(
SELECT
REPLACE(Left(SubString(Amt, PatIndex('%[0-9.-,]%', Amt), 8000), PatIndex('%[^0-9.,-]%', SubString(Amt, PatIndex('%[0-9.,-]%', Amt), 8000) + 'X')-1),',','') AS CleanAmt
FROM
#SomeTable
)
SELECT
SUM(CONVERT(DECIMAL(18,2), CleanAmt)) AS TotalAmt
FROM
NumericsOnly
WHERE
IsNumeric(CleanAmt)=1
General methodology is taken from here
I wouldn't use money as a data type as it is notorious for rounding error.
The error is due to SQL order of operations within your SUM(CAST(REPLACE... operation. This issue can be resolved by summing the column AFTER it's been staged to be summed via a subquery:
SELECT SUM(Field),...
FROM ( SELECT
Cast(REPLACE(Amt,',','') as NUMERIC) as 'Field'
,...
) [Q]
If the table you're summing is administered by a BI Team, get them to stage the data there. Happy Data Happy life.
You can try simply:
table1: has a column1 of type 'float'
instead of
SELECT column1 from Table1; gives values as seen in table.
Say this returns 15.1
However, if you try
Select column1 from Table1
FOR XML PATH('Table1'), Root('SomeRoot'), TYPE
returns: 1.510000000000000e+001
Has anyone seen this, and how was this fixed?
thanks in advance :)
This is what you get when you work with floating point numbers. You can try this though:
SELECT CONVERT(varchar(100), CAST(column1 AS decimal(38,2)))
you will just need to adjust the precision on the decimal to fit your needs.
Also assuming MSSQL, the str function might fit your needs (MSDN):
select str(column1, 3,1)
It's not necessary to convert the float value into a string to solve this problem.
Just convert the float value into a decimal or a numeric type, with the precision you want.
SELECT CAST(column1 AS decimal(38,2))
or
SELECT CAST(column1 AS numeric(18,5))
When then number will be parsed in XML by Sql Server, it won't be in an exponential form.
This approach is obviously faster than a string conversion (that would occur twice).
If I just write something like
select 10.00;
What type does this give me?
Here's a test I ran, to test the binary representation of these types. The big surprise here is that none of the casts actually match the first row!
select cast(123.866 as binary) union all
select cast(cast(123.866 as real) as binary) union all
select cast(cast(123.866 as float) as binary) union all
select cast(cast(123.866 as decimal) as binary) union all
select cast(cast(123.866 as numeric) as binary) union all
select cast(cast(123.866 as money) as binary) union all
select cast(cast(123.866 as smallmoney) as binary)
--------------
0x0000000000000000000000000000000000000000000006030001DAE30100
0x000000000000000000000000000000000000000000000000000042F7BB64
0x00000000000000000000000000000000000000000000405EF76C8B439581
0x00000000000000000000000000000000000000000000120000017C000000
0x00000000000000000000000000000000000000000000120000017C000000
0x00000000000000000000000000000000000000000000000000000012E684
0x00000000000000000000000000000000000000000000000000000012E684
Can anyone explain this?
Originally, all I wanted to do was avoid having to write the cast statement, assuming 123.866 was implicitly a decimal. So I thought to test if these two statements were the same:
select cast(123.866 as decimal)
select 123.866
In addition to #David M's answer, here's a way to find the type directly, found here: less than dot: How to implement a typeof operator in SQL by using sql_variant_property:
select
CAST(SQL_VARIANT_PROPERTY(123.866, 'BaseType') AS VARCHAR(20)) AS Type,
CAST(SQL_VARIANT_PROPERTY(123.866, 'Precision') AS INT) AS Precision,
CAST(SQL_VARIANT_PROPERTY(123.866, 'Scale') AS INT) AS Scale
Which gives me this answer:
Type Precision Type
numeric 6 3
Note that I tested this on SQL Server 2008, not 2005. I hope that function is available in 2005.
numeric is further described here: Data Types (Transact SQL), and here: decimal and numeric (Transact-SQL).
It's implicitly typed as decimal(6,3) - cast to this and you'll see the binary values match. It seems to use decimal at the smallest tpe to fit the value - in this case, 6 digits including 3 after the decimal place.
In your test, you are using union all. Did you know that all values will be cast to the same data type in a union query?
SQL Server Precision and Scale problems
This article also demonstrates what happens when you combine various data types together.
Are you sure you have to cast it in your query? When you select the result, what's next in line to process it? If it's an application, have the code properly determine and cast the type (if needed). If it's another query, you may be able to pass it along without any issues.