Error converting NVARCHAR to Float using a CAST in T-SQL - sql-server

I have a table which has values in the r_version_label column like:
*CURRENT*, *LATEST*, 0.1, 0.2, 0.3, *0.8.5,* 1.0, 1.1
The CURRENT, LATEST and legacy version numbers such as 0.8.5 I can ignore.
I am writing SQL as below:
WITH cte_version_label AS
(
SELECT DISTINCT r_version_label
FROM pharma_document_rp
WHERE r_version_label LIKE '%.%'
AND r_version_label NOT LIKE '%.%.%'
)
SELECT *
FROM cte_version_label
WHERE CAST(r_version_label AS float) = 0.1
But I am getting:
Msg 8114, Level 16, State 5, Line 1
Error converting data type nvarchar to float.
I can however do this:
WITH cte_version_label AS
(
SELECT DISTINCT r_version_label
FROM pharma_document_rp
WHERE r_version_label LIKE '%.%'
AND r_version_label NOT LIKE '%.%.%'
)
SELECT CAST(r_version_label AS float)
FROM cte_version_label
Which returns all the right values without error.
So why can't I cast in the WHERE clause, but can in the SELECT clause? Obviously there is not really a CAST issue as I am removing the offending items, otherwise the SELECT CAST would not work.
The issue is, I need to run a python script reading in version numbers from Excel and then look these up in the table. Excel converts 1.0 into 1. So I need the whole query to operate using "floats" not the string type version stored in the database.

John's suggestion to use try_convert is definitely a better option.
But in response to WHY the second query works and the first doesn't, have a look at the execution plans.
On my instance (SQL 2017 Enterprise) this is the Estimated execution plan of the first query (can't use the actual because the query errors out).
Have a look at the predicate used in the first node. It's trying to do the CAST (internally using CONVERT) in the first operation on your whole table. When that hits something like 0.8.5 it bails.
Now let's look at the execution plan for your second query that works (this one is the Actual execution plan).
Notice the predicate in the first node - it's just your string filter. The CAST does not happen until later down the execution chain, in the Compute Scalar node, AFTER values that offend the CAST have already been filtered out.

Related

SQL Server System.OutOfMemoryException

I am generating XML from 60 tables, and storing this xml in a table.
Table Name : Final_XML_Table
PK FK XML_Content (type xml)
1 1 "XML that I am generating from 60 tables"
When I am running below query , it gives memory exception :
Select * from Final_XML_Table
Things I have tried :
1. Results to text : I am getting only few lines from XML as text in output window
2. Results to file : I am getting only few lines from XML in file.
Please suggest, and also if there is any change , will I have to do this on server's SQL server as well while deployment.
I have also set XML_Data to unlimited :
This is not an answer, but to much for a comment...
The fact, that you are able to store the XML, shows clearly, that the XML is not to big for the database.
The fact that you get an out-of-memory exception with Select * from Final_XML_Table shows clearly, that SSMS has a problem on reading/displaying your XML.
You might try to check the length like here:
DECLARE #tbl TABLE (x XML);
INSERT INTO #tbl VALUES('<root><test>blah</test><test /><test2><x/></test2></root>');
SELECT * FROM #tbl; --This does not work for you
SELECT DATALENGTH(x) FROM #tbl; --This returns just "82" in this case
Might be, that due to a logical error in your XML's creation (a wrong join?) the XML contains multiple/repeated elements. You might try a query like this to get a count of nodes in order to check if this number is realistic:
SELECT x.value('count(//*)','int') FROM #tbl
For the exampe above this returns "5"
You might do the same with your original XML.
With a query like the following you can retrieve all node names of the first level, the second level and so on. You can check if this looks okay:
SELECT firstLevel.value('local-name(.)','varchar(max)') AS l1_node
,SecondLevel.value('local-name(.)','varchar(max)') AS l2_node
--add more
FROM #tbl
OUTER APPLY x.nodes('/*') AS A(firstLevel)
OUTER APPLY A.firstLevel.nodes('*') AS B(SecondLevel)
--add more
And - of course - you might open the ResourceMonitor to look at the actual usage of memory...
Come back with more details...
That error isn't a SQL Server error, it's from SSMS. It means that SSMS has run out of memory.
SSMS is only a 32bit application, so can only address 2GB of RAM. If it tries to address more than that, the error will occur. if you've had SSMS open and returned some very large datasets, that RAM is going to get used up.
In all honesty, if you're running a query like SELECT * FROM Final_XML_Table then I would hazard a guess that the dataset is huge. Add a WHERE clause, or don't return the dataset on screen. if you really need to view the data (all of it), export it to something else. But I very much doubt you need to look at every row, if you're returning around 2GB of data.

T-SQL converting string into float

In a stored procedure on my SQL Server, I am trying to convert values from a varchar column into a float format.
The values into the varchar column are numbers with a sign at the beginning and a '.' before decimals.
Examples: '+0000000000000044.09' or '-0000000000114995.61'
If I try this: convert(float,mystring), it doesn't work.
I have:
Error converting data type varchar to float
Is this kind of conversion possible?
Or is there another way to convert a string with a sign and a '.' into a float?
As your examples both work, I'd guess there's another value somewhere in your table that's causing the problem. In recent versions of SQL Server (2012 onwards), the TRY_CONVERT function can be useful for tracking down this kind of issue.
TRY_CONVERT will not throw an exception on a conversion failure, but instead return a NULL value, so you can figure out which values are causing the problem like this:
SELECT * FROM your_table WHERE TRY_CONVERT(FLOAT, your_column_name) IS NULL
If any rows are returned, those are the rows with problem values that can't be converted to FLOAT.
You can try using CAST(mystring as float) or TRY_CONVERT(float,mystring).
Thuough convert(float,mystring) also should work fine. I would suggest checking your data.

ms sql server executes 'then' before 'when' in case

when i try to select
select
case when (isnumeric(SUBSTRING([VZWECK2],1,9)) = 1)
then CONVERT(decimal,SUBSTRING([VZWECK2],1,9))
else null
end as [NUM]
from table
sql-server gives me:
Msg 8114, Level 16, State 5, Line 2
Error converting data type varchar to numeric.
[VZWECK2] is a char(27). is this a known bug? because it seems to me it executes the convert before it does the case, which defies the purpose of my select. i know that there are values that are not numeric obviously, which is why i need the case statement to weed them out.
for some reason selecting
select
case when (isnumeric(SUBSTRING([VZWECK2],1,9)) = 1)
then 99
else null
end as [NUM]
from table
yields no errors and behaves as expected
The problem is that ISNUMERIC is very forgiving, and that ISNUMERIC returns 1 is unfortunately no guarantee that CONVERT will work. This is why SQL Server 2012 and later introduced TRY_CAST and TRY_CONVERT.
If you are converting whole numbers, a more reliable check is to make sure the string consists of only digits with NOT LIKE '%[^0-9]%' (that is, it must not contain a non-digit anywhere). This is too restrictive for some formats (like floating point) but for integers it works nicely.
Do you know the value which throws the error? IsNumeric is not exactly fool-proof, for example:
select ISNUMERIC('$')
select ISNUMERIC('+')
select ISNUMERIC('-')
all yield 1
Alternatively, you could go with TRY_PARSE instead.
Edit: TRY_PARSE is introduced in sql server 2012, so may not be available to you.

How to fix "domain error" in SQL Server 2005 when using LOG() function to get product of set

I have a inline select statement to calculate the product of the set of values.
Since SQL Server 2005 doesn't have a built in Product aggregate function, I am using LOG/EXP to get it.
My select statement is:
(select exp(sum(log(value))) from table where value > 0)
Unfortunately I keep getting the following error:
Msg 3623, Level 16, State 1, Line 1
A domain error occurred.
I've ensured that none of the values are zero or negative so I'm not really sure why this error is occurring. Does anyone have any ideas?
One of the features of the query planner introduced in SQL 2005 is that, in some circumstances where the table statistics indicate it will be more efficient, the WHERE clause of a statement will be processed after the SELECT clause.
(I can't find the Books On-Line reference for this right now).
I suspect this is what is happening here. You either need to exclude the rows where value = 0 before carrying out the calculation - the most reliable way being to store the rows you need in a temporary (#) table - or to modify your query to handle zero internally:
SELECT EXP(SUM(LOG(ISNULL(NULLIF(VALUE,0),1)))) AS result
FROM [table]
The NULLIF\ISNULL pair I have added to your query substitutes 1 for 0 - I think this will work, but you will need to test it on your data.

XQuery in SQL server doing SUM over zero value

I'm trying to extract monetary sums stored in some poorly formated xml columns (there is no schema defined for the XML column which I guess is part of the problem). I'm getting a conversion error whenever I encounter a node with 0 as its value.
Example:
select xml.value('sum(/List/value)', 'numeric') sum
from (select cast('<List><value>1</value><value>2</value></List>' as xml) xml) a
gives the sum 3 while:
select xml.value('sum(/List/value)', 'numeric') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
raises the error: "Error converting data type nvarchar to numeric."
Any idea how I can make my query return 0 when summing up a list of zero-valued nodes?
Your comment suggests an answer to your problem.
Instead of converting to numeric, convert to float. Scientific notation will convert to float.
you can also use and if statement like this:
#x.value('if (sum(/List/value) = 0) then 0 else sum(/List/value)', 'numeric')
I stumbled across this question, and ultimately an answer, while looking at a similar issue with integers. Despite the delay since the last answer, I'm adding here in case it helps anyone else in future.
First your basic answer:
select xml.value('xs:decimal(sum(/List/value))', 'numeric') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
In XQuery you can cast the value to a standard XML Schema type, which will then be handled correctly by SQL Server.
PLEASE NOTE: the default "numeric" in SQL Server does not have any decimal places(scale of "0")! You probably intended to do something more like:
select xml.value('xs:decimal(sum(/List/value))', 'numeric(20,5))') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
(you cannot get SQL Server to infer the precision or scale from the value returned from the Xml, you must explicitly specify it)
Finally, the actual issue that I personally needed to address was almost exactly the same, except I was dealing with integers, which also struggle with the xml representation of "0" double values:
select xml.value('xs:int(sum(/List/value))', 'int') sum
from (select cast('<List><value>0</value><value>0</value></List>' as xml) xml) a
UPDATE: The problem with the decimal-handling solution I posted above (converting to decimal in XQuery before SQL gets to parsing the value) is that the aggregation actually happens with the (assumed/inferred) floating point (double) data type. If the values you have stored in your Xml require a high degree of precision, this may actually be the wrong thing to do - the floating-point aggregation may actually result in a loss of data. EG here we lose the last digit of the number we are summing:
select xml.value('xs:decimal(sum(/List/value))', 'numeric(28, 0)') sum
from (select cast('<List>
<value>1000000000000000000000000001</value>
<value>1000000000000000000000000001</value>
</List>' as xml) xml) a
(comes out to "2000000000000000000000000000", which is wrong)
This issue equally applies to other approaches offered here, such as explicitly reading the value as "float" in T-SQL.
To avoid this, here's a final option using an XQuery FLWOR expression to set the data type before the aggregation operation. In this case, the aggregation occurs correctly, and we have the correct summed value (while also handling "0" values if/when they occur):
select xml.value('sum(for $r in /List/value return xs:decimal($r))', 'numeric(28, 0)') sum
from (select cast('<List>
<value>1000000000000000000000000001</value>
<value>1000000000000000000000000001</value>
</List>' as xml) xml) a
(comes out to "2000000000000000000000000002", the correct value)

Resources