I'm finding this to be truly bizarre behaviour.
Here is my T-SQL:
declare #testText nvarchar(1000);
set #testText = '17D4,A,1';
select txt_value from fn_ParseText2Table (#testText , ',' )
where fn_ParseText2Table is a function that parses the text into a table where you can get the txt, int, and floating point values, if they work.
The 17D4 is a product code that I'm trying to extract within a larger query, and all other 3817 records work fine.
select (
select txt_value
from fn_ParseText2Table(t.primaryKeyValues , ',' ) as pk
where position = 1) as product_NBR
from database.dbo.tablesToParse as t
where t.tableName = 'ProductData'
I found the function here.
What I've found is that if the string starts with some numbers (I've tested anywhere from 1-4 ) followed by 'D', it fails with the 'Error converting data type varchar to numeric.' message.
All other combinations of text work. Lower case d is fine. C is fine, E, F, etc. So '17F5,A,1' is fine. Also 'asdf 17D5,A,1' is fine. '1D,A,1' is not fine.
I'm very confused. Is there a special escape character in T-SQL for 'D'?
Update:
I should clarify that the error occurs inside fn_ParseText2Table()
Update 2
It's SQL server 10 - 64 bit, running on a windows 2008 server.
As well, I've tested this in a sql mgr query window:
declare #testText nvarchar(1000);
set #testText = '17D4';
select isnumeric( #testText )
The IsNumeric() call returns 1, which is why the fn_ParseText2Table() function tries to cast it to an in and it fails. I could add an extra check to that function to lower the text first and see if that's also numeric.
I figured it was related to floating-point literals but I was surprised it wouldn't cast to a numeric type. I guess it only works when float is the target type. You'll find the explanation here:
http://www.sqlservercentral.com/Forums/Topic202581-8-1.aspx
This behavior doesn't match up with other SQL Server literals, ie, constants:
select 1d -- literal 1 with d treated as the column alias
select 1e -- literal 1.0 as a float
select cast('1d' as float), cast('1e' as float) -- neither of these will cast without an exponent value
select cast('1d0' as float), cast('1e0' as float) -- these work though
Related
I load excel file into sql as varchar(max) and got that Scientific e value which now I try to convert into numeric as I need to do compare that value, and here I'm running into problem.
This is main question: How and to what type I can convert this to compare with whole integer value ?
On the pic You can see how this seen in Excel, even formatted to text it somehow still loaded into varchar(max) not like char string. This can be seen from my test code.
DECLARE #C VARCHAR(MAX) = '1.1001562717e+011', #Nc VARCHAR(MAX) = '110015627174';
SELECT #c, LEN(#c) LenC ,
ISNUMERIC(#c) NumYN
---, CAST(#c AS DECIMAL(38,2)) cDec ---CAST(#c AS NUMERIC) cNum --, CAST(#c AS BIGINT) cInt
WHERE #c LIKE '%[^0-9]%'
AND ISNUMERIC(#c) = 1
To start, ISNUMERIC is a terrible function, it does not give good results; it is often wrong. If you try ISNUMERIC('1.1001562717e+011') you'll notice that you get the value 1, however, CONVERT(numeric(13,1),'1.1001562717e+011') will produce an error. A far better function is TRY_CONVERT (or TRY_CAST), which returns NULL if the conversion fails for the specific data type: TRY_CONVERT(numeric(13,1),'1.1001562717e+011').
Being specific on the data type is actually important here, as ISNUMERIC could be (incorrectly) suggesting that the value could be converted to at least 1 of the numeric data types; but that doesn't mean all of them. For scientific data types the only data type you can convert to is a float/real:
SELECT TRY_CONVERT(numeric(13,1),'1.1001562717e+011') AS Numeric,
TRY_CONVERT(bigint,'1.1001562717e+011') AS int,
TRY_CONVERT(float,'1.1001562717e+011') AS float,
TRY_CONVERT(money,'1.1001562717e+011') AS money;
Notice that only float has a value here. As you want a numeric as the final value, then you'll need to CONVERT the value twice:
CONVERT(numeric(13,1),TRY_CONVERT(float,'1.1001562717e+011'))
I have to count the digits after the decimal point in a database hosted by a MS Sql Server (2005 or 2008 does not matter), in order to correct some errors made by users.
I have the same problem on an Oracle database, but there things are less complicated.
Bottom line is on Oracle the select is:
select length( substr(to_char(MY_FIELD), instr(to_char(MY_FILED),'.',1,1)+1, length(to_char(MY_FILED)))) as digits_length
from MY_TABLE
where the filed My_filed is float(38).
On Ms Sql server I try to use:
select LEN(SUBSTRING(CAST(MY_FIELD AS VARCHAR), CHARINDEX('.',CAST(MY_FILED AS VARCHAR),1)+1, LEN(CAST(MY_FIELD AS VARCHAR)))) as digits_length
from MY_TABLE
The problem is that on MS Sql Server, when i cast MY_FIELD as varchar the float number is truncated by only 2 decimals and the count of the digits is wrong.
Can someone give me any hints?
Best regards.
SELECT
LEN(CAST(REVERSE(SUBSTRING(STR(MY_FIELD, 13, 11), CHARINDEX('.', STR(MY_FIELD, 13, 11)) + 1, 20)) AS decimal))
from TABLE
I have received from my friend a very simple solution which is just great. So I will post the workaround in order to help others in the same position as me.
First, make function:
create FUNCTION dbo.countDigits(#A float) RETURNS tinyint AS
BEGIN
declare #R tinyint
IF #A IS NULL
RETURN NULL
set #R = 0
while #A - str(#A, 18 + #R, #r) <> 0
begin
SET #R = #R + 1
end
RETURN #R
END
GO
Second:
select MY_FIELD,
dbo.countDigits(MY_FIELD)
from MY_TABLE
Using the function will get you the exact number of digits after the decimal point.
The first thing is to switch to using CONVERT rather than CAST. The difference is, with CONVERT, you can specify a format code. CAST uses whatever the default format code is:
When expression is float or real, style can be one of the values shown in the following table. Other values are processed as 0.
None of the formats are particularly appealing, but I think the best for you to use would be 2. So it would be:
CONVERT(varchar(25),MY_FIELD,2)
This will, unfortunately, give you the value in scientific notation and always with 16 digits e.g. 1.234567890123456e+000. To get the number of "real" digits, you need to split this number apart, work out the number of digits in the decimal portion, and offset it by the number provided in the exponent.
And, of course, insert usual caveats/warnings about trying to talk about digits when dealing with a number which has a defined binary representation. The number of "digits" of a particular float may vary depending on how it was calculated.
I'm not sure about speed. etc or the elegance of this code. it was for some ad-hoc testing to find the first decimal value . but this code could be changed to loop through all the decimals and find the last time a value was greater than zero easily.
DECLARE #NoOfDecimals int = 0
Declare #ROUNDINGPRECISION numeric(32,16) = -.00001000
select #ROUNDINGPRECISION = ABS(#ROUNDINGPRECISION)
select #ROUNDINGPRECISION = #ROUNDINGPRECISION - floor(#ROUNDINGPRECISION)
while #ROUNDINGPRECISION < 1
Begin
select #NoOfDecimals = #NoOfDecimals +1
select #ROUNDINGPRECISION = #ROUNDINGPRECISION * 10
end;
select #NoOfDecimals
I want to determine if a value is integer (like TryParse in .NET). Unfortunatelly ISNUMERIC does not fit me because I want to parse only integers and not every kind of number. Is there such thing as ISINT or something?
Here is some code to make things clear. If MY_FIELD is not int, this code would fail:
SELECT #MY_VAR = CAST(MY_FIELD AS INT)
FROM MY_TABLE
WHERE MY_OTHER_FIELD = 'MY_FILTER'
Thank you
Here's a blog post describing the creation of an IsInteger UDF.
Basically, it recommends adding '.e0' to the value and using IsNumeric. In this way, anything that already had a decimal point now has two decimal points, causing IsNumeric to be false, and anything already expressed in scientific notation is invalidated by the e0.
In his article Can I convert this string to an integer?, Itzik Ben-Gan provides a solution in pure T-SQL and another that uses the CLR.
Which solution should you choose?
Is the T-SQL or CLR Solution Better? The advantage of using the T-SQL
solution is that you don’t need to go outside the domain of T-SQL
programming. However, the CLR solution has two important advantages:
It's simpler and faster. When I tested both solutions against a table
that had 1,000,000 rows, the CLR solution took two seconds, rather
than seven seconds (for the T-SQL solution), to run on my laptop. So
the next time you need to check whether a given string can be
converted to an integer, you can include the T-SQL or CLR solution
that I provided in this article.
If you only want to maintain T-SQL, then use the pure T-SQL solution. If performance is more important than convenience, then use the CLR solution.
The pure T-SQL Solution is tricky. It combines the built-in ISNUMERIC function with pattern-matching and casting to check if the string represents an int.
SELECT keycol, string, ISNUMERIC(string) AS is_numeric,
CASE
WHEN ISNUMERIC(string) = 0 THEN 0
WHEN string LIKE '%[^-+ 0-9]%' THEN 0
WHEN CAST(string AS NUMERIC(38, 0))
NOT BETWEEN -2147483648. AND 2147483647. THEN 0
ELSE 1
END AS is_int
FROM dbo.T1;
The T-SQL part of the CLR solution is simpler. You call the fn_IsInt function just like you would call ISNUMERIC.
SELECT keycol, string, ISNUMERIC(string) AS is_numeric,
dbo.fn_IsInt(string) AS is_int
FROM dbo.T1;
The C# part is simply a wrapper for the .NET's parsing function Int32.TryParse. This works because the SQL Server int and the .NET Int32 are both 32-bit signed integers.
using System;
using System.Data.SqlTypes;
public partial class UserDefinedFunctions
{
[Microsoft.SqlServer.Server.SqlFunction]
public static SqlBoolean fn_IsInt(SqlString s)
{
if (s.IsNull)
return SqlBoolean.False;
else
{
Int32 i = 0;
return Int32.TryParse(s.Value, out i);
}
}
};
Please read Itzik's article for a full explanation of these code samples.
With sqlserver 2005 and later you can use regex-like character classes with LIKE operator. See here.
To check if a string is a non-negative integer (it is a sequence of decimal digits) you can test that it doesn't contain other characters.
SELECT numstr
FROM table
WHERE numstr NOT LIKE '%[^0-9]%'
Note1: This will return empty strings too.
Note2: Using LIKE '%[0-9]%' will return any string that contains at least a digit.
See fiddle
WHERE IsNumeric(MY_FIELD) = 1 AND CAST(MY_FIELD as VARCHAR(5)) NOT LIKE '%.%'
That is probably the simplest solution. Unless your MY_FIELD contains .00 or something of that sort. In which case, cast it to a float to remove any trailing .00s
Necromancing.
As of SQL-Server 2012+, you can use TRY_CAST, which returns NULL if the cast wasn't successful.
Example:
DECLARE #foo varchar(200)
SET #foo = '0123'
-- SET #foo = '-0123'
-- SET #foo = '+0123'
-- SET #foo = '+-0123'
-- SET #foo = '+-0123'
-- SET #foo = '.123'
-- SET #foo = '1.23'
-- SET #foo = '.'
-- SET #foo = '..'
-- SET #foo = '0123e10'
SELECT CASE WHEN TRY_CAST(#foo AS integer) IS NULL AND #foo IS NOT NULL THEN 0 ELSE 1 END AS isInteger
This is the only really reliable way.
Should you need support for SQL-Server 2008, then fall back to Sam DeHaan's answer:
SELECT CASE WHEN ISNUMERIC(#foo + '.e0') = 1 THEN 1 ELSE 0 END AS isInteger
SQL-Server < 2012 (aka 2008R2) will reach end of (extended) support by 2019-07-09.
At this time, which is very soon, support for < 2012 can be dropped.
I wouldn't use any of the other hacks at this point in time anymore.
Just tell your frugal customers to update - it's been over 10 years since 2008.
See whether the below query will help
SELECT *
FROM MY_TABLE
WHERE CHARINDEX('.',MY_FIELD) = 0 AND CHARINDEX(',',MY_FIELD) = 0
AND ISNUMERIC(MY_FIELD) = 1 AND CONVERT(FLOAT,MY_FIELD) / 2147483647 <= 1
The following is correct for a WHERE clause; to make a function wrap it in CASE WHEN.
ISNUMERIC(table.field) > 0 AND PATINDEX('%[^0123456789]%', table.field) = 0
This work around with IsNumeric function will work:
select * from A where ISNUMERIC(x) =1 and X not like '%.%'
or Use
select * from A where x **not like** '%[^0-9]%'
declare #i numeric(28,5) = 12.0001
if (#i/cast(#i as int) > 1)
begin
select 'this is not int'
end
else
begin
select 'this is int'
end
As of SQL Server 2012, the TRY_CONVERT and TRY_CAST functions were implemented. Thise are vast improvements over the ISNUMERIC solution, which can (and does) give false positives (or negatives). For example if you run the below:
SELECT CONVERT(int,V.S)
FROM (VALUES('1'),
('900'),
('hello'),
('12b'),
('1.1'),
('')) V(S)
WHERE ISNUMERIC(V.S) = 1;
Using TRY_CONVERT (or TRY_CAST) avoids that:
SELECT TRY_CONVERT(int,V.S),
V.S,
ISNUMERIC(V.S)
FROM (VALUES('1'),
('900'),
('hello'),
('12b'),
('1.1'),
('')) V(S)
--WHERE TRY_CONVERT(int,V.S) IS NOT NULL; --To filter to only convertable values
Notice that '1.1' returned NULL, which cause the error before (as a string represtation of a decimal cannot be converted to an int) but also that '' returned 0, even though ISNUMERIC states the value "can't be converted".
Use TRY_CONVERT which is an SQL alternative to TryParse in .NET. IsNumeric() isn’t aware that empty strings are counted as (integer)zero, and that some perfectly valid money symbols, by themselves, are not converted to (money)zero. reference
SELECT #MY_VAR = CASE WHEN TRY_CONVERT(INT,MY_FIELD) IS NOT NULL THEN MY_FIELD
ELSE 0
END
FROM MY_TABLE
WHERE MY_OTHER_FIELD = 'MY_FILTER'
I think that there is something wrong with your database design. I think it is a really bad idea to mix varchar and numbers in one column? What is the reason for that?
Of course you can check if there are any chars other than [0-9], but imagine you have a 1m rows in table and your are checking every row. I think it won't work well.
Anyway if you really want to do it I suggest doing it on the client side.
I have a feeling doing it this way is the work of satan, but as an alternative:
How about a TRY - CATCH?
DECLARE #Converted as INT
DECLARE #IsNumeric BIT
BEGIN TRY
SET #Converted = cast(#ValueToCheck as int)
SET #IsNumeric=1
END TRY
BEGIN CATCH
SET #IsNumeric=0
END CATCH
select IIF(#IsNumeric=1,'Integer','Not integer') as IsInteger
This works, though only in SQL Server 2008 and up.
I tried this script and got the answer
ISNUMERIC(Replace(Replace([enter_your_number],'+','A'),'-','A') + '.0e0')
for example for up question this is answer:
SELECT #MY_VAR = CAST(MY_FIELD AS INT)
FROM MY_TABLE
WHERE MY_OTHER_FIELD = 'MY_FILTER' and ISNUMERIC(Replace(Replace(MY_FIELD,'+','A'),'-','A') + '.0e0') = 1
Why not just do something like:
CASE
WHEN ROUND(MY_FIELD,0)=MY_FIELD THEN CAST(MY_FIELD AS INT)
ELSE MY_FIELD
END
as MY_FIELD2
Sometimes you don't get to design the database, you just have to work with what you are given. In my case it's a database located on a computer that I only have read access to which has been around since 2008.
I need to select from a column in a poorly designed database which is a varchar with numbers 1-100 but sometimes a random string. I used the following to get around it (although I wish I could have re designed the entire database).
SELECT A from TABLE where isnumeric(A)=1
I am not a Pro in SQL but what about checking if it is devideable by 1 ?
For me it does the job.
SELECT *
FROM table
WHERE fieldname % 1 = 0
Use PATINDEX
DECLARE #input VARCHAR(10)='102030.40'
SELECT PATINDEX('%[^0-9]%',RTRIM(LTRIM(#input))) AS IsNumber
reference
http://www.intellectsql.com/post-how-to-check-if-the-input-is-numeric/
Had the same question. I finally used
where ATTRIBUTE != round(ATTRIBUTE)
and it worked for me
WHERE IsNumeric(value + 'e0') = 1 AND CONVERT(FLOAT, value) BETWEEN -2147483648 AND 2147483647
Seeing as this is quite old, but my solution isn't here, i thought to add another possible way to do this:
--This query only returns values with decimals
SELECT ActualCost
FROM TransactionHistory
where cast(ActualCost as int) != ActualCost
--This query only returns values without decimals
SELECT ActualCost
FROM TransactionHistory
where cast(ActualCost as int) = ActualCost
The easy part here is checking if the selected value is the same when cast as an integer.
we can check if its a non integer by
SELECT number2
FROM table
WHERE number2 LIKE '%[^0-9]%' and (( right(number2 ,len(number2)-1) LIKE '%[^0-9]%' and lefT(number2 ,1) <> '-') or ( right(number2 ,len(number2)-1) LIKE '%[^0-9]%' and lefT(number2 ,1) in ( '-','+') ) )
DECLARE #zip_code NCHAR(10)
SET #zip_code = '1239'
IF TRY_PARSE( #zip_code AS INT) / TRY_PARSE( #zip_code AS INT) = 1 PRINT 'integer'
ELSE PRINT 'not integer'
This works fine in SQL Server
SELECT (SELECT ISNUMERIC(2) WHERE ISNUMERIC(2)=1 AND 2 NOT LIKE '%.%')
Case
When (LNSEQNBR / 16384)%1 = 0 then 1
else 0
end
I have a temporary table I'm using for parsing, #rp.
#rp contains an nvarchar(max) column, #rp.col8, which holds positive and negative numbers to two decimal places of precision e.g. `1234.26'.
I'm able to run the following query and get out a set of converted values out:
select * from
(
select CONVERT(decimal(18,2),rp.col8) as PARSEAMT
from #rp
where
--#rp filtering criteria
)q
However, when I try to query for PARSEAMT = 0 in the following manner, I get the standard '8114, Error converting data type varchar to numeric.':
select * from
(
select CONVERT(decimal(18,2),col8) as PARSEAMT
from #rp
where
--#rp filtering criteria
)q
where q.PARSEAMT = 0
Without that where clause, the query runs fine and generates the expected values.
I've also tried other clauses like where q.PARSEAMT = 0.00 and where q.PARSEAMT = convert(decimal(18,2),0).
What am I doing wrong in my comparison?
I was going to suggest you select PARSEAMT into another temp-table/table-variable but I can see you've already done that from your comments.
Out of interest what does the following yield?
select
col8
from
#rp
where
-- ISNUMERIC returns 1 when the input expression evaluates to a valid
-- numeric data type; otherwise it returns 0. Valid numeric data types
-- include the following:
isnumeric(col8) <> 1
I have some dirty input data that is being imported into a raw source table within SQL Server (2008 R2). Fields that are defined as decimal(9,2) or decimal(4,2) by the input provider are coming in as strings, however, the strings do not always conform to the data definition (go figure!).
We import the data from flat files into the raw tables,then apply some conversion scripts to insert the 'cleaned' data into tables with the proper data types assigned to columns.
For instance:
raw_table
TotalAmount varchar(12)
clean_table
TotalAmount decimal(9,2)
Now, my question is this. If I want to do some 'basic' cleanup on this, I would want to do it in a function along the lines of:
CREATE FUNCTION [dbo].[StringToDecimal]
(
#conversionString VARCHAR(12)
)
RETURNS DECIMAL(9,2)
AS
BEGIN
DECLARE #rsp DECIMAL(9,2)
IF ISNUMERIC( LTRIM(RTRIM(REPLACE(#conversionString,' ',''))) ) = 1
BEGIN
SET #rsp = ISNULL( CONVERT( decimal(17,6), NULLIF( LTRIM(RTRIM(REPLACE(#conversionString,' ',''))),'') ), 0 )
END
ELSE
BEGIN
SET #rsp = 0 -- or we can return NULL here
END
RETURN #rsp
END
However, how could one go about supporting various sized decimals in this mix? Is there a way to parametrize the response type? I considered just returning a decimal of the largest size we generally see, then converting it again on the other end, however, you run into arithmetic overflow issues.
Would appreciate any thoughts/insight into solving this one!
Is there a way to parametrize the response type?
It's simpler than you think. Just return as a VARCHAR and do the casting to decimal(x,y) from the VARCHAR. You don't even need to cast - you can directly assign a VARCHAR (as long as it holds valid decimal data) to a decimal column/variable.
I will create 2 functions instead. StringToDecimal2 does the actual conversion, but returns one of 6 "error codes". You can use it to check why a string is invalid. Or use the wrapper dbo.StringToDecimal which just turns the invalid codes into NULL.
CREATE FUNCTION [dbo].[StringToDecimal2]
(
#conversionString VARCHAR(12),
#precision int, -- total digits
#scale int -- after decimal point
)
RETURNS VARCHAR(100)
AS
BEGIN
-- remove spaces, we'll allow this error. no need to trim
set #conversionString = REPLACE(#conversionString,' ','')
-- note: 1,234.56 (thousands separated) will be invalid, so will 1,234,56 (European decimals)
-- well, ok, let's clean up the thousands separators. BUT! It will incorrectly scale European decimals
set #conversionString = REPLACE(#conversionString,',','')
-- we don't support scientific notation either, so 1e4 (10,000) is out
if #conversionString like '%[^0-9.+-]%' return 'INVALID1' -- only digits and decimal are valid (plus +-)
if #conversionString like '%.%.%' return 'INVALID2' -- too many decimals
if #conversionString like '_%[+-]%' return 'INVALID3' -- +- symbol not in the first position
if #conversionString like '[.+-]' return 'INVALID4' -- a single character from "+-."
if #conversionString like '[+-].' return 'INVALID5' -- symbol and decimal only
-- add a decimal place so it is easier to work with below
if #conversionString not like '%.%'
set #conversionString = #conversionString + '.'
-- allow decimal places to go only as far as scale
set #conversionString = left(#conversionString, charindex('.', #conversionString)+#scale)
-- ensure the data is within precision number of digits in total
if charindex('.', #conversionString) > #precision - #scale + 1
return 'INVALID6' -- too many digits before decimal
RETURN #conversionString
END
GO
CREATE FUNCTION [dbo].[StringToDecimal]
(
#conversionString VARCHAR(12),
#precision int, -- total digits
#scale int -- after decimal point
)
RETURNS VARCHAR(100)
AS
BEGIN
RETURN case when [dbo].[StringToDecimal2](#conversionString, #precision, #scale) like 'INVALID%'
then null else [dbo].[StringToDecimal2](#conversionString, #precision, #scale) end
END
GO
Some tests:
select [dbo].[StringToDecimal2]('12342342', 9,2)
select convert(decimal(9,2),[dbo].[StringToDecimal]('1234234', 9,2))
select convert(decimal(9,2),[dbo].[StringToDecimal]('12342342', 9,2))
select convert(decimal(9,2),[dbo].[StringToDecimal]('123423.3333', 9,2))
select convert(decimal(20,10),[dbo].[StringToDecimal]('123423sd.3333', 20,10))
select convert(decimal(20,10),[dbo].[StringToDecimal]('123423sd..3333', 20,10))
select convert(decimal(20,10),[dbo].[StringToDecimal]('-123423.3333', 20,10))
select convert(decimal(20,10),[dbo].[StringToDecimal]('+123423..3333', 20,10))
Thanks for the extra information. It sounds like you have three steps:
Remove all characters from the string that are not digits or a decimal point (do you ever get multiple points in one string?)
Convert to (9,5) or (4,1) as appropriate (how do you decide this? is there rounding? does 10X.781 become 10.78100 or 10.7 or 10.8?)
Insert/update the final value somewhere
Based on point 1 alone, I would immediately avoid TSQL and think about an external script or CLR procedure. A CLR function could do the parsing, but you still have the problem of returning different data types.
Since this appears to be some kind of ETL task, in my environment I would probably implement it as a script component in an SSIS package. The component would do the parsing and send the clean data to different outputs for further processing. If it was a one-time task I would use a Python script to parse the input data and generate INSERT or UPDATE statements.
I don't know if any of those solutions are suitable for you, but maybe it'll give you some ideas. And you should probably avoid the ISNUMERIC() function; search this site or Google to find some of the 'strange' input that it considers to be numeric.