ISJSON function in SQL server is inconsistent - sql-server
I'm using SQL server 2016 (SP3) enterprise edition.
I faced strange behavior. Took two different VALID JSONs defined as VARCHAR(8000) and checked whether they valid. One of them is 2484 length and one 4294 length.
One (the short one) returned true (valid) and one (long one) false (0=not valid).
Once I cast the not valid JSON to VARCHAR(max) - it returned true.
What's happen here?
DECLARE #parameter VARCHAR(8000)='{"ExpressionsArray":[{"Priority":2,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":11,"AttributeFieldName":"STR101","OperatorId":"14","OperatorValue":"Contains","Value":"ci","FormatType":"text"}],"Value":"city good2"},{"Priority":3,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":12,"AttributeFieldName":"FIELD339","OperatorId":"11","OperatorValue":"Between","Value":"70,80","FormatType":"between numbers"}],"Value":"very good"},{"Priority":4,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":13,"AttributeFieldName":"STR1","OperatorId":"11","OperatorValue":"Between","Value":"2022-08-02,2022-08-05","FormatType":"between dates"}],"Value":"date between 1"},{"Priority":5,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":14,"AttributeFieldName":"STR11","OperatorId":"5","OperatorValue":"Equals","Value":"lan","FormatType":"text"}],"Value":"lan"},{"Priority":6,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":15,"AttributeFieldName":"STR6","OperatorId":"26","OperatorValue":"This week","Value":"","FormatType":"disable"}],"Value":"This week date"},{"Priority":7,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":16,"AttributeFieldName":"STR7","OperatorId":"19","OperatorValue":"Today","Value":"","FormatType":"disable"}],"Value":"Today - date"},{"Priority":8,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":17,"AttributeFieldName":"STR9","OperatorId":"18","OperatorValue":"Tomorrow","Value":"","FormatType":"disable"}],"Value":"Tomorrow: date"},{"Priority":9,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":18,"AttributeFieldName":"STR5","OperatorId":"20","OperatorValue":"Yesterday","Value":"","FormatType":"disable"}],"Value":"Yesterday -u003e date"},{"Priority":10,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":19,"AttributeFieldName":"FIELD1","OperatorId":"7","OperatorValue":"u003e","Value":"8","FormatType":"number"}],"Value":"Number u003e5 $ test"},{"Priority":11,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":20,"AttributeFieldName":"STR0","OperatorId":"5","OperatorValue":"Equals","Value":"cat","FormatType":"text"}],"Value":"String u0026 Equals"},{"Priority":12,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":21,"AttributeFieldName":"FIELD2","OperatorId":"5","OperatorValue":"=","Value":"12","FormatType":"number"}],"Value":"number = 12"}],"ElseValue":"testik","Format":4,"DisplayName":"Cripto Test","Description":"desc: Cripto Test","FieldName":" ","PublishStatus":"NotPublished","Name":"CriptoTest","Type":"conditional","AttributeBaseType":"string","IsPersonalization":false}'
DECLARE #parameter1 VARCHAR(8000)='{"ExpressionsArray":[{"Priority":1,"ShowComplexExpression":false,"ComplexExpression":"#1 and #2 and #3 and #4 and #5 and #6 and #7 and #8 and #9 and #10","Conditions":[{"Position":1,"AttributeFieldName":"STR101","OperatorId":"14","OperatorValue":"Contains","Value":"ci","FormatType":"text"},{"Position":2,"AttributeFieldName":"STR13","OperatorId":"11","OperatorValue":"Between","Value":"2022-08-08,2022-08-12","FormatType":"between dates"},{"Position":3,"AttributeFieldName":"FIELD353","OperatorId":"11","OperatorValue":"Between","Value":"15,19","FormatType":"between numbers"},{"Position":4,"AttributeFieldName":"STR0","OperatorId":"14","OperatorValue":"Contains","Value":"cate","FormatType":"text"},{"Position":5,"AttributeFieldName":"FIELD362","OperatorId":"6","OperatorValue":"u003cu003e","Value":"5","FormatType":"number"},{"Position":6,"AttributeFieldName":"STR28","OperatorId":"13","OperatorValue":"Ends with","Value":"a","FormatType":"text"},{"Position":7,"AttributeFieldName":"STR15","OperatorId":"21","OperatorValue":"One of","Value":"tt","FormatType":"text"},{"Position":8,"AttributeFieldName":"FIELD522","OperatorId":"10","OperatorValue":"u003c=","Value":"7","FormatType":"number"},{"Position":9,"AttributeFieldName":"FIELD649","OperatorId":"6","OperatorValue":"u003cu003e","Value":"5","FormatType":"number"},{"Position":10,"AttributeFieldName":"STR3","OperatorId":"14","OperatorValue":"Contains","Value":"bobo","FormatType":"text"}],"Value":"city good"},{"Priority":2,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":11,"AttributeFieldName":"STR101","OperatorId":"14","OperatorValue":"Contains","Value":"ci","FormatType":"text"}],"Value":"city good2"},{"Priority":3,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":12,"AttributeFieldName":"FIELD339","OperatorId":"11","OperatorValue":"Between","Value":"70,80","FormatType":"between numbers"}],"Value":"very good"},{"Priority":4,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":13,"AttributeFieldName":"STR1","OperatorId":"11","OperatorValue":"Between","Value":"2022-08-02,2022-08-05","FormatType":"between dates"}],"Value":"date between 1"},{"Priority":5,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":14,"AttributeFieldName":"STR11","OperatorId":"5","OperatorValue":"Equals","Value":"lan","FormatType":"text"}],"Value":"lan"},{"Priority":6,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":15,"AttributeFieldName":"STR6","OperatorId":"26","OperatorValue":"This week","Value":"","FormatType":"disable"}],"Value":"This week date"},{"Priority":7,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":16,"AttributeFieldName":"STR7","OperatorId":"19","OperatorValue":"Today","Value":"","FormatType":"disable"}],"Value":"Today - date"},{"Priority":8,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":17,"AttributeFieldName":"STR9","OperatorId":"18","OperatorValue":"Tomorrow","Value":"","FormatType":"disable"}],"Value":"Tomorrow: date"},{"Priority":9,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":18,"AttributeFieldName":"STR5","OperatorId":"20","OperatorValue":"Yesterday","Value":"","FormatType":"disable"}],"Value":"Yesterday -u003e date"},{"Priority":10,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":19,"AttributeFieldName":"FIELD1","OperatorId":"7","OperatorValue":"u003e","Value":"8","FormatType":"number"}],"Value":"Number u003e5 $ test"},{"Priority":11,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":20,"AttributeFieldName":"STR0","OperatorId":"5","OperatorValue":"Equals","Value":"cat","FormatType":"text"}],"Value":"String u0026 Equals"},{"Priority":12,"ShowComplexExpression":false,"ComplexExpression":"#1","Conditions":[{"Position":21,"AttributeFieldName":"FIELD2","OperatorId":"5","OperatorValue":"=","Value":"12","FormatType":"number"}],"Value":"number = 12"}],"ElseValue":"testik","Format":4,"DisplayName":"Cripto Test","Description":"desc: Cripto Test","FieldName":" ","PublishStatus":"NotPublished","Name":"CriptoTest","Type":"conditional","AttributeBaseType":"string","IsPersonalization":false}'
SELECT ISJSON(#parameter) -- #parameter length 2848 - return 1 - OK
SELECT ISJSON(#parameter1) -- #parameter length 4294 return 0 - NOT OK
SELECT ISJSON(CAST(#parameter1 AS VARCHAR(max)) ) -- return 1 - OK
The problem is specifically to do with your choice of data type, varchar(8000). Though not documented, it is very likely that ISJSON (at least in SQL Server 2016) expects an nvarchar, however, you have defined a varchar. As a result, when the value is passed to ISJSON it is implicitly cast to an nvarchar.
As, however, this value isn't a MAX value, and SQL Server doesn't normally implicitly convert a non-MAX value to a MAX one, this means that the value is implicitly converted to an nvarchar(4000), and so for your value which is over 4,000 characters in length, truncation occurs.
As the value is truncated, this then causes the value being checked to no longer be valid, and so ISJSON returns 0, instead of 1.
The solution, therefore, would be to define your parameters as an nvarchar(MAX); which is the correct data type for JSON in SQL Server.
This, as lptr highlights, appears to only be a problem in SQL Server 2016 (2016 vs 2017 & 2019). what Microsoft's fix was under the hood is unclear; I can't find any documentation or bug fixes addressing the matter. I would assume that the data type isn't implicitly cast to an nvarchar and a varchar is used, but it might implicitly cast it to an nvarchar(MAX).
Related
Odd behaviour of rowstamp column in SQL Server
Is the use of the datatype rowversion in a table always reliable? I have seen a problem in comparing a value to a column of this datatype (ie select * from MyTable where tStamp > #value) and on investigating this - I have found some odd results. I am not sure if this is my misunderstanding, or a bigger issue that means the datatype cannot be relied on. In the database is a table (MyTable) and the tStamp column is a rowversion datatype in which the minimum value is 0x00000004355B68B7. There is a stored procedure which accepts as input a parameter of the type rowversion, but that in turn calls a procedure that uses an input parameter of data type binary(8). So the first procedure is declared like create procedure proc1 #inputTS rowversion and the second is declared like create procedure proc2 #inputTS binary(8) I assumed initially that the problem I was seeing was due to the fact that the datatypes were different (when proc1 calls proc2 it passes the value #inputTS over to it). And when I changed proc2 to use rowversion, I got the expected results. But then I tried a number of tests and what I saw from those ... was odd. If I use the value 0x0000000070000000: declare #testRV rowversion = 0x0000000070000000 declare #testBin binary(8) = #testRV select * from MyTable where tStamp > #testRV -- First Query select * from MyTable where tStamp > #testBin -- Second Query I found that the first select returned no rows and the second returned all rows. They should both have returned all rows. If I changed the value to 0x000000006FFFFFFF or 0x0000000070000001, then both queries did return all rows. If I use the value 0x0000000080000000 then both queries return no rows. Using this value - 0x0000000441CED675 - both returned rows but this value - 0x0000000481CED675 - neither returned rows. And then using this value - 0x0000000080000001 - the first query returned all rows, the second returned none At one point I was thinking that internally, SQL was treating the values as 2 integers - ie 0x0000000080000000 was 0x00000000 and 0x80000000. Since that's one higher than the max int size and is treated as -2147483648, it looked like the problem was where the "lower" integer was being treated as a negative. But that doesn't explain the behaviour when using 0x0000000070000000 or 0x0000000080000001. I have tried this on SQL Server versions from 2008 R2 to 2017 and got the same results each time. Am I missing something about how rowversion can be used?
How to overcome limitation of ISNUMERIC column "ERROR : overflowed an int column"
I have a requirement wherein I have to check whether value entered in VARCHAR(200) column is numeric or not, and if so a relevant error message should be passed. I am validating the same with ISNUMERIC function since my column value is varchar so user can enter more than 10 characters as well due to which I am getting this error: overflowed an int column Because of the other business support, I can not change the data type of the column to int. As of now I have implemented LEN() < 10 condition before checking ISNUMERIC but seeking if any alternate and better option available.
If you work on Sql server 2012 Than its better to Use TRY_Convert() Function.it will give NULL as output rather than impose error declare #d varchar(200)='940852774565564' if ((select ISNUMERIC(#d))=1) select Try_Convert(#d as bigint) else Convert the value to bigint rather than INT declare #d varchar(200)='940852774565564' if ((select ISNUMERIC(#d))=1) select cast(#d as bigint)
ms sql server executes 'then' before 'when' in case
when i try to select select case when (isnumeric(SUBSTRING([VZWECK2],1,9)) = 1) then CONVERT(decimal,SUBSTRING([VZWECK2],1,9)) else null end as [NUM] from table sql-server gives me: Msg 8114, Level 16, State 5, Line 2 Error converting data type varchar to numeric. [VZWECK2] is a char(27). is this a known bug? because it seems to me it executes the convert before it does the case, which defies the purpose of my select. i know that there are values that are not numeric obviously, which is why i need the case statement to weed them out. for some reason selecting select case when (isnumeric(SUBSTRING([VZWECK2],1,9)) = 1) then 99 else null end as [NUM] from table yields no errors and behaves as expected
The problem is that ISNUMERIC is very forgiving, and that ISNUMERIC returns 1 is unfortunately no guarantee that CONVERT will work. This is why SQL Server 2012 and later introduced TRY_CAST and TRY_CONVERT. If you are converting whole numbers, a more reliable check is to make sure the string consists of only digits with NOT LIKE '%[^0-9]%' (that is, it must not contain a non-digit anywhere). This is too restrictive for some formats (like floating point) but for integers it works nicely.
Do you know the value which throws the error? IsNumeric is not exactly fool-proof, for example:
select ISNUMERIC('$')
select ISNUMERIC('+')
select ISNUMERIC('-')
all yield 1
Alternatively, you could go with TRY_PARSE instead.
Edit: TRY_PARSE is introduced in sql server 2012, so may not be available to you.
Why is T-SQL ISNULL() truncating the string and COALESCE is not?
Given the following:
SELECT ISNULL('XY' + NULL, 'ABCDEFGHIJ') -- Outputs ABC (Why?)
SELECT COALESCE('XY' + NULL, 'ABCDEFGHIJ') -- Outputs ABCDEFGHIJ
Why are these statements returning different results?
According to Microsoft documentation, for function: ISNULL(check_expression, replacement_value) replacement_value must be of a type that is implicitly convertible to the type of check_expression. Note that type for 'xy'+NULL is VARCHAR(3). Because of this your string 'ABCDEFGHIJ' is cast to VARCHAR(3) and thus trimmed. It sounds strange why it is not VARCHAR(2), but this is the way it is - one character longer than 'xy'. You can play with this SQLFiddle and see for yourself that type for 'xy'+NULL is the same as for expression CASE WHEN 1=2 THEN 'XYZ' ELSE NULL END, which is NULL but is implicitly compatible to VARCHAR(3). It seems that for expression 'xy'+NULL perceived length can be computed as 'xy' string length (2) plus 1 for every NULL added. For example, type of 'xy'+NULL+NULL is VARCHAR(4), type for 'xy'+NULL+NULL+NULL is VARCHAR(5) and so on - check out this SQLFiddle. This is extremely weird, but that is how MS SQL Server 2008 and 2012 work.
You can check all the difference here, its very clear MSDN : http://msdn.microsoft.com/en-us/library/ms190349.aspx MSDN Blog : http://blogs.msdn.com/b/sqltips/archive/2008/06/26/differences-between-isnull-and-coalesce.aspx
ISNULL() converts the replacement value to the type of the check expression. In this case, the type of the check expression is CHAR(2), so converting the replacement value truncates it (are you sure you're getting ABC and not just AB?). From the Microsoft documentation: replacement_value can be truncated if replacement_value is longer than check_expression.
T-SQL data type REAL FLOAT(n) on SQL Server
Is anyone aware of the reasons why the following line of T-SQL code works (under SQL Server 2008)? SELECT cast(2 as real(10)) I expect the server to complain in the very same way it does when I attempt this: declare #x real(10) set #x = 4.1234567 select #x and I get: Msg 2716, Level 16, State 1, Line 16 Column, parameter, or variable #2: Cannot specify a column width on data type real. I realise that for MS SQL 2008 there is a data type REAL equivalent to FLOAT(24). I just would like to know why CAST does not complain, i.e., it was possible to write nonsense as REAL(4,2) within CAST (under SQL 2000), it works fine with REAL(10) under SQL 2008 (again in CAST)? Any suggestions and clarification appreciated. Thank you.
It would appear that CAST (and CONVERT) translates type synonyms (such as real) into their underlying type (i.e. float), for which length specifiers are valid. For another example, you can do: SELECT CONVERT(sysname(10),'abcdefghijkl') But you cannot declare a variable of type sysname(10), since sysname is a synonym for nvarchar(128) (or varchar(30), depending on SQL version) In short, I'd say it's a quirk of how CAST/CONVERT work, but shouldn't be relied upon.