The sql-server OPENJSON() function can take a json array and convert it into sql table with key-value pairs, e.g.:
DECLARE #json NVARCHAR(MAX);
SET #json = '{
"key1": "val1",
"key2": "val2",
"key3": "val3"
}';
SELECT * FROM OPENJSON(#json, '$')
Result:
key value type
--------------------
key1 val1 1
key2 val2 1
key3 val3 1
What is the best general-purpose method for converting this key/value table back into a json array?
Why? If we can do this with a single function, it opens up a range of json modifications which are otherwise not possible on sql server, e.g.:
Re-order elements
Rename properties (key names)
Split json array into smaller arrays / combine json arrays
Compare json arrays (which key/value elements exists in both jsons? What are the differences?)
Clean json (remove syntactical whitespace/newlines to compress it)
Now, I could start to do simple CONCAT('"',[key],'":"',[value]), then do a comma-list-aggregration. But if I want a code that is both easy to apply across my codebase and works for all data types, this is not a simple task. By looking at the json format definition, the conversion should take into account a) the 6 different data types, b) escape characters, c) SQL NULL/json null handling, d) what I may have overlooked I.e. at minimum, the below example should be supported:
DECLARE #test_json NVARCHAR(MAX);
SET #test_json = '{
"myNull": null,
"myString": "start_\\_\"_\/_\b_\f_\n_\r_\t_\u2600_stop",
"myNumber": 3.14,
"myBool": true,
"myArray": ["1", 2],
"myObject": {"key":"val"}
}'
SELECT * FROM OPENJSON(#test_json, '$')
Result:
key value type
------------------------------------------------
myNull NULL 0
myString start_\_"_/___ _ _ _☀_stop 1
myNumber 3.14 2
myBool true 3
myArray ["1", 2] 4
myObject {"key":"val"} 5
For the string-aggregation part, we have long suffered the 'FOR XML PATH'-pain. Luckily we have STRING_AGG() on SQL2017/AzureDB, and I will accept a solution depending on STRING_AGG().
You can do with this command, using FOR JSON
select * from table for json auto
My result:
[{"LogId":1,"DtLog":"2017-09-30T21:04:45.6700000","FileId":1},
{"LogId":2,"DtLog":"2017-09-30T21:08:35.8633333","FileId":3},{"LogId":3,"DtLog":"2017-09-30T21:08:36.4433333","FileId":2},{"LogId":4,"DtLog":"2017-09-30T21:08:36.9866667","FileId":12},{"LogId":5,"DtLog":"2017-09-30T21:15:22.5366667","FileId":13},{"LogId":6,"DtLog":"2017-09-30T21:38:43.7866667","FileId":17}]
I use string_agg
declare #json table ( Name varchar(80), Value varchar(max) )
insert into #json
select [Key], Value from openjson(#attributes)
insert into #json values ( 'name', #name )
insert into #json values ( 'title', #title )
insert into #json values ( 'description', #description )
set #attributes = '{' + (select STRING_AGG( '"' + Name + '":"' +
REPLACE (value, '"', '\"' ) +'"', ',') from #json) + '}'
Related
This is the JSON definition that is going to be provided (just a short example) and the code that I have implemented to get the expected result:
declare #json nvarchar(max)
set #json = '{
"testJson":{
"testID":"Test1",
"Value":[
{
"Value1":"",
"Value2":"",
"Value3":"",
"Type": "1A"
},
{
"Value1":"123",
"Value2":"456",
"Value3":"Automatic",
"Type": "2A"
},
{
"Value1":"789",
"Value2":"159",
"Value3":"Manual",
"Value4":"Success" ,
"Type": "3A"
}
]
}
}'
select
'ValueFields' as groupDef,
-- b.[key],
-- c.[key],
STRING_AGG( c.value , ' | ') as val
from
openjson(#json, '$.testJson.Value') as b
cross apply
openjson(b.value) as c
where
b.[key] not in (select b.[key]
from openjson(#json, '$.testJson.Value') as b
where b.value like ('%1A%'))
As you can see each element in the array can have different quantity of attributes (value1,.., value4..), and I only need to consider those elements where the type attribute is not equal to "1A". The query gives me the result requested, however, I am wondering how can I improve the performance of the code given that I'm using the like operator in the sub select, and obviously the original JSON file could a considerable number of elements in the array.
…
select b.Value --,c.value
from
openjson(#json, '$.testJson.Value')
with
(
Value nvarchar(max) '$' as json,
Type varchar(100) '$.Type'
) as b
--cross apply openjson(b.Value) as c
where b.Type <> '1A'
SELECT
'ValueFields' as groupDef,
J.value as val
FROM
OPENJSON(#json,'$.testJson.Value') J
WHERE
JSON_VALUE([value],'$.Type') <> '1A'
I have a column in SQL table that has json value like below:
[
{"address":{"value":"A9"},
"value":{"type":11,"value":"John"}},
{"address":{"value":"A10"},
"value":{"type":11,"value":"Doe"}}]
MSDN Examples for JSON_VALUE or JSON_QUERY require a json object at root. How can I query above to return rows that have "address" as A9 and "value" as John? I'm using SQL Azure.
Something like this:
declare #json nvarchar(max) = '[
{"address":{"value":"A9"},
"value":{"type":11,"value":"John"}},
{"address":{"value":"A10"},
"value":{"type":11,"value":"Doe"}}]'
select a.*
from openjson(#json) r
cross apply openjson(r.value)
with (
address nvarchar(200) '$.address.value',
name nvarchar(200) '$.value.value'
) a
where address = N'A9'
and name = N'John'
outputs
address name
------- -----
A9 John
(1 row affected)
It may not be entirely relevant to the OP's post as the usage is different, however it is possible to retrieve arbitrary items from a root-level unnamed JSON array e.g.
declare #json nvarchar(max) = '[
{"address":
{"value":"A9"},
"value":
{"type":11,"value":"John"}
},
{"address":
{"value":"A10"},
"value":
{"type":11,"value":"Doe"}
}
]'
select
JSON_VALUE(
JSON_QUERY(#json, '$[0]'),
'$.address.value') as 'First address.value',
JSON_VALUE(
JSON_QUERY(#json, '$[1]'),
'$.address.value') as 'Second address.value'
Output :
First address.value Second address.value
A9 A10
In our SQL Server table we have a json object stored with an array of strings. I want to programatically split that string into several columns. However, I cannot seem to get it to work or even if it's possible.
Is this a possibility to create multiple columns within the WITH clause or it is a smarter move to do it within the select statement?
I trimmed down some of the code to give a simplistic idea of what's given.
The example JSON is similar to { "arr": ["str1 - str2"] }
SELECT b.* FROM [table] a
OUTER APPLY
OPENJSON(a.value, '$.arr')
WITH
(
strSplit1 VARCHAR(100) SPLIT('$.arr', '-',1),
strSplit2 VARCHAR(100) SPLIT('$.arr', '-',2)
) b
Due to the tag [tsql] and the usage of OPENJSON I assume this is SQL-Server. But might be wrong... Please always specify your RDBMS (with version).
Your JSON is rather weird... I think you've overdone it while trying to simplify this for brevity...
Try this:
DECLARE #tbl TABLE(ID INT IDENTITY,YourJSON NVARCHAR(MAX));
INSERT INTO #tbl VALUES(N'{ "arr": ["str1 - str2"] }') --weird example...
,(N'{ "arr": ["a","b","c"] }'); --array with three elements
SELECT t.ID
,B.[value] AS arr
FROM #tbl t
CROSS APPLY OPENJSON(YourJSON)
WITH(arr NVARCHAR(MAX) AS JSON) A
CROSS APPLY OPENJSON(A.arr) B;
A rather short approach (but fitting to this simple example only) was this:
SELECT t.ID
,A.*
FROM #tbl t
OUTER APPLY OPENJSON(JSON_QUERY(YourJSON,'$.arr')) A
Hint
JSON support was introduced with SQL-Server 2016
UPDATE: If the JSON's content is a weird CSV-string...
There's a trick to transform a CSV into a JSON-array. Try this
DECLARE #tbl TABLE(ID INT IDENTITY,YourJSON NVARCHAR(MAX));
INSERT INTO #tbl VALUES(N'{ "arr": ["str1 - str2"] }') --weird example...
,(N'{ "arr": ["a","b","c"] }') --array with three elements
,(N'{ "arr": ["x-y-z"] }'); --array with three elements in a weird CSV format
SELECT t.ID
,B.[value] AS arr
,C.[value]
FROM #tbl t
CROSS APPLY OPENJSON(YourJSON)
WITH(arr NVARCHAR(MAX) AS JSON) A
CROSS APPLY OPENJSON(A.arr) B
CROSS APPLY OPENJSON('["' + REPLACE(B.[value],'-','","') + '"]') C;
Some simple replacements in OPENJSON('["' + REPLACE(B.[value],'-','","') + '"]') will create a JSON array out of your CSV-string, which can be opened in OPENJSON.
I'm not aware of any way to split a string within JSON. I wonder if the issue is down to your JSON containing a single string rather than multiple values?
The below example shows how to extract each string from the array; and if you wish to go further and split those strings on the hyphen, shows how to do that using SQL's normal SUBSTRING and CHARINDEX functions.
create table [table]
(
value nvarchar(max)
)
insert [table](value)
values ('{ "arr": ["str1 - str2"] }'), ('{ "arr": ["1234 - 5678","abc - def"] }')
SELECT b.value
, rtrim(substring(b.value,1,charindex('-',b.value)-1))
, ltrim(substring(b.value,charindex('-',b.value)+1,len(b.value)))
FROM [table] a
OUTER APPLY OPENJSON(a.value, '$.arr') b
If you want all values in a single column, you can use the string_split function: https://learn.microsoft.com/en-us/sql/t-sql/functions/string-split-transact-sql?view=sql-server-2017
SELECT ltrim(rtrim(c.value))
FROM [table] a
OUTER APPLY OPENJSON(a.value, '$.arr') b
OUTER APPLY STRING_SPLIT(b.value, '-') c
I have variable called #prmclientcode which is nvarchar. The input to this variable can be a single client code or multiple client codes separated by comma. For e.g.
#prmclientcode='1'
or
#prmclientcode='1,2,3'.
I am comparing this variable to a client code column in of the tables. The data type of this column is numeric(6,0). I tried converting the variable data type like below
SNCA_CLIENT_CODE IN ('''+convert(numeric(6,0),#prmclientcode+''')) (The query is inside a dynamic sql).
But when I try executing this I get the error
Arithmetic overflow error converting nvarchar to data type numeric.
Can anyone please help me here!
Thanks!
You need to convert the numeric(6,0) column to nvarchar data type. You can use below scrip to convert it to nvarchar, before processing:
SNCA_CLIENT_CODE IN ('''+convert(cast( numeric(6,0) as nvarchar(max) ),#prmclientcode+'''))
Please try with the below code snippet.
DECLARE #ProductTotals TABLE
(
ProductID int
)
INSERT INTO #ProductTotals VALUES(1)
INSERT INTO #ProductTotals VALUES(11)
INSERT INTO #ProductTotals VALUES(3)
DECLARE #prmclientcode VARCHAR(MAX)='1'
SELECT * FROM #ProductTotals
SELECT * FROM #ProductTotals WHERE CHARINDEX(',' + CAST(ProductID AS VARCHAR(MAX)) + ',' , ',' + ISNULL(#prmclientcode,ProductID) + ',') > 0
Let me know if any concern.
use following code in order to separate your variable:
DECLARE
#T VARCHAR(100) = '1,2,3,23,342',
#I int = 1
;WITH x(I, num) AS (
SELECT 1, CHARINDEX(',',#T,#I)
UNION ALL
SELECT num+1,CHARINDEX(',',#T,num+1)
FROM x
WHERE num+1<LEN(#T)
AND num<>0
)
SELECT SUBSTRING(#T,I,CASE WHEN num=0 THEN LEN(#T)+1 ELSE num END -I)
FROM x
Use can use either table function or dynamic sql query, both options will work.
Let me know if you need more help
We have a result set that has three fields and each of those fields is either null or contains a comma separated list of strings.
We need to combine all three into one comma separated list and eliminate duplicates.
What is the best way to do that?
I found a nice function that can split a string and return a table:
T-SQL split string
I tried to create a UDF that would take three varchar parameters and call that split string function three times, combine them into one table, and then use a FOR XML from there and return it as one comma separated string.
But SQL is complaining about having a SELECT in a function.
Here's an example using the SplitString function you referenced.
DECLARE
#X varchar(max) = 'A, C, F'
, #Y varchar(max) = null
, #Z varchar(max) = 'A, D, E, A'
;WITH SplitResults as
(
-- Note: the function does not remove leading spaces.
SELECT LTRIM([Name]) [Name] FROM SplitString(#X)
UNION
SELECT LTRIM([Name]) [Name] FROM SplitString(#Y)
UNION
SELECT LTRIM([Name]) [Name] FROM SplitString(#Z)
)
SELECT STUFF((
SELECT ', ' + [Name]
FROM SplitResults
FOR XML PATH(''), TYPE
-- Note: here we're pulling the value out in case any characters were escaped, ie. &
-- and then STUFF is removing the leading ,<space>
).value('.', 'nvarchar(max)'), 1, 2, '')
I would not store data as a comma separated string in a single field. Separate the string to a new table and combine it to a string again when you need to.
Finding duplicates and managing the data will also be much easier.
I've used this function before (I didn't write it, and unfortunately cannot remember where I found it) to split a string and add a key (in this case an int) to the data as a separate table, linking back to the original table's PK
CREATE FUNCTION SplitWithID (#id int, #sep VARCHAR(10), #s VARCHAR(MAX))
RETURNS #t TABLE
(
id int,
val VARCHAR(MAX)
)
AS
BEGIN
DECLARE #xml XML
SET #XML = N'<root><r>' + REPLACE(#s, #sep, '</r><r>') + '</r></root>'
INSERT INTO #t(id,val)
SELECT #id, r.value('.','VARCHAR(40)') as Item
FROM #xml.nodes('//root/r') AS RECORDS(r)
RETURN
END
GO
Once you have the data on separate rows you can use any duplicate removal technique to clean the data before applying a primary key to the table.