MariaDB JSON_ARRAY_APPEND Object instead of it's string representation - arrays

I am receiving quoted string values in an array instead of an array of objects when I use JSON_ARRAY_APPEND() to insert a string that represents an object.
I need a way to force the value inserted to the array to be the object instead of it's string representation.
Server:
10.2.18-MariaDB-log
MariaDB Server
Linux x86_64
Here is a sample I am trying to get work:
set #NewArrayItem = '{"item2": "value2"}';
SELECT JSON_ARRAY_APPEND('{"SomeData": "SomeValue", "AnArray": [{"item1": "value1"}]}', '$.AnArray', #NewArrayItem ) as outval;
The second element in the array ($.AnArray[1]) is a string instead of an object.
I am expecting:
{"SomeData": "SomeValue", "AnArray": [{"item1": "value1"}, {"item2": "value2"}]}
But I actually get:
{"SomeData": "SomeValue", "AnArray": [{"item1": "value1"}, "{\"item2\": \"value2\"}"]}
I see that the following works, but my constraint is that value #NewArrayItem is a properly formatted string from another application:
SELECT JSON_ARRAY_APPEND('{"SomeData": "SomeValue", "AnArray": [{"item1": "value1"}]}', '$.AnArray', JSON_OBJECT('item2','value2') ) as outval;

I solved this with a combination of JSON_SET, JSON_MERGE, and JSON_QUERY:
set #ExistingData = '{"SomeData": "SomeValue", "AnArray": [{"item1": "value1"}]}';
set #NewArrayItem = '{"item2": "value2"}';
SELECT JSON_SET(#ExistingData, '$.AnArray', JSON_MERGE(ifnull(JSON_QUERY(#ExistingData, '$.AnArray'),'[]'),#NewArrayItem) ) as outval;
As a bonus, it also works for the case where the array does not exist:
set #ExistingData = '{"SomeData": "SomeValue"}';
set #NewArrayItem = '{"item2": "value2"}';
SELECT JSON_SET(#ExistingData, '$.AnArray', JSON_MERGE(ifnull(JSON_QUERY(#ExistingData, '$.AnArray'),'[]'),#NewArrayItem) ) as outval;
Still looking for a more simple answer.

I try this example
SET #json = '[]';
SELECT #json; // []
SET #json = JSON_ARRAY_APPEND(#json, '$', JSON_OBJECT("id", 1, "name", "Month1"));
SELECT #json; // [{"id": 1, "name": "Month1"}]
SET #json = JSON_ARRAY_APPEND(#json, '$', JSON_OBJECT("id", 2, "name", "Month2"));
SELECT #json; // [{"id": 1, "name": "Month1"}, {"id": 2, "name": "Month2"}]

Related

Parsing string with multiple delimiters into columns

I want to split strings into columns.
My columns should be:
account_id, resource_type, resource_name
I have a JSON file source that I have been trying to parse via ADF data flow. That hasn't worked for me, hence I flattened the data and brought it into SQL Server (I am open to parsing values via ADF or SQL if anyone can show me how). Please check the JSON file at the bottom.
Use this code to query the data I am working with.
CREATE TABLE test.test2
(
resource_type nvarchar(max) NULL
)
INSERT INTO test.test2 ([resource_type])
VALUES
('account_id:224526257458,resource_type:buckets,resource_name:camp-stage-artifactory'),
('account_id:535533456241,resource_type:buckets,resource_name:tni-prod-diva-backups'),
('account_id:369798452057,resource_type:buckets,resource_name:369798452057-s3-manifests'),
('account_id:460085747812,resource_type:buckets,resource_name:vessel-incident-report-nonprod-accesslogs')
The output that I should be able to query in SQL Server should like this:
account_id
resource_type
resource_name
224526257458
buckets
camp-stage-artifactory
535533456241
buckets
tni-prod-diva-backups
and so forth.
Please help me out and ask for clarification if needed. Thanks in advance.
EDIT:
Source JSON Format:
{
"start_date": "2021-12-01 00:00:00+00:00",
"end_date": "2021-12-31 23:59:59+00:00",
"resource_type": "all",
"records": [
{
"directconnect_connections": [
"account_id:227148359287,resource_type:directconnect_connections,resource_name:'dxcon-fh40evn5'",
"account_id:401311080156,resource_type:directconnect_connections,resource_name:'dxcon-ffxgf6kh'",
"account_id:401311080156,resource_type:directconnect_connections,resource_name:'dxcon-fg5j5v6o'",
"account_id:227148359287,resource_type:directconnect_connections,resource_name:'dxcon-fgvfo1ej'"
]
},
{
"virtual_interfaces": [
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:'dxvif-fgvj25vt'",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:'dxvif-fgbw5gs0'",
"account_id:401311080156,resource_type:virtual_interfaces,resource_name:'dxvif-ffnosohr'",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:'dxvif-fg18bdhl'",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:'dxvif-ffmf6h64'",
"account_id:390251991779,resource_type:virtual_interfaces,resource_name:'dxvif-fgkxjhcj'",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:'dxvif-ffp6kl3f'"
]
}
]
}
Since you don't have a valid JSON string and not wanting to get in the business of string manipulation... perhaps this will help.
Select B.*
From test2 A
Cross Apply ( Select account_id = max(case when value like 'account_id:%' then stuff(value,1,11,'') end )
,resource_type = max(case when value like 'resource_type:%' then stuff(value,1,14,'') end )
,resource_name = max(case when value like 'resource_name:%' then stuff(value,1,14,'') end )
from string_split(resource_type,',')
)B
Results
account_id resource_type resource_name
224526257458 buckets camp-stage-artifactory
535533456241 buckets tni-prod-diva-backups
369798452057 buckets 369798452057-s3-manifests
460085747812 buckets vessel-incident-report-nonprod-accesslogs
Unfortunately, the values inside the arrays are not valid JSON. You can patch them up by adding {} to the beginning/end, and adding " on either side of : and ,.
DECLARE #json nvarchar(max) = N'{
"start_date": "2021-12-01 00:00:00+00:00",
"end_date": "2021-12-31 23:59:59+00:00",
"resource_type": "all",
"records": [
{
"directconnect_connections": [
"account_id:227148359287,resource_type:directconnect_connections,resource_name:''dxcon-fh40evn5''",
"account_id:401311080156,resource_type:directconnect_connections,resource_name:''dxcon-ffxgf6kh''",
"account_id:401311080156,resource_type:directconnect_connections,resource_name:''dxcon-fg5j5v6o''",
"account_id:227148359287,resource_type:directconnect_connections,resource_name:''dxcon-fgvfo1ej''"
]
},
{
"virtual_interfaces": [
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:''dxvif-fgvj25vt''",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:''dxvif-fgbw5gs0''",
"account_id:401311080156,resource_type:virtual_interfaces,resource_name:''dxvif-ffnosohr''",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:''dxvif-fg18bdhl''",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:''dxvif-ffmf6h64''",
"account_id:390251991779,resource_type:virtual_interfaces,resource_name:''dxvif-fgkxjhcj''",
"account_id:227148359287,resource_type:virtual_interfaces,resource_name:''dxvif-ffp6kl3f''"
]
}
]
}';
SELECT
j4.account_id,
j4.resource_type,
TRIM('''' FROM j4.resource_name) resource_name
FROM OPENJSON(#json, '$.records') j1
CROSS APPLY OPENJSON(j1.value) j2
CROSS APPLY OPENJSON(j2.value) j3
CROSS APPLY OPENJSON('{"' + REPLACE(REPLACE(j3.value, ':', '":"'), ',', '","') + '"}')
WITH (
account_id bigint,
resource_type varchar(20),
resource_name varchar(100)
) j4;
db<>fiddle
The first three calls to OPENJSON have no schema, so the resultset is three columns: key value and type. In the case of arrays (j1 and j3), key is the index into the array. In the case of single objects (j2), key is each property name.

Update/Add attribute to json array in T-SQL

All of the examples examples I see dealing with json arrays have the array nested under a top level object. I have a json array in a column:
[{"key": "value1"}, {"key": "value2"}]
I would like to run a sql script to add/update a key for each element in the array, resulting in:
[{"key": "value1", "otherKey": "otherValue"}, {"key": "value", "otherKey": "otherValue"}]
Yes, in my case I want the same value set for each array member. I've tried:
declare #info nvarchar(max)
SET #info = '[{"key": "value1"}, {"key": "value2"}]'
print JSON_MODIFY(#info, '[0].otherKey', '""')
and fails with "JSON path is not properly formatted. Unexpected character '[' is found at position 0."
This is in MSSQL 2017.
The approach, that can be used, depends on JSON structure (I assume, that the count of the items in the JSON array is not fixed):
If the input JSON array has items (JSON objects) with fixed key/keys, you may use a combination of OPENJSON() with explicit schema (to parse this JSON as table) and FOR JSON (to modify and return the rows as JSON)
If the input JSON has items with different structure, you may use a combination of OPENJSON() with default schema, JSON_MODIFY() and STRING_AGG().
JSON:
declare #info nvarchar(max)
SET #info = '[{"key": "value1"}, {"key": "value2"}]'
Statement for fixed structure:
SELECT #info = (
SELECT [key], 'OtherValue' AS OtherKey
FROM OPENJSON(#info) WITH ([key] varchar(100) '$.key')
FOR JSON PATH
)
Statement for variable structure:
SELECT #info = CONCAT('[', STRING_AGG(JSON_MODIFY([value], '$.OtherKey', 'OtherValue'), ','), ']')
FROM OPENJSON(#info)
SELECT #info
Result:
[{"key":"value1","OtherKey":"OtherValue"},{"key":"value2","OtherKey":"OtherValue"}]
Note, that the reason for the error is that the value of path parameter ([0].otherKey) is wrong. The correct path expression is $[0].otherKey for the first item in the JSON array.
Have you tried adding $ before the indexer sign? Like:
declare #info nvarchar(max)
SET #info = '[{"key": "value1"}, {"key": "value2"}]'
SELECT JSON_MODIFY(#info, '$[0].otherKey', '""')
This gives the following output:
[{"key": "value1","otherKey":"\"\""}, {"key": "value2"}]

How to update JSON array with PostgreSQL

I have the following inconvenience, I want to update a key of an JSON array using only PostgreSQL. I have the following json:
[
{
"ch":"1",
"id":"12",
"area":"0",
"level":"Superficial",
"width":"",
"length":"",
"othern":"5",
"percent":"100",
"location":" 2nd finger base"
},
{
"ch":"1",
"id":"13",
"area":"0",
"level":"Skin",
"width":"",
"length":"",
"othern":"1",
"percent":"100",
"location":" Abdomen "
}
]
I need to update the "othern" to another number if the "othern" = X
(X is any number that I pass to the query. Example, update othern if othern = 5).
This JSON can be much bigger, so I need something that can iterate in the JSON array and find all the "othern" that match X number and replace with the new one. Thank you!
I have tried with these functions json of Postgresql, but I do not give with the correct result:
SELECT * FROM jsonb_to_recordset('[{"ch":"1", "id":"12", "area":"0", "level":"Superficial", "width":"", "length":"", "othern":"5", "percent":"100", "location":" 2nd finger base"}, {"ch":"1", "id":"13", "area":"0", "level":"Skin", "width":"", "length":"", "othern":"1", "percent":"100", "location":" Abdomen "}]'::jsonb)
AS t (othern text);
I found this function in SQL that is similar to what I need but honestly SQL is not my strength:
CREATE OR REPLACE FUNCTION "json_array_update_index"(
"json" json,
"index_to_update" INTEGER,
"value_to_update" anyelement
)
RETURNS json
LANGUAGE sql
IMMUTABLE
STRICT
AS $function$
SELECT concat('[', string_agg("element"::text, ','), ']')::json
FROM (SELECT CASE row_number() OVER () - 1
WHEN "index_to_update" THEN to_json("value_to_update")
ELSE "element"
END "element"
FROM json_array_elements("json") AS "element") AS "elements"
$function$;
UPDATE plan_base
SET atts = json_array_update_index([{"ch":"1", "id":"12", "area":"0", "level":"Superficial", "width":"", "length":"", "othern":"5", "percent":"100", "location":" 2nd finger base"}, {"ch":"1", "id":"13", "area":"0", "level":"Skin", "width":"", "length":"", "othern":"1", "percent":"100", "location":" Abdomen "}], '{"othern"}', '{"othern":"71"}'::json)
WHERE id = 2;
The function you provided changes a JSON input, gives out the changed JSON and updates a table parallel.
For a simple update, you don't need a function:
demo:db<>fiddle
UPDATE mytable
SET myjson = s.json_array
FROM (
SELECT
jsonb_agg(
CASE WHEN elems ->> 'othern' = '5' THEN
jsonb_set(elems, '{othern}', '"7"')
ELSE elems END
) as json_array
FROM
mytable,
jsonb_array_elements(myjson) elems
) s
jsonb_array_elements() expands the array into one row per element
jsonb_set() changes the value of each othern field. The relevant JSON objects can be found with a CASE clause
jsonb_agg() reaggregates the elements into an array again.
This array can be used to update your column.
If you really need a function which gets the parameters and returns the changed JSON, then this could be a solution. Of course, this doesn't execute an update. I am not quite sure if you want to achieve this:
demo:db<>fiddle
CREATE OR REPLACE FUNCTION json_array_update_index(_myjson jsonb, _val_to_change int, _dest_val int)
RETURNS jsonb
AS $$
DECLARE
_json_output jsonb;
BEGIN
SELECT
jsonb_agg(
CASE WHEN elems ->> 'othern' = _val_to_change::text THEN
jsonb_set(elems, '{othern}', _dest_val::text::jsonb)
ELSE elems END
) as json_array
FROM
jsonb_array_elements(_myjson) elems
INTO _json_output;
RETURN _json_output;
END;
$$ LANGUAGE 'plpgsql';
If you want to combine both as you did in your question, of course, you can do this:
demo:db<>fiddle
CREATE OR REPLACE FUNCTION json_array_update_index(_myjson jsonb, _val_to_change int, _dest_val int)
RETURNS jsonb
AS $$
DECLARE
_json_output jsonb;
BEGIN
UPDATE mytable
SET myjson = s.json_array
FROM (
SELECT
jsonb_agg(
CASE WHEN elems ->> 'othern' = '5' THEN
jsonb_set(elems, '{othern}', '"7"')
ELSE elems END
) as json_array
FROM
mytable,
jsonb_array_elements(myjson) elems
) s
RETURNING myjson INTO _json_output;
RETURN _json_output;
END;
$$ LANGUAGE 'plpgsql';

SQL Server - OPENJSON output with an explicit schema

I have the below array of an HTTP request, in JSON format:
[
{
"Code":"856956645",
"Type":"Colet",
"MeasuredWeight":0.0,
"VolumetricWeight":0.0,
"ConfirmationName":null,
"Observation":" 100 DE SFATURI OASELE",
"ResponseCode":null,
"Event":
[
{
"Date":"2018-11-16T16:22:29.397",
"EventId":73,
"Description":"Ridicare din comanda client",
"LocalityName":"BUCURESTI"
},
{
"Date":"2018-11-17T08:55:06.14",
"EventId":5,
"Description":"Spre destinatar ",
"LocalityName":"BUCURESTI"
}
]
}
]
How could I extract the value of Description element, within the second set of values. I tried with OPENJSON but I couldn't do it:
SELECT *
FROM OPENJSON(#json)
WITH (
Description nvarchar(100) '$.Event.Description'
);
Try nesting instead. Nor sure why your attempt didn't work; I've not had much use for OPENJSON as yet apart from when playing around, however, this works:
SELECT J.Code, J.[Type], E.[Description]
FROM OPENJSON(#json)
WITH (Code bigint '$.Code',
[Type] varchar(10) '$.Type',
[Event] nvarchar(MAX) AS JSON) J
CROSS APPLY OPENJSON([Event])
WITH ([Description] varchar(100) '$.Description',
EventID int '$.EventId') E
WHERE E.EventID = 5;
Edit: Worked out why your attempt wasn't working. The JSON you have has a new JSON object in the Event node, they're not simply properties, like in the documentation's second example here. The entities are wrapped in further brackets ([]), not just braces ({}), and hence why you have to parse the next layer again as a separate JSON object.

Inserting data into user defined type in Cassandra

I am having trouble inserting JSON and updating data to my Cassandra table.
I have a table with the following structure:
CREATE TYPE keyspace.connected_hw (
connected_element text,
support_list frozen<list<text>>
);
CREATE TABLE keyspace.sample_table (
id text,
name text,
connected_items frozen<keyspace.connected_hw>
)
My insert script looks like this :
INSERT INTO keyspace.sample_table JSON '
{
"id": "12345",
"name": "object 1",
"connected_items": [
{
"connected_element": "56789",
"support_list": ""
}
]
}
'
After I run this, I get the following error :
Error decoding JSON value for connected_items: Expected a map, but got a ArrayList :
I tried to insert the other fields without the 'connected_items" list first, then updating it afterwards :
INSERT INTO keyspace.sample_table JSON '
{
"id": "12345",
"name": "object 1",
}
'
This works until I try to update 'connected_items' with
UPDATE keyspace.sample_table
SET connected_items = [{connected_element:'56789',support_list:['type1','type2']}]
where id='12345'
After running this I get the following error :
InvalidQueryException: Invalid list literal for connected_items of type frozen<connected_hw>
You are declaring connected_items as type connected_hw but you are inserting value as list of 'connected_hw ' and also support_list is a list of text not just text
You Either change your insert/update format like below :
INSERT INTO sample_table JSON '{"id": "12345", "name": "object 1", "connected_items": { "connected_element": "56789", "support_list": ["a","b"] } }' ;
And
UPDATE sample_table SET connected_items = {connected_element:'56789',support_list:['type1','type2']} where id='12345' ;
Or Change the type of connected_items to list of connected_hw
ALTER TABLE sample_table DROP connected_items ;
ALTER TABLE sample_table ADD connected_items list<frozen <connected_hw>>;

Resources