How to transform a JSON array nested inside an object inside another array in Postgres? - arrays

I'm using Postgres 9.6 and have a JSON field called credits with the following structure; A list of credits, each with a position and multiple people that can be in that position.
[
{
"position": "Set Designers",
people: [
"Joe Blow",
"Tom Thumb"
]
}
]
I need to transform the nested people array, which are currently just strings representing their names, into objects that have a name and image_url field, like this
[
{
"position": "Set Designers",
people: [
{ "name": "Joe Blow", "image_url": "" },
{ "name": "Tom Thumb", "image_url": "" }
]
}
]
So far I've only been able to find decent examples of doing this on either the parent JSON array or on an array field nested inside a single JSON object.
So far this is all I've been able to manage and even it is mangling the result.
UPDATE campaigns
SET credits = (
SELECT jsonb_build_array(el)
FROM jsonb_array_elements(credits::jsonb) AS el
)::jsonb
;

Create an auxiliary function to simplify the rather complex operation:
create or replace function transform_my_array(arr jsonb)
returns jsonb language sql as $$
select case when coalesce(arr, '[]') = '[]' then '[]'
else jsonb_agg(jsonb_build_object('name', value, 'image_url', '')) end
from jsonb_array_elements(arr)
$$;
With the function the update is not so horrible:
update campaigns
set credits = (
select jsonb_agg(jsonb_set(el, '{people}', transform_my_array(el->'people')))
from jsonb_array_elements(credits::jsonb) as el
)::jsonb
;
Working example in rextester.

Related

Full Text Search in OrientDB JSON Data

I have following data in OrientDB 3.0.27 where some of the values are in JSON Array and some are string
{
"#type": "d",
"#rid": "#57:0",
"#version": 2,
"#class": "abc_class",
"user_name": [
"7/1 LIBOR Product"
],
"user_Accountability": [],
"user_Rollout_32_date": [],
"user_Brands": [
"AppNet"
],
"user_lastModificationTime": [
"2019-11-27 06:40:35"
],
"user_columnPercentage": [
"0.00"
],
"user_systemId": [
"06114a87-a099-0c30c60b49c4"
],
"user_lastModificationUser": [
"system"
],
"system_type": "Product",
"user_createDate": [
"2017-10-27 09:58:42"
],
"system_modelId": "bian_model",
"user_parent": [
"a12a41bd-af6f-0ca028af480d"
],
"user_Strategic_32_value": [],
"system_oeId": "06114a87-a099-0c30c60b49c4",
"user_description": [],
"#fieldTypes": "user_name=e,user_Accountability=e,user_Rollout_32_date=e,user_Brands=e,user_lastModificationTime=e,user_columnPercentage=e,user_systemId=e,user_lastModificationUser=e,user_createDate=e,user_parent=e,user_Strategic_32_value=e,user_description=e"
}
I have tried following queries:
select * from `abc_class ` where any() = ["AppNet"] limit 2;
select * from `abc_class ` where any() like '%a099%' limit 2;
Both of the above queries work since they are respecting the datatype of the field.
I want to run a contains query which will search in ANY field with ANY data type (like String, number, JSON Array, etc) more of like a - full text search.
select * from `abc_class ` where any() like '%AppNet%' limit 2;
The above query doesn't work since the real value is inside JSON Array. Tried almost all the things from filtering section documentation
How can I achieve full-text search like functionality with the existing data?
EDIT # 1
After doing more research now I'm able to atleast convert the array value into string and then run like operator on it, like below;
select * from `abc_class` where user_name.asString() like '%LIBOR%'
However, using any().asString() doesn't result any result
select * from `abc_class` where any().asString() like '%LIBOR%'
If the above query can be enhanced somehow to query any column as string, then the problem can be resolved.
If all the column values needs to be searched then we can create a JSON object of the full row data and convert it into String.
Then query the string with like keyword, as follows:
select * from `abc_class` where #this.toJSON().asString() like '%LIBOR%'
If we will be converting to #this.asString() directly then we'll be getting the count of array elements instead of the real data inside the array elements like below:
abc_class#57:4{system_modelId:model,system_oeId:14f4b593-a57d-4d37ad070a10,system_type:Product,user_lastModificationUser:[1],user_name:[1],user_description:[0],user_Accountability:[0],user_lastModificationTime:[1],user_Rollout_32_date:[0],user_Strategic_32_value:[0],user_createDate:[1],user_Brands:[0],user_parent:[1],user_systemId:[1],user_columnCompletenessPercentage:[1]} v2
Therefore, we need to first convert into JSON and then into String to query the full record using #this.toJSON().asString()
References:
https://orientdb.com/docs/last/sql/SQL-Methods.html
https://orientdb.com/docs/last/sql/SQL-Where.html
https://orientdb.com/docs/last/sql/SQL-Syntax.html

Using $rename in MongoDB for an item inside an array of objects

Consider the following MongoDB collection of a few thousand Objects:
{
_id: ObjectId("xxx")
FM_ID: "123"
Meter_Readings: Array
0: Object
Date: 2011-10-07
Begin_Read: true
Reading: 652
1: Object
Date: 2018-10-01
Begin_Reading: true
Reading: 851
}
The wrong key was entered for 2018 into the array and needs to be renamed to "Begin_Read". I have a list using another aggregate of all the objects that have the incorrect key. The objects within the array don't have an _id value, so are hard to select. I was thinking I could iterate through the collection and find the array index of the errored Readings and using the _id of the object to perform the $rename on the key.
I am trying to get the index of the array, but cannot seem to select it correctly. The following aggregate is what I have:
[
{
'$match': {
'_id': ObjectId('xxx')
}
}, {
'$project': {
'index': {
'$indexOfArray': [
'$Meter_Readings', {
'$eq': [
'$Meter_Readings.Begin_Reading', True
]
}
]
}
}
}
]
Its result is always -1 which I think means my expression must be wrong as the expected result would be 1.
I'm using Python for this script (can use javascript as well), if there is a better way to do this (maybe a filter?), I'm open to alternatives, just what I've come up with.
I fixed this myself. I was close with the aggregate but needed to look at a different field for some reason that one did not work:
{
'$project': {
'index': {
'$indexOfArray': [
'$Meter_Readings.Water_Year', 2018
]
}
}
}
What I did learn was the to find an object within an array you can just reference it in the array identifier in the $indexOfArray method. I hope that might help someone else.

Query AQL syntax required for following JSON structure

I have a JSON structure like this
{
"Items": {
"Apple": {
"Type": 2,
"keyVal": "6044e3a3-c064-4171-927c-2440e2f65660"
},
"Lemons": {
"Type": 1,
"keyVal": "79c45f4d-4f62-4c8e-8de1-79e04fc9b95d"
}
},
"Species": 0,
"Name": "Test Fruit",
"Description": "Test Des",
"Creator": "xyz",
"SKey" : "123"
}
This is present in a collection named Fruits.
Query:
I am trying to write and AQL query to find the SKey where KeyVal value in Items is some value.
I am traditionally used to the SQL syntax but this is the first time I am venturing into the AQL(Arango DB).
Any help with the Syntax is appretiated
The basics of AQL are explained here really well: https://docs.arangodb.com/3.3/AQL/index.html
FOR item IN Items FILTER item.keyVal == "someValue" RETURN item
Would be your minimal SQL SELECT ... WHERE statement.
BTW: There is a comparative introduction to be found here:
https://arangodb.com/why-arangodb/sql-aql-comparison/
A good way to learn AQL is to try small pieces of code an return the result for inspection, to gradually create more complex queries.
For example, let's return one of the nested keyVal values:
FOR doc IN Fruits
RETURN doc.Items.Apple.keyVal
// "6044e3a3-c064-4171-927c-2440e2f65660"
To filter by Apple keyVal and return SKey, you can do:
FOR doc IN Fruits
FILTER doc.Items.Apple.keyVal == "6044e3a3-c064-4171-927c-2440e2f65660"
RETURN doc.SKey
// "123"
You can return both keyVal values too:
FOR doc IN Fruits
RETURN [
doc.Items.Apple.keyVal,
doc.Items.Lemons.keyVal
]
// [
// "6044e3a3-c064-4171-927c-2440e2f65660",
// "79c45f4d-4f62-4c8e-8de1-79e04fc9b95d"
// ]
To return SKey if either is equal to some value, try this:
FOR doc IN Fruits
FILTER "79c45f4d-4f62-4c8e-8de1-79e04fc9b95d" IN [
doc.Items.Apple.keyVal,
doc.Items.Lemons.keyVal
]
RETURN doc.SKey
Note: IN is used here as array operator, like is {value} contained in {array}.
To return all keyVal values hardcoding the attribute paths, you can make use of the ATTRIBUTES() AQL function:
FOR doc IN Fruits
FOR attr IN ATTRIBUTES(doc.Items)
RETURN doc.Items[attr].keyVal
To return SKey if any of the nested keyVal values match, we can do:
FOR doc IN Fruits
LET keyVals = (FOR attr IN ATTRIBUTES(doc.Items)
RETURN doc.Items[attr].keyVal
)
FILTER "6044e3a3-c064-4171-927c-2440e2f65660" IN keyVals
RETURN doc.SKey
Note: this uses a subquery to capture the intermediate result.
To test if all specified values are contained, you could do:
LET ids = [
"79c45f4d-4f62-4c8e-8de1-79e04fc9b95d",
"6044e3a3-c064-4171-927c-2440e2f65660"
]
FOR doc IN Fruits
LET keyVals = (FOR attr IN ATTRIBUTES(doc.Items)
RETURN doc.Items[attr].keyVal
)
FILTER ids ALL IN keyVals
RETURN doc.SKey
ALL IN is an array comparison operator.
Note that it would require a change to your data model if you wanted to use indexes without hardcoding the attributes paths, and also different queries.

Querying an array within an array with Postgres JSONB query

I have some JSON in a field in my Postgres 9.4 db and I want to find rows where the given name is a certain value, where the field is named model and the JSON structure is as follows:
{
"resourceType": "Person",
"id": "8a7b72b1-49ec-43e5-bd21-bc62674d9875",
"name": [
{
"family": [
"NEWMAN"
],
"given": [
"JOHN"
]
}
]
}
So I tried this: SELECT * FROM current WHERE model->'name' #> '{"given":["JOHN"]}'; (as well as various other guesses) but that does not match the above data. How should I do this?
Use the function jsonb_array_elements():
select t.*
from current t,
jsonb_array_elements(model->'name') names
where names->'given' ? 'JOHN'

jq: select only an array which contains element A but not element B

My data is a series of JSON arrays. Each array has one or more elements with name and id keys:
[
{
"name": "first_source",
"id": "abcdef"
},
{
"name": "second_source",
"id": "ghijkl"
},
{
"name": "third_source",
"id": "opqrst"
}
]
How, using jq, do I select only the arrays which contain an element with "first source" as the name value, but which don't contain "second_source" as the name value of any element?
This only returns an element for further processing:
jq '.[] | select (.name == "first_source")
But I clearly need to return the entire array for my scenario to work.
You can use this filter:
select(
(map(.name == "first_source") | any) and
(map(.name != "second_source") | all)
)
You need to test all the elements of an array for an existence of the names. You can do that by mapping each object to your condition and use the any or all filter appropriately.
Here, you want to see if any item is named "first_source" and all items are not named "second_source".

Resources