I'm looking for a way to query nested arrays in ArangoDB.
The JSON structure I have is:
{
"uid": "bykwwla4prqi",
"category": "party",
"notBefore": "2016-04-19T08:43:35.388+01:00",
"notAfter": "9999-12-31T23:59:59.999+01:00",
"version": 1.0,
"aspects": [
"participant"
],
"description": [
{ "value": "User Homer Simpson, main actor in 'The Simpsons'", "lang": "en"}
],
"properties": [
{
"property": [
"urn:project:domain:attribute:surname"
],
"values": [
"Simpson"
]
},
{
"property": [
"urn:project:domain:attribute:givennames"
],
"values": [
"Homer",
"Jay"
]
}
]
}
I tried to use a query like the following to find all parties having a given name 'Jay':
FOR r IN resource
FILTER "urn:project:domain:attribute:givennames" IN r.properties[*].targets[*]
AND "Jay" IN r.properties[*].values[*]
RETURN r
but unfortunately it does not work - it returns an empty array. If I use a '1' instead of '*' for the properties array it works. But the array of the properties has no fixed structure.
Does anybody have an idea how to solve this?
Thanks a lot!
You can inspect what the filter does using a simple trick: you RETURN the actual filter condition:
db._query(`FOR r IN resource RETURN r.properties[*].property[*]`).toArray()
[
[
[
"urn:project:domain:attribute:surname"
],
[
"urn:project:domain:attribute:givennames"
]
]
]
which makes it pretty clear whats going on. The IN operator can only work on one dimensional arrays. You could work around this by using FLATTEN() to remove the sub layers:
db._query(`FOR r IN resource RETURN FLATTEN(r.properties[*].property[*])`).toArray()
[
[
"urn:project:domain:attribute:surname",
"urn:project:domain:attribute:givennames"
]
]
However, while your documents are valid json (I guess its converted from xml?) you should alter the structure as one would do it in json:
"properties" : {
"urn:project:domain:attribute:surname":[
"Simpson"
],
"urn:project:domain:attribute:givennames": [
"Homer",
"Jay"
]
}
Since the FILTER combination you specify would also find any other Jay (not only those found in givennames) and the usage of FLATTEN() will prohibit using indices in your filter statement. You don't want to use queries that can't use indices on reasonably sized collections for performance reasons.
In Contrast you can use an array index on givennames with the above document layout:
db.resource.ensureIndex({type: "hash",
fields:
["properties.urn:project:domain:attribute:givennames[*]"]
})
Now doublecheck the explain for the query:
db._explain("FOR r IN resource FILTER 'Jay' IN " +
"r.properties.`urn:project:domain:attribute:givennames` RETURN r")
...
6 IndexNode 1 - FOR r IN resource /* hash index scan */
...
Indexes used:
By Type Collection Unique Sparse Selectivity Fields Ranges
6 hash resource false false 100.00 % \
[ `properties.urn:project:domain:attribute:givennames[*]` ] \
("Jay" in r.`properties`.`urn:project:domain:attribute:givennames`)
that its using the index.
Related
I have JSON that dynamically comes to me in the format:
{
"world": 583,
"peace": [
{
"id": "Happy",
"valid": true,
"version": "9"
},
{
"id": "Old",
"valid": false,
"version": "2020"
},
{
"id": "New",
"valid": true,
"version": "2021"
},
{
"id": "Year",
"valid": true,
"version": "5"
}
]
}
I'm brand new to jq, and I've read the tutorial and the manual, and several questions here on Stack Overflow, including:
How to sort a json file by keys and values of those keys in jq
I want to use jq to output all id's that are valid ("valid"=true) and have the output sorted by id.
So in this example, I would like the output to be:
Happy
New
Year
So far, I have:
jq '..|.peace[]|select(.valid)|=sort_by(.id)'
But that issues a Cannot index string with string "id" error.
How can I make this work?
Thank you, and Happy New Year!
Keep the .peace array intact for filtering & sorting then break away from it:
jq --raw-output '.peace | map(select(.valid).id) | sort[]' <f.json>
^ ^ ^ ^
A B C D
Where:
A: Keep array intact
B: map + select = filter & keep id only
C: sort result (which is a list of strings (i.e. ids))
D: "spread" each item out of the array
--raw-output will output as "text" and not as string, e.g. Happy instead of "Happy"
jq play fiddle available here.
Acknowledgement: this answer has been vastly improved by #peak suggestion
I want my JSON schema to accept a list but the values in the list are from a set list and can apparear in any order!
I.e. ["GOV","CRD", "CON"] is acceptable, but so is ["CRD", "GOV", "COM"].
My current thinking is something along these lines:
"sources":{"type": "array",
"uniqueItems": true,
"emum": ["CRD", "GOV", "COM", "CON", "OTH", "UTL", "PRO", "TEL", "POS", "INS", "CCJ", "POP", "VOT", "MVR", "PPS", "DRV", "PMC"]},
But I'm not entirely sure that's going to do what I want. I've read up on items where you can define the values in the list, but it looks like that would set the order and also the number of items in the list. Although both can be worked around using oneOf combined with definitions.
E.g. (shortened for space saving reasons) Please feel free to correct this if I'm wrong:
{
"definitions": {
"source":{"emum": ["CRD", "GOV", "COM", "CON", "OTH", "UTL", "PRO", "TEL", "POS", "INS", "CCJ", "POP", "VOT", "MVR", "PPS", "DRV", "PMC", ""]},
}
"sources":{"type": "array",
"uniqueItems": true,
"items": {
"source": {"$ref": "#/definitions/source"},
"source": {"$ref": "#/definitions/source"},
"source": {"$ref": "#/definitions/source"},
.
.
.
}
}
}
My question is: Is there a nicer way to do this?
You don't have to specify every possible order. When the array is made limited by enum, the items can come in any order. However, you have to specify the type of the enumerated values.
"sources":{
"type": "array",
"uniqueItems": true,
"items": {
"type": "string",
"emum": ["CRD", "GOV", "COM"]
}
I have some JSON in a field in my Postgres 9.4 db and I want to find rows where the given name is a certain value, where the field is named model and the JSON structure is as follows:
{
"resourceType": "Person",
"id": "8a7b72b1-49ec-43e5-bd21-bc62674d9875",
"name": [
{
"family": [
"NEWMAN"
],
"given": [
"JOHN"
]
}
]
}
So I tried this: SELECT * FROM current WHERE model->'name' #> '{"given":["JOHN"]}'; (as well as various other guesses) but that does not match the above data. How should I do this?
Use the function jsonb_array_elements():
select t.*
from current t,
jsonb_array_elements(model->'name') names
where names->'given' ? 'JOHN'
Likely a close question to JQ: Nested JSON transformation but I wasn't able to get my head around it.
Sample JSON:
"value": [
{
"FeatureStatus": [
{
"FeatureName": "Sway1",
"FeatureServiceStatus": "ServiceOperational"
},
{
"FeatureName": "Sway2",
"FeatureServiceStatus": "ServiceDegraded"
}
],
"Id": "SwayEnterprise",
},
{
"FeatureStatus": [
{
"FeatureName": "yammerfeatures",
"FeatureServiceStatus": "ServiceOperational"
}
],
"Id": "yammer"
}
]
What I want to do is create an output with jq which results in the following;
{"Sway":"Sway1":"ServiceOperational"},
{"Sway":"Sway2":"ServiceDegraded"},
{"Yammer":"yammerfeatures":"ServiceOperational"}
My various attempts either end up with thousands of non-unique (i.e Yammer with Sway status), or only one Id with x number of FeatureServiceStatus.
Any pointers would be greatly appreciated. I've gone through the tutorial and the cookbook. I am perhaps 2.5 days into using jq.
Assuming that the enclosing braces have been added to make the input valid JSON, the filter:
.value[]
| [.Id] + (.FeatureStatus[] | [ .FeatureName, .FeatureServiceStatus ])
produces:
["SwayEnterprise","Sway1","ServiceOperational"]
["SwayEnterprise","Sway2","ServiceDegraded"]
["yammer","yammerfeatures","ServiceOperational"]
You can then easily reformat this as desired.
I have a JSON result that I am trying to convert into an array using Newtonsoft.Json.
My JSON result i get from the website is along the lines of (formated for readability):
{
"headers":
[
"Shift Date",
"Shift Number"
],
"values":
[
["2016-06-19T00:00:00",0],
["2016-06-19T00:00:00",2],
["2016-06-19T00:00:00",1]
]
}
Code examples I have found say that i should be able to use
Dim arr As JArray = JArray.Parse(response.Content)
This results in an error though with:
An unhandled exception of type 'Newtonsoft.Json.JsonReaderException' occurred in Newtonsoft.Json.dll
Additional information: Error reading JArray from JsonReader. Current JsonReader item is not an array: StartObject. Path '', line 1, position 1.
Any guidance on what could be causing the issue? I suspect it is something with the "headers" but unable to find online any suggestions on how to resolve
You are using JArray but your input is actually an object - i.e. the data is
{..stuff..}
rather than
[ {..stuff..} ].
If you restructure your input to be:
[{
"headers": [
"Shift Date",
"Shift Number"
],
"values": [
["2016-06-19T00:00:00", 0],
["2016-06-19T00:00:00", 2],
["2016-06-19T00:00:00", 1]
]
}]
You can then use the JArray.Parse(strJson) method.
In the current structure, you should use JObject.Parse.