I'm getting data in an specific way from an API and I have to convert it to a cleaner version of it.
What I get from the API is a JSON like this (you can see that there is some information duplicated as for the first fields but the investor is different).
{
"clubhouse": [
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1234",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
}
]
},
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "4321",
"gender": "02"
},
"inamount": "1700000",
"ratio": "12"
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1333",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
}
]
},
{
"id": "03",
"statusId": "ok",
"stateid": "5",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "",
"gender": ""
},
"inamount": "",
"ratio": ""
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1334",
"gender": "02"
},
"inamount": "1900000",
"ratio": "12"
}
]
}
]
}
I need to merge the investors and eliminate the duplicated information, the the expected result will be
{
"clubhouse": [
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1234",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
},
{
"investor": {
"id": "4321",
"gender": "02"
},
"inamount": "1700000",
"ratio": "12"
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1333",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
},
{
"investor": {
"id": "1334",
"gender": "02"
},
"inamount": "1900000",
"ratio": "12"
}
]
},
{
"id": "03",
"statusId": "ok",
"stateid": "5",
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1555",
"gender": "01"
},
"inamount": "2000000",
"ratio": "15"
}
]
}
]
}
I'd try a couple of JOLTS and I got to merge the fields but not eliminate the duplicates.
You can start with grouping by id values such as
[
{
// group by "id" values to create separate objects
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "#(1,id).&",
"investors": {
"*": {
"*": {
"#": "#(4,id).&3[&4].&" // &3 -> going 3 levels up to grab literal "investors", [&4] -> going 4 levels up the tree in order to reach the indexes of "clubhouse" array, & -> replicate the leaf node values for the current key-value pair
}
}
}
}
}
}
},
{
// get rid of "null" values
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
},
{
// pick only the first components from the repeated values populated within the arrays
"operation": "cardinality",
"spec": {
"*": {
"*": "ONE",
"investors": "MANY"
}
}
},
{
// get rid of object labels
"operation": "shift",
"spec": {
"*": ""
}
}
]
Related
I previously have this issue of merging data into another one to avoid duplicates and make a cleaner version of the JSON. I got a solution in here that worked like a charm for a while but after I got more information arrayed inside the JSON things got a little bit tricky.
I have this array:
{
"clubhouse": [
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "1"
},
{
"nationalityid": "2"
},
{
"nationalityid": "3"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1234",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
}
]
},
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "1"
},
{
"nationalityid": "2"
},
{
"nationalityid": "3"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "4321",
"gender": "02"
},
"inamount": "1700000",
"ratio": "12"
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1333",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
}
]
},
{
"id": "03",
"statusId": "ok",
"stateid": "5",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "",
"gender": ""
},
"inamount": "",
"ratio": ""
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1334",
"gender": "02"
},
"inamount": "1900000",
"ratio": "12"
}
]
}
]
}
I was using this JOLT but it doesnt work with the nationalities,since it loses the array they are in.
[
{
// group by "id" values to create separate objects
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "#(1,id).&",
"investors": {
"*": {
"*": {
"#": "#(4,id).&3[&4].&" // &3 -> going 3 levels up to grab literal "investors", [&4] -> going 4 levels up the tree in order to reach the indexes of "clubhouse" array, & -> replicate the leaf node values for the current key-value pair
}
}
}
}
}
}
},
{
// get rid of "null" values
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
},
{
// pick only the first components from the repeated values populated within the arrays
"operation": "cardinality",
"spec": {
"*": {
"*": "ONE",
"investors": "MANY"
}
}
},
{
// get rid of object labels
"operation": "shift",
"spec": {
"*": ""
}
}
]
What I need to get is something like this:
{
"clubhouse": [
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "1"
},
{
"nationalityid": "2"
},
{
"nationalityid": "3"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1234",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
},
{
"investor": {
"id": "4321",
"gender": "02"
},
"inamount": "1700000",
"ratio": "12"
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1333",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
},
{
"investor": {
"id": "1334",
"gender": "02"
},
"inamount": "1900000",
"ratio": "12"
}
]
},
{
"id": "03",
"statusId": "ok",
"stateid": "5",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "",
"gender": ""
},
"inamount": "",
"ratio": ""
}
]
}
]
}
You can rearrange the first shift transformation by adding a new object tagged "nationalities" which has one level reduced identifiers compared to the already existing object tagged "investors", and the existing cardinality transformation would already pick only the first array among repeated identical "nationalities" arrays if the remaining specs are kept as they are, such as the below one
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "#(1,id).&",
"nationalities": {
"*": {
"#": "#(3,id).&2[&3][]"
}
},
"investors": {
"*": {
"*": {
"#": "#(4,id).&3[&4].&"
}
}
}
}
}
}
},
...
]
I've a little problem, fetch query over react to Graphql work.
But when I ask a complex query, I receive this answer :
How I can read all variable but not the variable in the "operation":
Name
ID
How I can read this 2 Variables ?
[
{
"__typename": "Journal",
"id": "1",
"name": "Journal1",
"state": "ACTIVE",
"operation": {
"__typename": "Operation",
"id": "1",
"name": "Oper1OnlyName"
}
},
{
"__typename": "Journal",
"id": "2",
"name": "Default",
"state": "ACTIVE",
"operation": {
"__typename": "Operation",
"id": "15",
"name": "Oper15ID"
}
},
{
"__typename": "Journal",
"id": "3",
"name": "Nachrichten",
"state": "ACTIVE",
"operation": {
"__typename": "Operation",
"id": "15",
"name": "Oper15ID"
}
},
{
"__typename": "Journal",
"id": "4",
"name": "WEMA",
"state": "ACTIVE",
"operation": null
},
I want to remove a name/value pair from inside a json array and print it outside. I started by trying this and then expanding the whole request to be a json array. The solution mentioned above does not seem to be working.
Input :
[
{
"createdBy": "Admin",
"createdDate": "2022-09-08",
"modifiedBy": "Admin",
"attrs": [
{
"name": "Type",
"value": "Postpaid"
},
{
"name": "subscriber",
"value": "Paid"
},
{
"name": "Details",
"value": {
"createdDate": "today",
"description": "offer",
"id": null
}
}
],
"relatedInfo": [
{
"type": "Number",
"name": "000000"
},
{
"type": "Type",
"name": "Post"
}
]
},
{
"createdBy": "Admin",
"createdDate": "2022-09-08",
"modifiedBy": "Admin",
"attrs": [
{
"name": "Type",
"value": "Postpaid"
},
{
"name": "subscriber",
"value": "Paid"
},
{
"name": "Details",
"value": {
"createdDate": "today",
"description": "offer",
"id": null
}
}
],
"relatedInfo": [
{
"type": "Number",
"name": "000000"
},
{
"type": "Type",
"name": "Post"
}
]
}
]
Desired Output :
[
{
"createdBy": "Admin",
"createdDate": "2022-09-08",
"modifiedBy": "Admin",
"attrs": [
{
"name": "Type",
"value": "Postpaid"
},
{
"name": "subscriber",
"value": "Paid"
}
],
"Details": {
"createdDate": "today",
"description": "offer",
"id": null
},
"relatedInfo": [
{
"type": "Number",
"name": "000000"
},
{
"type": "Type",
"name": "Post"
}
]
},
{
"createdBy": "Admin",
"createdDate": "2022-09-08",
"modifiedBy": "Admin",
"attrs": [
{
"name": "Type",
"value": "Postpaid"
},
{
"name": "subscriber",
"value": "Paid"
}
],
"Details": {
"createdDate": "today",
"description": "offer",
"id": null
},
"relatedInfo": [
{
"type": "Number",
"name": "000000"
},
{
"type": "Type",
"name": "Post"
}
]
}
]
Current Jolt spec:
[
{
"operation": "shift",
"spec": {
"*": "[&]",
"attrs": {
"*": {
"name": {
"*": { "#2": "&4" },
"Details": {
"#(2,value)": "&1"
}
}
}
}
}
}
]
I can't seem to figure out how the jolt spec would change in case of the array
So far so good, just need to combine the attributes at a common node. To do this, I've used the identifiers [&1] and [&5] in order to reach the level of the outermost index within the tree such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": "[&1].&",
"attrs": {
"*": {
"name": {
"*": {
"#2": "[&5].&4"
},
"Details": {
"#(2,value)": "[&5].&1"
}
}
}
}
}
}
}
]
{
"$id": "1",
"listTasks": {
"$id": "2",
"$values": [
{
"$id": "3",
"id": 1,
"name": "Task1G1",
"priorityID": 1,
"dueDate": "2022-07-14T04:12:14.114",
"createdAt": "2022-07-14T11:13:06.1808811",
"enumerable": {
"$id": "4",
"$values": [
2
]
}
},
{
"$id": "5",
"id": 2,
"name": "string",
"priorityID": 1,
"dueDate": "2022-07-29T08:55:06.156",
"createdAt": "2022-07-14T15:55:57.0330615",
"enumerable": {
"$id": "6",
"$values": [
3
]
}
}
]
}
}
I want to access the secondary $value in this json (enumerable object), I use this code below:
useEffect(() => {
axios
.get(API_URL + Task/GetTaskInGroup?GroupID=${id}, {})
.then((response) => {
**setTaskList(response.data.listTasks.$values.enumerable.$value);**
})
.catch((error) => {
console.log(error.response.data);
});
}, [id]);
But it didn't work, I tried setTaskList(response.data.listTasks.$values) then it work (I can get all data but I can't access to enumerable object)
Try with computed property names (myObject[..]) like below:
response.data.listTasks["$values"][0].enumerable["$values"]
const data = { "$id": "1", "listTasks": { "$id": "2", "$values": [ { "$id": "3", "id": 1, "name": "Task1G1", "priorityID": 1, "dueDate": "2022-07-14T04:12:14.114", "createdAt": "2022-07-14T11:13:06.1808811", "enumerable": { "$id": "4", "$values": [ 2 ] } }, { "$id": "5", "id": 2, "name": "string", "priorityID": 1, "dueDate": "2022-07-29T08:55:06.156", "createdAt": "2022-07-14T15:55:57.0330615", "enumerable": { "$id": "6", "$values": [ 3 ] } } ] } }
console.log(data.listTasks["$values"][0].enumerable["$values"]);
I'm new on lucence, solr queries, I have doubts about how to make a query to nested documents.
I have nested document indexed, as below
[
{
"id": "1",
"title": "Solr1",
"_childDocuments_": [
{
"id": "2",
"title": "Solr2",
"_childDocuments_": [
{
"id": "3",
"title": "Solr3",
"_childDocuments_": [
{
"id": "4",
"title": "SolrCloud supports it"
}
],
"something_else":"irrelevant"
}
],
"something_else":"irrelevant"
}
],
"something_else":"irrelevant"
},
{
"id": "5",
"title": Solr5",
"_childDocuments_": [
{
"id": "6",
"title": "SolrCloud here as well"
}
]
}
]
How do I search title:SolrCloud, and listed all children's parents? Such as
[
{
"id": "1",
"title": "Solr1",
"_childDocuments_": [
{
"id": "2",
"title": "Solr2",
"_childDocuments_": [
{
"id": "3",
"title": "Solr3",
"_childDocuments_": [
{
"id": "4",
"title": "SolrCloud supports it"
}
]
}
]
}
]
},
{
"id": "5",
"title": Solr5",
"_childDocuments_": [
{
"id": "6",
"title": "SolrCloud here as well"
}
]
}
]
which listed all parents of document 4(Sorl1, Solr2, Solr3) and document 6(Solr5).
And the depth of documents is not constants.
My current solution is that massaging data, add trace into original data, so I will know document come from. such as
[
{
"id": "1",
"title": "Solr1",
"_childDocuments_": [
{
"id": "2",
"title": "Solr2",
**"parent_id":"1",**
**"trace":"Solr1",**
"_childDocuments_": [
{
"id": "3",
"title": "Solr3",
**"parent_id":"2",**
**"trace":"Solr1/Solr2",**
"_childDocuments_": [
{
"id": "4",
"title": "SolrCloud supports it"
**"parent_id":"3",**
**"trace":"Solr1/Solr2/Solr3",**
}
],
"something_else":"irrelevant"
}
],
"something_else":"irrelevant"
}
],
"something_else":"irrelevant"
},
{
"id": "5",
"title": Solr5",
"_childDocuments_": [
{
"id": "6",
**"parent_id":"5",**
**"trace":"Solr5",**
"title": "SolrCloud here as well"
}
]
}
]
So after indexed, I could know who's parent document from result.
Could someone agree on this?
Looking for better solution than this.