Can Jolt flatten an array of objects which contain array? For example, is there a way to write Jolt transformation specs for the following input and output?
Jolt https://github.com/bazaarvoice/jolt
Input:
[
{"Id": "111", ["mobile": "1111", "home": "1112"]},
{"Id": "222", ["mobile": "2221"]}
]
Output:
[
{"Id": "111", "mobile": "1111"},
{"Id": "111", "home": "1112"},
{"Id": "222", "mobile": "2221"}
]
I didn't find a way to represent the output array indexes.
assuming input is fixed as
[
{
"Id": "111",
"Val": [
{
"mobile": "1111"
},
{
"home": "1112"
}
]
},
{
"Id": "222",
"Val": [
{
"mobile": "2221"
}
]
}
]
then applying the following transformation
[
{
"operation": "shift",
"spec": {
"*": {
"Id": null,
"Val": {
"*": {
"#": "[&3].[&0]",
"#(2,Id)": "[&3].[&0].id"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
reach the expected output:
[ {
"mobile" : "1111",
"id" : "111"
}, {
"home" : "1112",
"id" : "111"
}, {
"mobile" : "2221",
"id" : "222"
} ]
Related
I am trying to transform the JSON as below expected output. Stuck with the spec below. Can someone help in this?
There is an inner array with name "content" in the "results" array which I want to make it as a part of main array.
Input JSON
{
"total": 100,
"start": 1,
"page-length": 10,
"results": [
{
"index": 1,
"uri": "uri1/uri2",
"extracted": {
"kind": "object",
"content": [
{
"code": "A1",
"region": "APAC"
}
]
}
},
{
"index": 2,
"uri": "uri1/uri2",
"extracted": {
"kind": "object",
"content": [
{
"code": "B1",
"region": "AMER"
}
]
}
},
{
"index": 3,
"uri": "uri1/uri2",
"extracted": {
"kind": "object",
"content": [
{
"code": "C1",
"region": "APAC"
}
]
}
}
]
}
Expected json output
[
{
"code": "A1",
"region": "APAC"
},
{
"code": "B1",
"region": "AMER"
},
{
"code": "C1",
"region": "APAC"
}
]
Spec Tried
[
{
"operation": "shift",
"spec": {
"results": {
"*": {
"extracted": { "content": { "#": "&" } }
}
}
}
},
{
"operation": "shift",
"spec": {
"content": {
"#": "&"
}
}
}
]
Find the output below I am getting on Jolt tool
You can use # wildcard nested within square brackets in order to reach the level of the indexes of the "results" array such that
[
{
"operation": "shift",
"spec": {
"results": {
"*": {//the indexes of the "results" array
"extracted": {
"content": {
"*": {//the indexes of the "content" array
"*": "[#5].&"
}
}
}
}
}
}
}
]
Also the following spec, which repeats the content of the inner array without keys, will give the same result :
[
{
"operation": "shift",
"spec": {
"results": {
"*": {
"extracted": {
"content": {
"*": "[]"// [] seems like redundant but kept for the case the array has a single object.
}
}
}
}
}
}
]
which is quite similar to your tried one.
I previously have this issue of merging data into another one to avoid duplicates and make a cleaner version of the JSON. I got a solution in here that worked like a charm for a while but after I got more information arrayed inside the JSON things got a little bit tricky.
I have this array:
{
"clubhouse": [
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "1"
},
{
"nationalityid": "2"
},
{
"nationalityid": "3"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1234",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
}
]
},
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "1"
},
{
"nationalityid": "2"
},
{
"nationalityid": "3"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "4321",
"gender": "02"
},
"inamount": "1700000",
"ratio": "12"
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1333",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
}
]
},
{
"id": "03",
"statusId": "ok",
"stateid": "5",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "",
"gender": ""
},
"inamount": "",
"ratio": ""
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1334",
"gender": "02"
},
"inamount": "1900000",
"ratio": "12"
}
]
}
]
}
I was using this JOLT but it doesnt work with the nationalities,since it loses the array they are in.
[
{
// group by "id" values to create separate objects
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "#(1,id).&",
"investors": {
"*": {
"*": {
"#": "#(4,id).&3[&4].&" // &3 -> going 3 levels up to grab literal "investors", [&4] -> going 4 levels up the tree in order to reach the indexes of "clubhouse" array, & -> replicate the leaf node values for the current key-value pair
}
}
}
}
}
}
},
{
// get rid of "null" values
"operation": "modify-overwrite-beta",
"spec": {
"*": "=recursivelySquashNulls"
}
},
{
// pick only the first components from the repeated values populated within the arrays
"operation": "cardinality",
"spec": {
"*": {
"*": "ONE",
"investors": "MANY"
}
}
},
{
// get rid of object labels
"operation": "shift",
"spec": {
"*": ""
}
}
]
What I need to get is something like this:
{
"clubhouse": [
{
"id": "01",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "1"
},
{
"nationalityid": "2"
},
{
"nationalityid": "3"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1234",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
},
{
"investor": {
"id": "4321",
"gender": "02"
},
"inamount": "1700000",
"ratio": "12"
}
]
},
{
"id": "02",
"statusId": "ok",
"stateid": "2",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "1333",
"gender": "01"
},
"inamount": "1500000",
"ratio": "12"
},
{
"investor": {
"id": "1334",
"gender": "02"
},
"inamount": "1900000",
"ratio": "12"
}
]
},
{
"id": "03",
"statusId": "ok",
"stateid": "5",
"nationalities": [
{
"nationalityid": "3"
},
{
"nationalityid": "4"
},
{
"nationalityid": "5"
}
],
"TypeId": "3",
"investors": [
{
"investor": {
"id": "",
"gender": ""
},
"inamount": "",
"ratio": ""
}
]
}
]
}
You can rearrange the first shift transformation by adding a new object tagged "nationalities" which has one level reduced identifiers compared to the already existing object tagged "investors", and the existing cardinality transformation would already pick only the first array among repeated identical "nationalities" arrays if the remaining specs are kept as they are, such as the below one
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": "#(1,id).&",
"nationalities": {
"*": {
"#": "#(3,id).&2[&3][]"
}
},
"investors": {
"*": {
"*": {
"#": "#(4,id).&3[&4].&"
}
}
}
}
}
}
},
...
]
Input is :
{
"Size": "2",
"done": "true",
"records": [
{
"Id": "a7g6s0000004GZuAAM",
"NN": "00096411.0",
"Name": "ISOLIN TRADE & INVEST"
},
{
"Id": "a7g6s0000004GZzAAM",
"Number": "00096412.0",
"Name": "ISOLIN"
}
]
}
Spec used:
[
{
"operation": "remove",
"spec": {
"records": {
"*": {
"attributes": " "
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": { //iterate on each object of records
"*": { //iterate on each element of object
"$": "Items[#1].Fields[].Name",
"#": "Items[#1].Fields[].Value"
}
}
}
}
}
]
Current Output:
{
"Size": "2",
"done": "true",
"Items": [
{
"Fields": [
{
"Name": "Id"
},
{
"Value": "a7g6s0000004GZuAAM"
},
{
"Name": "NN"
},
{
"Value": "00096411.0"
},
{
"Name": "Name"
},
{
"Value": "ISOLIN TRADE & INVEST"
},
{
"Name": "Id"
},
{
"Value": "a7g6s0000004GZzAAM"
},
{
"Name": "Number"
},
{
"Value": "00096412.0"
},
{
"Name": "Name"
},
{
"Value": "ISOLIN"
}
]
}
]
}
Expected output:
{
"Size": "2",
"done": "true",
"Items": [
{
"Fields": [
{
"Name": "Id",
"Value": "a7g6s0000004GZuAAM"
},
{
"Name": "NN",
"Value": "00096411.0"
},
{
"Name": "Name",
"Value": "ISOLIN TRADE & INVEST"
},
{
"Name": "Id",
"Value": "a7g6s0000004GZzAAM"
},
{
"Name": "Number",
"Value": "00096412.0"
},
{
"Name": "Name",
"Value": "ISOLIN"
}
]
}
]
}
You can seperate those attributes into individual objects through use of Items.&2.[#2]. pattern as prefix such as
[
{
"operation": "shift",
"spec": {
"*": "&",
"records": {
"*": {
"*": {
"$": "Items.&2.[#2].Name",
"#": "Items.&2.[#2].Value"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&",
"Items": {
"*": {
"*": "Fields[]"
}
}
}
}
]
I am struggeling with some jolt transformation. I need to extract informations from an array, but also need some uppper level informations.
I have bills and some bills have multiple attachments. I want to store this attachments in a Postgress db and for each attachment aplly the bill id...
My input
[
{
"bill_id": 1,
"entities": [
{
"type": "alpha",
"data": "foo"
},
{
"type": "beta",
"data": "bar"
}
]
},
{
"bill_id": 2,
"entities": []
}
]
My desired output
[
{
"bill_id": 1,
"type": "alpha",
"data": "foo"
},
{
"bill_id": 1,
"type": "beta",
"data": "bar"
}
]
I would be very glad, if someone could help me out
Well, i found an answer, that perfectly matches my needs. A little bit tricky, but it is working fine with two shifts:
[
{
"operation": "shift",
"spec": {
"*": {
"entities": {
"*": {
"#(2,bill_id)": "[&3].[&1].bill_id",
"type": "[&3].[&1].type",
"data": "[&3].[&1].data"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
I am trying to transform array value as string to my object using jolt. When I convert array value to String, I am not able to get the proper value in my object.
Input Json :
"id": [
"123",
"245"
],
"lastName": [
"john",
"Mary"
],
"firstName": [
"doe",
"Ann"
],
"subjects": [
[
"['Maths']",
"['Science']",
"['English']"
],
[
"['English']",
"['Accounts']",
"['Art']"
]
]
}
Jolt Spec:
[{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"$": "[&2].&3"
}
}
}
}
}]
Expected output :
[ {
"id" : "123",
"lastName" : "john",
"firstName" : "doe",
"subjects" : [ ["['Maths']","['Science']","['English']"] ]
}, {
"id" : "245",
"lastName" : "Mary",
"firstName" : "Ann",
"subjects" : [ ["['English']","['Accounts']","['Art']"] ]
} ]
Actual Output:
[ {
"id" : "123",
"lastName" : "john",
"firstName" : "doe",
"subjects" : [ "0", "1", "2" ]
}, {
"id" : "245",
"lastName" : "Mary",
"firstName" : "Ann",
"subjects" : [ "0", "1", "2" ]
} ]
Please help. Thanks in advance.
Check if this solves it. Think you were missing a level at the subjects array:
[{
"operation": "shift",
"spec": {
"*": {
"*": {
"*": {
"$": "[&2].&3"
}
}
},
"subjects": {
"*": {
"*": {
"*": {
"$": "[&3].&4"
}
}
}
}
}
}]
cheers