I am struggeling with some jolt transformation. I need to extract informations from an array, but also need some uppper level informations.
I have bills and some bills have multiple attachments. I want to store this attachments in a Postgress db and for each attachment aplly the bill id...
My input
[
{
"bill_id": 1,
"entities": [
{
"type": "alpha",
"data": "foo"
},
{
"type": "beta",
"data": "bar"
}
]
},
{
"bill_id": 2,
"entities": []
}
]
My desired output
[
{
"bill_id": 1,
"type": "alpha",
"data": "foo"
},
{
"bill_id": 1,
"type": "beta",
"data": "bar"
}
]
I would be very glad, if someone could help me out
Well, i found an answer, that perfectly matches my needs. A little bit tricky, but it is working fine with two shifts:
[
{
"operation": "shift",
"spec": {
"*": {
"entities": {
"*": {
"#(2,bill_id)": "[&3].[&1].bill_id",
"type": "[&3].[&1].type",
"data": "[&3].[&1].data"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
Related
I am trying to transform the JSON as below expected output. Stuck with the spec below. Can someone help in this?
There is an inner array with name "content" in the "results" array which I want to make it as a part of main array.
Input JSON
{
"total": 100,
"start": 1,
"page-length": 10,
"results": [
{
"index": 1,
"uri": "uri1/uri2",
"extracted": {
"kind": "object",
"content": [
{
"code": "A1",
"region": "APAC"
}
]
}
},
{
"index": 2,
"uri": "uri1/uri2",
"extracted": {
"kind": "object",
"content": [
{
"code": "B1",
"region": "AMER"
}
]
}
},
{
"index": 3,
"uri": "uri1/uri2",
"extracted": {
"kind": "object",
"content": [
{
"code": "C1",
"region": "APAC"
}
]
}
}
]
}
Expected json output
[
{
"code": "A1",
"region": "APAC"
},
{
"code": "B1",
"region": "AMER"
},
{
"code": "C1",
"region": "APAC"
}
]
Spec Tried
[
{
"operation": "shift",
"spec": {
"results": {
"*": {
"extracted": { "content": { "#": "&" } }
}
}
}
},
{
"operation": "shift",
"spec": {
"content": {
"#": "&"
}
}
}
]
Find the output below I am getting on Jolt tool
You can use # wildcard nested within square brackets in order to reach the level of the indexes of the "results" array such that
[
{
"operation": "shift",
"spec": {
"results": {
"*": {//the indexes of the "results" array
"extracted": {
"content": {
"*": {//the indexes of the "content" array
"*": "[#5].&"
}
}
}
}
}
}
}
]
Also the following spec, which repeats the content of the inner array without keys, will give the same result :
[
{
"operation": "shift",
"spec": {
"results": {
"*": {
"extracted": {
"content": {
"*": "[]"// [] seems like redundant but kept for the case the array has a single object.
}
}
}
}
}
}
]
which is quite similar to your tried one.
How can I break and flatten nested JSON with arrays using Jolt transformations
from:
{
"product": [
"test1",
"test2"
],
"Purchase": [
1,
2
],
"Renewal": [
1,
2
]
}
to:
[
{
"product": "test1",
"Purchase": 1,
"Renewal": 1
},
{
"product": "test2",
"Purchase": 2,
"Renewal": 2
}
]
I want to flatten this array json file in order to fit to sql db format.
You can use a shift transformation spec such as
[
{
"operation": "shift",
"spec": {
"*": {
"*": {
"#": "[#2].&2"
}
}
}
}
]
where &2 leaf separates values into new sub-arrays by going to levels up to grab their current respective array names, and [#2] re-organizes them to form array of objects.
the demo on the site http://jolt-demo.appspot.com/ is
Edit : If the input is like the below one as asked in the last comment
{
"product": [
"test- 1",
"test- 2"
],
"Purchase": [
2,
2
],
"Renewal": 1
}
then you can consider using
[
{
"operation": "shift",
"spec": {
"*r*": {
"*": {
"#": "[#2].&2",
"#(2,Renewal)": "[#2].Renewal"
}
}
}
},
{
"operation": "cardinality",
"spec": {
"*": {
"Re*": "ONE"
}
}
}
]
I have deeply nested data that looks like the following. The data may look needlessly nested, but I've removed unimportant data to reduce some clutter.
[
{
"id": "1",
"entries": [
{
"data": {
"values": [
{
"data_of_interest": {
"frames": [
{
"filename": "arg",
"module": "bar",
"package": "foo",
"platform": "blargh"
},
{
"filename": "arg1",
"module": "barge",
"package": "something",
"platform": "blargh.io"
}
]
}
}
]
}
}
]
},
{
"id": "2",
"entries": [
{
"data": {
"values": [
{
"data_of_interest": {
"frames": [
{
"filename": "app",
"module": null,
"package": null,
"platform": null
}
]
}
}
]
}
}
]
}
]
What I want to do is distribute the 'id' field into each record in the 'frames' array, as well as the index of each frame, then flatten out all the frames into a single list. The resulting data would look like this:
[
{
"id": 1,
"frame_idx": 0,
"filename": "arg",
"module": "bar",
"package": "foo",
"platform": "blargh"
},
{
"id": 1,
"frame_idx": 1,
"filename": "arg1",
"module": "barge",
"package": "something",
"platform": "blargh.io"
},
{
"id": 2,
"frame_idx": 0,
"filename": "app",
"module": null,
"package": null,
"platform": null
}
]
For the life of me, I can't figure out how to properly distribute the id or index into the frame records. My attempts always place id and index beside, but not within the frame records.
Start by deep diving upto the innermost level(level of indexes of the "frames" list). Then, determine the common factor(#(9,id)) in order to nest the result under common arrays while seperating by [&1] such as
[
{
"operation": "shift",
"spec": {
"*": { // top level indexes
"entries": {
"*": { //indexes of "entries"
"*": { // "data"
"*": { // "values"
"*": { // indexes of "values"
"*": { // "data_of_interest"
"*": { // "frames"
"*": { // indexes of "frames"
"#(8,id)": "#(9,id).[&1].id", // go 8 or 9 levels up respectively in order to grab the value of "id"
"$": "#(9,id).[&1].frame_idx",
"*": "#(9,id).[&1].&"
}
}
}
}
}
}
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": ""
}
}
}
]
the demo on the site http://jolt-demo.appspot.com is
I want to do a Jolt transformation with a json array, and i only want the ones with certain attribute.
For example:
Input:
{
"characteristic": [
{
"name": "BrandId",
"value": "b"
},
{
"name": "status",
"value": "SENT"
},
{
"name": "statusTxt",
"value": "sent"
}
]
}
I want output to be
{
"status":"SENT",
"statusTxt":"sent"
}
This will lead to the output you want based on your input:
[
{
"operation": "shift",
"spec": {
"characteristic": {
"*": {
"name": {
"status": {
"#(2,value)": "status"
},
"statusTxt": {
"#(2,value)": "statusTxt"
}
}
}
}
}
}
]
Can Jolt flatten an array of objects which contain array? For example, is there a way to write Jolt transformation specs for the following input and output?
Jolt https://github.com/bazaarvoice/jolt
Input:
[
{"Id": "111", ["mobile": "1111", "home": "1112"]},
{"Id": "222", ["mobile": "2221"]}
]
Output:
[
{"Id": "111", "mobile": "1111"},
{"Id": "111", "home": "1112"},
{"Id": "222", "mobile": "2221"}
]
I didn't find a way to represent the output array indexes.
assuming input is fixed as
[
{
"Id": "111",
"Val": [
{
"mobile": "1111"
},
{
"home": "1112"
}
]
},
{
"Id": "222",
"Val": [
{
"mobile": "2221"
}
]
}
]
then applying the following transformation
[
{
"operation": "shift",
"spec": {
"*": {
"Id": null,
"Val": {
"*": {
"#": "[&3].[&0]",
"#(2,Id)": "[&3].[&0].id"
}
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"*": "[]"
}
}
}
]
reach the expected output:
[ {
"mobile" : "1111",
"id" : "111"
}, {
"home" : "1112",
"id" : "111"
}, {
"mobile" : "2221",
"id" : "222"
} ]