Looping through multiple line items in a single array output in JOLT - arrays

I want to write a Jolt definition that loops through each object in my attachment array, and move the object to the data array. At the same time, it should map single object attachment nest, and move the object to the data object.
Currently my spec below is working when my attachment block is an array.
Input :
{
"IntegrationEntities": {
"integrationEntity": [
{
"integrationEntityHeader": {
"attachments": {
"attachment": [
{
"name": "EV10044.docx"
},
{
"name": "Test1.txt"
}
]
}
}
}
]
}
}
JOLT Spec :
[
{
"operation": "shift",
"spec": {
"IntegrationEntities": {
"integrationEntity": {
"*": {
"integrationEntityHeader": {
"attachments": {
"attachment": {
"*": {
"name": "data[&1].filename"
}
}
}
}
}
}
}
}
}
]
Current Output :
{
"data": [
{
"filename": "EV10044.docx"
},
{
"filename": "Test1.txt"
}
]
}
I want to handle the scenario in a way where regardless if the attachment is an array or object it should give the input. Currently it fails if the attachment block is an object and gives the output as NULL.
IF Input is:
{
"IntegrationEntities": {
"integrationEntity": [
{
"integrationEntityHeader": {
"attachments": {
"attachment": {
"name": "EV10044.docx"
}
}
}
}
]
}
}
Desired Output:
{
"data": {
"filename": "EV10044.docx"
}
}

You can use this spec:
[
{
"operation": "shift",
"spec": {
"IntegrationEntities": {
"integrationEntity": {
"*": {
"integrationEntityHeader": {
"attachments": {
"attachment": {
"name": "data.filename",
"*": {
"name": "data[&1].filename"
}
}
}
}
}
}
}
}
}
]
I just added the below line to your spec. So if you have name in the attachment it is assumed that you have 1 attachment. If no name is found it is assumed you have an array and your other codes is correct.
"name": "data.filename",

An option is to use a shift transformation spec in which the objects are qualified by path expressions and the arrays are constructed along with sub-indexes by symbolic expression "*": { such as
[
{
"operation": "shift",
"spec": {
"#IntegrationEntities.integrationEntity": {
"*": { // indexes of the array
"#integrationEntityHeader.attachments.attachment": {
"*": { // indexes of the array
"*": "data[].file&", // if "attachment" is an array
"#1,name": "data.file&" // if "attachment" is an object
}
}
}
}
}
}
]
Btw, if you didn't need to rename the innermost attribute name to fieldname, the following shorter spec would suffice :
[
{
"operation": "shift",
"spec": {
"#IntegrationEntities.integrationEntity": {
"*": {
"#integrationEntityHeader.attachments.attachment": "data"
}
}
}
}
]

Related

Perform cardinality-Many in JOLT

I want to write a Jolt definition using cardinality-Many that can transform object "PO_POD_LN_EVW1" into list and ignore if its already a list.
Input JSON :
{
"PURCHASE_ORDER_DISPATCH": {
"MsgData": {
"Transaction": {
"PSCAMA": {
"PUBLISH_RULE_ID": {
"IsChanged": "Y"
}
},
"PO_POD_HDR_EVW1": {
"STATE_BILL": "",
"CURRENCY_CD": {
"IsChanged": "Y",
"content": "USD"
},
"ADDRESS4_VNDR": "",
"PO_POD_LN_EVW1": {
"WG_ACCOUNT": 641100,
"LINE_NBR": {
"IsChanged": "Y",
"content": 1
},
"ITM_ID_VNDR": "B0798CX2Q9",
"PO_POD_SHP_EVW1": {
"LINE_NBR": {
"IsChanged": "Y",
"content": 1
}
}
},
"WG_ADDR_SEQ_NUM": 1
}
}
}
}
}
JOLT Spec :
{
"operation": "cardinality",
"spec": {
"PURCHASE_ORDER_DISPATCH": {
"MsgData": {
"Transaction": {
"PO_POD_HDR_EVW1": "MANY"
}
}
}
}
}
Getting this error: Failed to Transform
Your jolt spec is correct, But You should wrap all of your specs in the [] array.
Try this:
[
{
"operation": "cardinality",
"spec": {
"PURCHASE_ORDER_DISPATCH": {
"MsgData": {
"Transaction": {
"PO_POD_HDR_EVW1": "MANY"
}
}
}
}
}
]

Transform Object in Array with Jolt

I'm trying to convert a JSON object to an array contained one element
Input :
{
"record": {
"file_header": {
"file_name": "TEST_FILE"
},
"scheda_contatto": {
"phone_number": "5555-111-222-333",
"type_call": 1
}
}
}
i want scheda_contatto to be an array with only 1 element
Output:
{
"record": {
"file_header": {
"file_name": "TEST_FILE"
},
"scheda_contatto": [
{
"phone_number": "5555-111-222-333",
"type_call": 1
}
]
}
}
You can use &1[0].& in order to nest all innermost sub-elements(&) of the single object by square-brackets([0]) with key name scheda_contatto(&1 under s*) while prefixing each value identifier by &1 and &2 respectively in order to replicate the wrapper key name record for both such as
[
{
"operation": "shift",
"spec": {
"rec*": {
"*": "&1.&",
"s*": {
"*": "&2.&1[0].&"
}
}
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is

Is is possible to concatenate values from two different objects in Jolt

I would like to know if it is possible in Jolt to concatenate values from different objects. I've successfully concatenated "orderType" (value=ABC) and "minorCode" (value 0003) from the "orderInformation" object to create an output key of "Job.JobTypeCd" (value ABC0003). I would like to add the "serviceType" value as a prefix to that new key so the output value would be "123ABC0003".
Example input
{
"orderInformation": {
"orderType": "ABC",
"minorCode": "0003"
},
"account": {
"serviceType": "123"
}
}
Current Spec
[
{
"operation": "modify-default-beta",
"spec": {
"orderInformation": {
"JobType": "=concat(#(1,orderType),#(1,minorCode))"
}
}
},
{
"operation": "shift",
"spec": {
"orderInformation": {
"JobType": "Job.JobTypeCd"
},
"account": {
"serviceType": "Job.AddThisAsPrefix2JobTypeCd"
}
}
}
]
Current output
{
"Job" : {
"JobTypeCd" : "ABC0003",
"AddThisAsPrefix2JobTypeCd" : "123"
}
}
Desired output
{
"Job" : {
"JobTypeCd" : "123ABC0003"
}
}
You can include the serviceType value to the same concat operation as below.
"JobType": "=concat(#(2,account.serviceType),#(1,orderType),#(1,minorCode))"
Full spec
[
{
"operation": "modify-default-beta",
"spec": {
"orderInformation": {
"JobType": "=concat(#(2,account.serviceType),#(1,orderType),#(1,minorCode))"
}
}
},
{
"operation": "shift",
"spec": {
"orderInformation": {
"JobType": "Job.JobTypeCd"
}
}
}
]

jolt shift array up a level, but preserve the array even if empty

I struggled hard with how to title this one. This is from the twitter sample stream. I need the hashtag text pulled out into an array at the top level. I have that working, but if there is just one hashtag it jolt's to a single string, and if there are no hashtags (twitter gives back an empty array in that case), then the field disappears. I want to do the same thing with the URL field & user mentions.
Input JSON:
{"entities": {
"hashtags": [{"text": "somethingClever",
"indices": [22, 28]},
{"text": "somethingLessClever",
"indices": [30, 40]}],
"urls": [],
"user_mentions": [{
"screen_name": "username",
"indices": [3, 19]
}],
"symbols": []
}
Desired output:
{
"hashtags": ["somethingClever"],
"urls": [],
"user_mentions": ["username"]
}
But what I'm getting is this:
Actual output, note screen_name is not an array and urls is altogether missing. That is the crux of my problem.
{
"hashtag_text" : [ "somethingClever", "somethingLessClever" ],
"screen_name" : "username"
}
Here is my jolt spec so far:
[
{
"operation": "shift",
"spec": {
"entities": {
"hashtags": { "*": { "text": ".hashtag_text" } },
"urls": { "*": { "url_text": ".urls" } },
"user_mentions": { "*": { "screen_name": ".&" } }
}
}
}
]
I figured it out. I used shift to bring the necessary array elements to the next level up (from entities to root). Next I used the default operation to make it still be an array even if the array was empty. Lastly, I used the cardinality operation to make it be an array even if there is only one element.
Hopefully this helps somebody.
[
{
"operation": "shift",
"spec": {
"entities": {
"hashtags": { "*": { "text": ".hashtags" } },
"urls": { "*": { "url_text": ".urls" } },
"user_mentions": { "*": { "screen_name": ".&" } }
}
}
},
{
"operation": "default",
"spec": {
"hashtags[]": {},
"urls[]": {},
"screen_name[]": {}
}
},
{
"operation": "cardinality",
"spec": {
"hashtags": "MANY",
"urls": "MANY",
"screen_name": "MANY"
}
}
]

How to use jolt to parse array of array

I want to use jolt to parse the json. But I can't get the age value in the first level successfully. May someone help me?
The input json is like this:
{
"name": "abc",
"age": "20",
"Photos": [
{
"a": "AAA.jpg",
"b": "BBB.jpg",
"XXX123": [
{
"v1": "AAA.jpg",
"v2": "BBB.jpg"
}
]
}
]
}
My Spec:
[
{
"operation": "shift",
"spec": {
"Photos": {
"*": {
"XXX123": {
"*": {
//"*": "&2.[&1].&",
"#(2,a)": "&2.[&1].a",
"#(2,b)": "&2.[&1].b",
"#(3,age)": "&3.[&1].age"
}
}
}
}
}
}
]
The output I want:
{
"XXX123" : [ {
"a" : "AAA.jpg",
"b" : "BBB.jpg",
"age" : "20"
} ]
}
Spec
[
{
"operation": "shift",
"spec": {
"Photos": {
"*": {
"XXX123": {
"*": {
//"*": "&2.[&1].&",
"#(2,a)": "&2.[&1].a",
"#(2,b)": "&2.[&1].b",
"#(4,age)": "&2.[&1].age"
}
}
}
}
}
}
]
You were close.
"#(3,age)": "&3.[&1].age"
needed
"#(4,age)": "&2.[&1].age"
From where that line was you needed to go up the tree 5 levels.
Counting from Zero
0 -> "*"
1 -> "XXX123"
2 -> "*"
3 -> "Photos"
4 -> your top level input
then lookup "age" in the top level map

Resources