I want to write a Jolt definition using cardinality-Many that can transform object "PO_POD_LN_EVW1" into list and ignore if its already a list.
Input JSON :
{
"PURCHASE_ORDER_DISPATCH": {
"MsgData": {
"Transaction": {
"PSCAMA": {
"PUBLISH_RULE_ID": {
"IsChanged": "Y"
}
},
"PO_POD_HDR_EVW1": {
"STATE_BILL": "",
"CURRENCY_CD": {
"IsChanged": "Y",
"content": "USD"
},
"ADDRESS4_VNDR": "",
"PO_POD_LN_EVW1": {
"WG_ACCOUNT": 641100,
"LINE_NBR": {
"IsChanged": "Y",
"content": 1
},
"ITM_ID_VNDR": "B0798CX2Q9",
"PO_POD_SHP_EVW1": {
"LINE_NBR": {
"IsChanged": "Y",
"content": 1
}
}
},
"WG_ADDR_SEQ_NUM": 1
}
}
}
}
}
JOLT Spec :
{
"operation": "cardinality",
"spec": {
"PURCHASE_ORDER_DISPATCH": {
"MsgData": {
"Transaction": {
"PO_POD_HDR_EVW1": "MANY"
}
}
}
}
}
Getting this error: Failed to Transform
Your jolt spec is correct, But You should wrap all of your specs in the [] array.
Try this:
[
{
"operation": "cardinality",
"spec": {
"PURCHASE_ORDER_DISPATCH": {
"MsgData": {
"Transaction": {
"PO_POD_HDR_EVW1": "MANY"
}
}
}
}
}
]
Related
I want to write a Jolt definition that loops through each object in my attachment array, and move the object to the data array. At the same time, it should map single object attachment nest, and move the object to the data object.
Currently my spec below is working when my attachment block is an array.
Input :
{
"IntegrationEntities": {
"integrationEntity": [
{
"integrationEntityHeader": {
"attachments": {
"attachment": [
{
"name": "EV10044.docx"
},
{
"name": "Test1.txt"
}
]
}
}
}
]
}
}
JOLT Spec :
[
{
"operation": "shift",
"spec": {
"IntegrationEntities": {
"integrationEntity": {
"*": {
"integrationEntityHeader": {
"attachments": {
"attachment": {
"*": {
"name": "data[&1].filename"
}
}
}
}
}
}
}
}
}
]
Current Output :
{
"data": [
{
"filename": "EV10044.docx"
},
{
"filename": "Test1.txt"
}
]
}
I want to handle the scenario in a way where regardless if the attachment is an array or object it should give the input. Currently it fails if the attachment block is an object and gives the output as NULL.
IF Input is:
{
"IntegrationEntities": {
"integrationEntity": [
{
"integrationEntityHeader": {
"attachments": {
"attachment": {
"name": "EV10044.docx"
}
}
}
}
]
}
}
Desired Output:
{
"data": {
"filename": "EV10044.docx"
}
}
You can use this spec:
[
{
"operation": "shift",
"spec": {
"IntegrationEntities": {
"integrationEntity": {
"*": {
"integrationEntityHeader": {
"attachments": {
"attachment": {
"name": "data.filename",
"*": {
"name": "data[&1].filename"
}
}
}
}
}
}
}
}
}
]
I just added the below line to your spec. So if you have name in the attachment it is assumed that you have 1 attachment. If no name is found it is assumed you have an array and your other codes is correct.
"name": "data.filename",
An option is to use a shift transformation spec in which the objects are qualified by path expressions and the arrays are constructed along with sub-indexes by symbolic expression "*": { such as
[
{
"operation": "shift",
"spec": {
"#IntegrationEntities.integrationEntity": {
"*": { // indexes of the array
"#integrationEntityHeader.attachments.attachment": {
"*": { // indexes of the array
"*": "data[].file&", // if "attachment" is an array
"#1,name": "data.file&" // if "attachment" is an object
}
}
}
}
}
}
]
Btw, if you didn't need to rename the innermost attribute name to fieldname, the following shorter spec would suffice :
[
{
"operation": "shift",
"spec": {
"#IntegrationEntities.integrationEntity": {
"*": {
"#integrationEntityHeader.attachments.attachment": "data"
}
}
}
}
]
I would like to know if it is possible in Jolt to concatenate values from different objects. I've successfully concatenated "orderType" (value=ABC) and "minorCode" (value 0003) from the "orderInformation" object to create an output key of "Job.JobTypeCd" (value ABC0003). I would like to add the "serviceType" value as a prefix to that new key so the output value would be "123ABC0003".
Example input
{
"orderInformation": {
"orderType": "ABC",
"minorCode": "0003"
},
"account": {
"serviceType": "123"
}
}
Current Spec
[
{
"operation": "modify-default-beta",
"spec": {
"orderInformation": {
"JobType": "=concat(#(1,orderType),#(1,minorCode))"
}
}
},
{
"operation": "shift",
"spec": {
"orderInformation": {
"JobType": "Job.JobTypeCd"
},
"account": {
"serviceType": "Job.AddThisAsPrefix2JobTypeCd"
}
}
}
]
Current output
{
"Job" : {
"JobTypeCd" : "ABC0003",
"AddThisAsPrefix2JobTypeCd" : "123"
}
}
Desired output
{
"Job" : {
"JobTypeCd" : "123ABC0003"
}
}
You can include the serviceType value to the same concat operation as below.
"JobType": "=concat(#(2,account.serviceType),#(1,orderType),#(1,minorCode))"
Full spec
[
{
"operation": "modify-default-beta",
"spec": {
"orderInformation": {
"JobType": "=concat(#(2,account.serviceType),#(1,orderType),#(1,minorCode))"
}
}
},
{
"operation": "shift",
"spec": {
"orderInformation": {
"JobType": "Job.JobTypeCd"
}
}
}
]
I have a complex structure similar to:
{
type: "Data",
data: [
"item1": {...},
"item2": {...},
...
"itemN": {
"otherStructure": {
"testData": [
{
"values": {...}
},
{
"values": {
"importantKey": ObjectId("23a2345gf651")
}
}
]
}
}
]
}
For such kind of data structure I want to update data type for all these importantKeys from ObjectId into string.
I was trying queries similar to:
db.collection.updateMany(
{type:"Data"},
{$toString: "data.$[element].otherStructure.testData.$[element].values.importantKey"},
{"data.element.otherStructure.testData.element.values.importantKey": {$type: "objectId"}})
But all these tries were no successful.
So, are there any adequate solutions for updating such data?
UPDATE
Sorry for confusing, my structure is more complex:
data.content.$[element].otherStructure.testData.keys.$[element].values.$[element].meta.importantKey
All these $[element] elements means objects with list of elements.
You may use this workaround:
db.collection.aggregate([
{
$addFields: {
"data.content": {
$map: {
input: "$data.content",
as: "data",
in: {
otherStructure: {
testData: {
keys: {
$map: {
input: "$$data.otherStructure.testData.keys",
as: "testData",
in: {
"values": {
$map: {
input: "$$testData.values",
as: "values",
in: {
"meta": {
"importantObject": {
"importantKey": {
$toString: "$$values.meta.importantObject.importantKey"
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
},
{$out:"collection"}
])
MongoPlayground
I have below data:
{
"results":[
{
"ID":"1",
"products":[
{
"product":"car",
"number":"5"
},
{
"product":"computer",
"number":"212"
}
]
},
{
"ID":"2",
"products":[
{
"product":"car",
"number":"9"
},
{
"product":"computer",
"number":"463"
},
{
"product":"bicycle",
"number":"5"
}
]
}
]
}
And my query is below:
{
"query":{
"bool":{
"must":[
{
"wildcard":{
"results.products.product":"*car*"
}
},
{
"wildcard":{
"results.products.number":"*5*"
}
}
]
}
}
}
What I expect is to get only ID1. Because only it has a product with { "product":"car", "number":"5" } record. But what I get is both ID1 and ID2 because ID2's first record has "product":"car" and third record has "number":"5" records separately.
How can I fix this query?
You need to define your products as a nested type when creating mapping. Try with following mapping example:
PUT http://localhost:9200/indexname
{
"mappings": {
"typename": {
"properties": {
"products" : {
"type" : "nested"
}
}
}
}
}
Then you can use nested queries to match entire elements of your array - just as you need to.
{
"query": {
"nested": {
"path": "products",
"query": {
"bool": {
"must": [
{ "wildcard": { "products.product": "*car*" }},
{ "wildcard": { "products.number": "*5*" }}
]
}
}
}
}
}
I am new in groovy and I want to construct a json object with the builder
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{ "match": { "content": "scontent" } },
{ "match": { "title":"stitle" } }
]
}
},
{
"bool": {
"should": [
{ "match": { "a1": "v1" } },
{ "match": { "a2":"v2" } },
... and so on ...
{ "match": { "an":"vn" } }
]
}
}
]
}
},
"highlight": {
"fields": {
"content":{}
}
}
}
I search a lot of on other posts on stackoverflow and I write this code
So I did this but no way to get what I want :
JsonBuilder builder = new JsonBuilder()
def body = builder {
from Lib.or(qQuery.start, 0)
size Lib.or(qQuery.num, 10)
query {
bool {
must [
{
bool {
should [
{ match { content 'scontent' } },
{ match { title 'stitle' } }
]
}
},
{
bool {
should myVals.collect {[
'match' : { it.key it.value }
]}
}
}
]
}
}
highlight {
fields {
content {}
}
}
}
Thanks for any help !
I think you can make this work with the JsonBuilder as is, but it is usually easier to create the data structure using maps and lists (which is what the builder outputs) in groovy as you have more control there.
Example code:
import groovy.json.*
def data = [
query: [
bool: [
must: [
[bool:
[should: [
[match: [ content: 'scontent']],
[match: [ title: 'stitle']]
]]
],
[bool:
[should: [
[match: [ a1: 'v1']],
[match: [ a2: 'v2']],
[match: [ vn: 'vn']]
]]
]
]
]
]
]
println JsonOutput.prettyPrint(JsonOutput.toJson(data))
produces:
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"match": {
"content": "scontent"
}
},
{
"match": {
"title": "stitle"
}
}
]
}
},
{
"bool": {
"should": [
{
"match": {
"a1": "v1"
}
},
{
"match": {
"a2": "v2"
}
},
{
"match": {
"vn": "vn"
}
}
]
}
}
]
}
}
}
I did not include your full json as it takes up some space, but the structure is there. Note the use of lists ([valueA, valueB]) vs maps ([someKey: someValue]) in the data structure.
Granted this makes the JsonBuilder less than 100% useful but I haven't seen any concise ways of including lists of large json objects in a list within the structure. You can do:
def json = JsonBuilder()
json.query {
bool('list', 'of', 'values')
}
but for larger structures as list elements I would say go with the lists and maps approach.