Im trying to map data from database to salesforce in mule is using CreateBulk option in Salesforce Connector
-->
<flow name="datamapperFlow1" doc:name="datamapperFlow1">
<poll doc:name="Poll">
<fixed-frequency-scheduler frequency="30" timeUnit="SECONDS"/>
<db:select config-ref="Generic_Database_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[select * from Persons1]]></db:parameterized-query>
</db:select>
</poll>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
<foreach doc:name="For Each">
<data-mapper:transform config-ref="Map_To_HRISASI__c" doc:name="Map To HRISASI__c"/>
</foreach>
<sfdc:create-bulk config-ref="Salesforce" type="HRISASI__c" doc:name="Salesforce">
<sfdc:objects ref="#[payload]"/>
</sfdc:create-bulk>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>
A batch job is created but records are failed to insert with the below error in sales force
<result>
<errors>
<fields>PHONE</fields>
<fields>EMPNAME</fields>
<fields>EMPLOYEEID</fields>
<message>
HRISASI__c: bad field names on insert/update call: PHONE, EMPNAME, EMPLOYEEID
</message>
<statusCode>INVALID_FIELD_FOR_INSERT_UPDATE</statusCode>
</errors>
<success>false</success>
<created>false</created>
</result>
my Data Mapper script looks like the below
//MEL
//START -> DO NOT REMOVE
output.__id = input.__id;
//END -> DO NOT REMOVE
output.EmployeeId__c = input.EmpId;
output.EmpName__c = input.Empname;
output.Phone__c = input.Phone;
Could someone please tell me what is the reason for the error
Thankyou in advance
The problem was because of using the datamapper inside the for each.
Payload outside the foreach doesn't contains the mapping and hence error was thrown when trying to map the data to salesforce
Related
I have to retrieve 20 tables from ODATA service with mule 3. The solution that I have created is using scatter gather to make these http request calls and the fetched payload is saved into files.
It works fine only it look not very elegant, because you see scatter gather with a lot of http request connectors. There is no ODATA connector for 3.9 mule and Foreach was not possible to loop into the calls. Do you guys have any idea how to solve it with less connectors or logic?
<flow name="proces-flow">
<poll doc:name="Poll">
<fixed-frequency-scheduler frequency="30000"/>
<logger message="start flow" level="INFO" doc:name="Logger"/>
</poll>
<scatter-gather doc:name="Scatter-Gather">
<http:request config-ref="HTTP_Request_Configuration" path="${http.cat.path}" method="GET" doc:name="HTTP"/>
<http:request config-ref="HTTP_Request_Configuration" path="${http.sub.path}" method="GET" doc:name="HTTP"/>
<http:request config-ref="HTTP_Request_Configuration" path="${http.sub2.path}" method="GET" doc:name="HTTP"/>
<http:request config-ref="HTTP_Request_Configuration" path="${http.sub3.path}" method="GET" doc:name="HTTP"/>
<http:request-config name="HTTP_Request_Configuration1" protocol="HTTPS" host="caller-test.service.com" port="443" basePath="/repo/v1/odata/Incidents?$filter=callDate gt 2019-09-01T08:00:00Z and callDate lt 2019-09-01T09:00:00Z" doc:name="HTTP Request Configuration">
</scatter-gather>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
payload]]></dw:set-payload>
</dw:transform-message>
<file:outbound-endpoint path="C:\Local\fileshare-service" outputPattern="output.json" responseTimeout="10000" doc:name="File"/>
<exception-strategy ref="" doc:name="Reference Exception Strategy"/>
</flow>
As you can see if I have to make 20 queries I have to add 20 calls. Because it's a ODATA I use the base path to add the hole query in it.
This works but, as I say, it's not the solution you want if the application will grow. For those how have query inside the request I have to create new http request configuration every time, because I use the base path to add variable etc and that can't be done in usual path.
I am trying to access parent "Record ID" (Auto generated ID) in a MULESOFT Flow and store it into child table of SalesForce and store that record id in one variable and pass it to child table through DataWeave Transformation. I am using one flow for this task. Can anyone please tell me how can i get parent table Record ID? Thanks.
Here is my flow Confi. XML file
<flow name="shoppingcartpocmcpostsfuserFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/customeruser" allowedMethods="POST" doc:name="HTTP"/>
<object-to-byte-array-transformer doc:name="Object to Byte Array"/>
<dw:transform-message doc:name="Transform Message" metadata:id="4851f81e-2a1b-4e5e-a34f-cc328e6dd23d">
<dw:input-payload mimeType="application/json"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
[{
Name: payload.ShoppingCartUser.ShoppingCartUserName,
FirstName__c: payload.ShoppingCartUser.FirstName,
LastName__c: payload.ShoppingCartUser.LastName,
EmailID__c: payload.ShoppingCartUser.EmailID,
PhoneNumber__c: payload.ShoppingCartUser.PhoneNumber,
CompanyName__c: payload.ShoppingCartUser.Company,
CustomerType__c: payload.ShoppingCartUser.Customer_Type
}]]]></dw:set-payload>
</dw:transform-message>
<sfdc:create config-ref="Salesforce__Basic_Authentication" type="MS_ShoppingCartUser__c" doc:name="Salesforce Connector for User Details">
<sfdc:objects ref="#[payload]"/>
</sfdc:create>
<object-to-string-transformer doc:name="Object to String"/>
<set-variable variableName="id" value="#[flowVars.id]" doc:name="Variable"/>
<dw:transform-message doc:name="Transform Message for User Address" metadata:id="4851f81e-2a1b-4e5e-a34f-cc328e6dd23d">
<dw:input-payload mimeType="application/json"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
[{
AddressLine1__c: payload.ShoppingCartUser.AddressLine_1,
AddressLine2__c: payload.ShoppingCartUser.AddressLine_2,
CountyName__c: payload.ShoppingCartUser.County,
City__c: payload.ShoppingCartUser.City,
Country__c: payload.ShoppingCartUser.Country,
State__c: payload.ShoppingCartUser.State,
ZipCode__c: payload.ShoppingCartUser.Zipcode,
EmailID__c: id
}]]]></dw:set-payload>
</dw:transform-message>
<sfdc:create config-ref="Salesforce__Basic_Authentication" type="MS_ShoppingCart__c" doc:name="Salesforce Connector for User Address">
<sfdc:objects ref="#[payload]"/>
</sfdc:create>
<object-to-string-transformer doc:name="Object to String"/>
</flow>
The sfdc:create operation will return a 'Result' https://docs.mulesoft.com/connectors/salesforce/salesforce-connector-reference-96#Result
It has an 'Id' field. You can access the id using 'payload.Id'
Note in your example you will need to remove the object-to-string to access it. It will also overwrite your payload. So you need to use an enricher if you need the same payload values in your second transform.
I query the data(10 - 20 records) from salesforce contact object and from that i need to capture id value into the variable. to do that i'm setting payload as #[org.apache.commons.collections.IteratorUtils.toList(payload)] but not working.
PFB is the sample code:
<flow name="SetFunctionRole-table">
<sfdc:query config-ref="SFA_NOL_CLOUDHUB2" query="dsql:SELECT Id, AccountId,Account_BP_ID__c,Account_BT_Code__c,Birthdate,Department,Email,Fax,FirstName,LastName,MiddleName,MobilePhone,Name FROM Contact WHERE AccountId = '#[flowVars.varCustomer_Id]' ORDER BY AccountId ASC" doc:name="get contacts from AccountId"/>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
<set-payload value="#[org.apache.commons.collections.IteratorUtils.toList(payload)]" doc:name="Set Payload"/>
<foreach collection="#[payload]" doc:name="For Each">
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
<set-variable variableName="varContact_id" value="#[payload['Id']]" doc:name="Variable"/>
<sfdc:query-single config-ref="SFA_NOL_CLOUDHUB2" query="dsql:SELECT Function__c,Id FROM Contact_Function_Role__c WHERE Contact__c = '#[flowVars.varContact_id]'" doc:name="Salesforce"/>
<logger message="#['Inserting key:' + flowVars.varContact_id + ' and value: ' + payload.Id]" level="INFO" doc:name="Logger"/>
<objectstore:store config-ref="ObjectStore__Connector" key="#[flowVars.varContact_id]" value-ref="#[payload.Id]" overwrite="true" doc:name="ObjectStore"/>
</foreach>
</flow>
Getting below error message at set payload:
ERROR 2017-07-24 20:05:50,333 [[ws21.2-prod].SetFunctionRole-table.stage1.02] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Execution of the expression "org.apache.commons.collections.IteratorUtils.toList(payload)" failed. (org.mule.api.expression.ExpressionRuntimeException).
Payload : org.mule.streaming.ConsumerIterator#665c8b2e
Payload Type : org.mule.streaming.ConsumerIterator
Element : /SetFunctionRole-table/processors/2 # ws21.2-prod:create-prospect.xml:177 (Set Payload)
Element XML : <set-payload value="#[org.apache.commons.collections.IteratorUtils.toList(payload)]" doc:name="Set Payload"></set-payload>
--------------------------------------------------------------------------------
Root Exception stack trace:
[UnexpectedErrorFault [ApiFault exceptionCode='INVALID_OPERATION_WITH_EXPIRED_PASSWORD'
exceptionMessage='The users password has expired, you must call SetPassword before attempting any other API operations'
extendedErrorDetails='{[0]}'
]
]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
Why are you trying to Set the Payload is such a 'creative' way, not necessary and is not going to work either.
The SalesForce connector output is of type org.mule.streaming.ConsumerIterator which can be read directly by a For Each scope, so this already is a collection.
Remove the Set Payload and see if that works, if you debug you should see the behaviour, I've just tested this and this works.
I want to write a mule application which will read the database for unprocessed records, club them in JSON payload and then hit a REST webservice, REST webservice(Hosted on different server) will process the records and return the JSON output. Now I have to parse the JSON and update the database for the processed flag.
I am able to get the response from the REST webservice, I have used Byte array to string transform to log in the logger.
Do I need to write foreach component to process the payload one by one and update the database?
here is my configuration XML
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:oauth2="http://www.mulesoft.org/schema/mule/oauth2" xmlns:db="http://www.mulesoft.org/schema/mule/db" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/oauth2 http://www.mulesoft.org/schema/mule/oauth2/current/mule-oauth2.xsd">
<db:mysql-config name="MySQL_Configuration" host="localhost" port="3306" user="root" database="my_db_name" doc:name="MySQL Configuration"/>
<http:request-config name="HTTP_Request_Configuration" protocol="HTTPS" host="example.net" port="8000" basePath="Salvage" doc:name="HTTP Request Configuration"/>
<flow name="cwg_clientFlow">
<poll doc:name="Poll">
<db:select config-ref="MySQL_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[SELECT * FROM cwg_ws_data WHERE SyncFlag = 0]]></db:parameterized-query>
</db:select>
</poll>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
<json:object-to-json-transformer doc:name="Object to JSON" encoding="UTF-8" mimeType="application/json"/>
<logger message="JSON Payload is #[payload]" level="INFO" doc:name="Logger"/>
<http:request config-ref="HTTP_Request_Configuration" path="/muleCWG" method="POST" doc:name="HTTP">
<http:request-builder>
<http:header headerName="access_token" value="U9P4CjhCsadIQzfJi13dHYdQLCfhmAi9OxYM8d7c"/>
</http:request-builder>
<http:success-status-code-validator values="200"/>
</http:request>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<logger message="webservice response #[payload]" level="INFO" doc:name="Logger"/>
</flow>
</mule>
Please advice.
-Paresh (kendreparesh#gmail.com)
You can either use Bulk Update with mule bulk mode = "true" with parameterized query or Batch processing. But in your case bulk update will be helpful. No need to use for-each scope it will reduce the performance.
It doesn,t matter what is the count of records Bulk update can works on thousands of records. You can try something like following
<flow name="testdbFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/test" doc:name="HTTP"/>
<set-payload value="{"1234567890123456":"Y","1234567890123457":"Y"}" mimeType="application/json" doc:name="Set Payload"/>
<dw:transform-message doc:name="Transform Message">
<dw:input-payload />
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
(payload mapObject {
x: {
key : $$,
value : $
}
}).*x]]>
</dw:set-payload>
</dw:transform-message>
<json:json-to-object-transformer returnClass="java.util.List" doc:name="JSON to Object"/>
<db:update config-ref="MySQL_Configuration" bulkMode="true" doc:name="Database">
<db:dynamic-query><![CDATA[UPDATE cwg_ws_data SET SyncFlag = '#[payload.value]' WHERE IMEI = '#[payload.key]']]></db:dynamic-query>
</db:update>
</flow>
I have tested this with same input and its working fine. FYI dataweave is used for converting json to meaningful and accessible payload.
Hope this helps.
I am using Mule 3.6 to read a CSV file that contains 100 records and convert them into a JSON array. The JSON is then posted as messages to AMQP and then inserted into Mongodb.
I can only see one record when I look at the Mongodb collection, but in debug it should the full array up to the Mongodb connector. How can I get all the objects in the JSON array saved in Mongo?
Also, I need to save 1 JSON object in the array as 1 record in Mongodb which I have struggled to get working with foreach.
XML code for my flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:context="http://www.springframework.org/schema/context"
xmlns:mongo="http://www.mulesoft.org/schema/mule/mongo" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:amqp="http://www.mulesoft.org/schema/mule/amqp" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data-mapper" xmlns:ftp="http://www.mulesoft.org/schema/mule/ee/ftp" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.6.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-current.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/ee/ftp http://www.mulesoft.org/schema/mule/ee/ftp/current/mule-ftp-ee.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/amqp http://www.mulesoft.org/schema/mule/amqp/current/mule-amqp.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/mongo http://www.mulesoft.org/schema/mule/mongo/current/mule-mongo.xsd">
<context:property-placeholder location="mule-app.properties" />
<data-mapper:config name="dept_mapper" transformationGraphPath="dept_mapper.grf" doc:name="dept_mapper"/>
<amqp:connector name="AMQP_Connector" validateConnections="true" doc:name="AMQP-0-9 Connector"/>
<mongo:config name="Mongo_DB" username="admin" database="test" doc:name="Mongo DB"/>
<flow name="read-flow">
<file:inbound-endpoint path="${file.unprocessed.location}" moveToPattern="#[message.inboundProperties['originalFilename']]" moveToDirectory="${file.processed.location}" responseTimeout="10000" mimeType="text/csv" doc:name="Files">
<file:filename-regex-filter pattern=".*.csv" caseSensitive="true"/>
</file:inbound-endpoint>
<choice doc:name="Choice">
<when expression="#[wildcard(org.mule.util.StringUtils.upperCase("dept*"), org.mule.util.StringUtils.upperCase(message.inboundProperties['originalFilename']))]">
<data-mapper:transform config-ref="dept_mapper" doc:name="dept Mapper"/>
</when>
<otherwise>
<logger level="INFO" doc:name="Logger"/>
</otherwise>
</choice>
<object-to-string-transformer doc:name="Object to String"/>
<set-variable variableName="exchangeRoutingKey" value="#[wildcard(org.mule.util.StringUtils.upperCase("dept*"), org.mule.util.StringUtils.upperCase(message.inboundProperties['originalFilename'])) ? 'depts' : (wildcard(org.mule.util.StringUtils.upperCase("sales*"), org.mule.util.StringUtils.upperCase(message.inboundProperties['originalFilename'])) ? 'sales' : 'employee')]" doc:name="Set Routing Key"/>
<amqp:outbound-endpoint exchangeName="${rabbitmq.exchange}" routingKey="#[flowVars.exchangeRoutingKey]" exchangeType="topic" responseTimeout="10000" connector-ref="AMQP_Connector" doc:name="Message Queue"/>
</flow>
<flow name="read-dept-messages-flow">
<amqp:inbound-endpoint exchangeName="${rabbitmq.exchange}" queueName="${rabbitmq.queue.depts}" routingKey="${rabbitmq.queue.routing.id.depts}" responseTimeout="10000" exchange-pattern="request-response" connector-ref="AMQP_Connector" doc:name="dept Message Queue"/>
<flow-ref name="db-processing-subflow" doc:name="db-processing-subflow"/>
</flow>
<sub-flow name="db-processing-subflow">
<object-to-string-transformer doc:name="Object to String"/>
<json:json-to-object-transformer returnClass="java.lang.Object" doc:name="JSON to Object"/>
<logger level="INFO" doc:name="Logger"/>
<mongo:json-to-dbobject doc:name="Mongo DB"/>
<mongo:insert-object config-ref="Mongo_DB" collection="depts" doc:name="Insert debt"/>
</sub-flow>
</mule>
Thanks for any help