Camel rest JSON deserialize with START_ARRAY - apache-camel

I'm try to convert a application/json from rest service to a list of POJO.
But I can't
My input is a list of Event in camel:put I cannot refer a list.
<dataFormats>
<json id="eventJsonList" prettyPrint="true" library="Jackson" useList="true"
unmarshalTypeName="example.model.Event" />
</dataFormats>
<camel:rest path="events" consumes="application/json" produces="application/json">
<camel:put
uri="/save">
<to uri="direct:save-events" />
</camel:put>
</camel:rest>
using Camel 2.22

I think you need to use camel:unmarshall to tell the route to use your data format.
<camel:rest path="events" consumes="application/json" produces="application/json">
<camel:put uri="/save">
<camel:route>
<camel:unmarshal ref="eventJsonList" />
<camel:to uri="direct:save-events" />
</camel:route>
</camel:put>
</camel:rest>

Property bindingMode="off" must be disabled and the body converted to String:
<dataFormats>
<json id="eventJsonList" prettyPrint="true" library="Jackson" useList="true"
unmarshalTypeName="example.model.Event" />
</dataFormats>
<camel:rest path="events" bindingMode="off" consumes="application/json" produces="application/json">
<camel:put
uri="/save">
<to uri="direct:save-events" />
</camel:put>
</camel:rest>
<camel:route id="save-events">
<from uri="direct:save-events" />
<camel:convertBodyTo type="java.lang.String"/>
<camel:unmarshal ref="eventJsonList"/>
<to uri="mock:result"/>
</camel:route>

Related

Setting timeout for Apache Camel CXF consumer component

I want to set timeout for Apache Camel CXF consumer component.
My code looks something like:
<route>
<from uri="direct:NDS/getUserInformation" />
<to uri="freemarker:file:/application/DT/adapter/NDSLookupService.ftl" />
<bean ref="ndsServiceLogger" method="logNDSRequest" />
<setHeader headerName="SOAPAction">
<simple>getLookUpServiceDetails</simple>
</setHeader>
<bean ref="ndsServiceLogger" method="logNDSServiceStartTime" />
<toD uri="${headers.nds_url}?wsdlURL=/application/DT/adapter/NDSLookupService.wsdl&serviceName={http://webservices.lookup.sdp.bharti.ibm.com}NDSLookupServiceService&portName={http://webservices.lookup.sdp.bharti.ibm.com}NDSLookupService&dataFormat=MESSAGE" />
<bean ref="ndsServiceLogger" method="logNDSServiceEndTime" />
<bean ref="ndsServiceLogger" method="logNDSResponse" />
<convertBodyTo type="java.lang.String" />
</route>
How about to try properties http.connection.timeout and
http.receive.timeout?
UPD: it does not work... I can just give you ideas where to go...
I use CXF bus for endpoints and Asynchronous transport (doc at Asynchronous Client HTTP Transport .
It is possible to set timeouts there like below.
<cxf-core:bus bus="bus-common-outbound">
<cxf-core:properties>
<spring:entry key="use.async.http.conduit" value="true" />
<spring:entry key="org.apache.cxf.transport.http.async.usePolicy" value="ALWAYS" />
<spring:entry key="org.apache.cxf.transport.http.async.SO_TIMEOUT" value="45000" />
</cxf-core:properties>
</cxf-core:bus>
and endpoint has property "bus" pointed to that by its "bus" property like: bus=bus-common-outbound
maybe you can use something like that or dig deeper for default synchronous transport...

How to add resultset to csv file in camel

My xml is given below
<camelContext trace="false" xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder id="placeholder" location="classpath:application.properties" />
<!--Route:1 for POLLUX Data Processing -->
<route id="processPolluxData-Route" startupOrder="1">
<from uri="{{POLLUX_INPUT_PATH}}?noop=true"/>
<unmarshal ref="csvBindyDataformatForPolluxData"/>
<camel:bean ref="polluxDataController" method="processPolluxData"/>
<camel:log message="Line:${body}" loggingLevel="INFO"/>
<to uri="sqlComponent:{{sql.insertPolluxData}}?batch=true" />
</route>
<!-- Route:2 for RSI Data Processing -->
<route id="processRsiData-Route" startupOrder="2">
<from uri="{{RSI_INPUT_PATH}}?noop=true"/>
<unmarshal ref="csvBindyDataformatForRsiData"/>
<camel:bean ref="rsiDataController" method="processRsiData"/>
<camel:log message="Line:${body}" loggingLevel="INFO"/>
<to uri="sqlComponent:{{sql.insertRsiData}}?batch=true" />
</route>
<!-- Route for Global Data Processing -->
<route id="processGlobalData-Route" >
<from uri="sqlComponent:{{sql.selectOrder}}?consumer.useIterator=false" />
<camel:bean ref="globalDataController" method="processGlobalData" />
<marshal>
<csv delimiter=","/>
</marshal>
<log message="${body}" />
<setHeader headerName="camelFilename">
<constant>result.csv</constant>
</setHeader>
<to uri="{{GLOBAL_OUTPUT_PATH}}?fileExist=Append" />
</route>
My sql statement is
sql.selectOrder=select STID,CLLTR,SOURCE from GSI_DEVL.POLLUX_DATA
bean class for processing result set is
public class GlobalDataController {
List<Map<String, Object>> globalStationProccessedList = new ArrayList<Map<String, Object>>();
List<Map<String, Object>> globalStationMap = new ArrayList<Map<String, Object>>();
#SuppressWarnings("unchecked")
public List<Map<String, Object>> processGlobalData(Exchange exchange) throws Exception {
// System.out.println("Processing " + exchange.getIn().getBody());
globalStationMap = (List<Map<String, Object>>) exchange.getIn().getBody();
globalStationProccessedList.addAll(globalStationMap);
return globalStationProccessedList;
}
}
Problem now is Route 1 data is transffered to csv file with exact number of rows in database.But no data in the route 2 is append to the csv file
I am using camel 2.16
If the problem only in large number of files (not in the file format), then here is the solution:
<route id="processOrder-route">
<from uri="sqlComponent:{{sql.selectOrder}}"/>
<camel:bean ref="controllerformarshalling" method="processGlobalData" />
<marshal >
<csv delimiter="," useMaps="true" > </csv>
</marshal>
<log message="${body}"/>
<setHeader headerName="CamelFileName">
<constant>result.csv</constant>
</setHeader>
<to uri="file://D://cameltest//output&fileExist=Append" />
</route>
For next pool you can set another file name, based on current time, maybe.
Have you tried setting this parameter on the sql component?
consumer.useIterator boolean true
Camel 2.11: SQL consumer only: If true each row returned when
polling will be processed individually. If false the entire
java.util.List of data is set as the IN body.
Try setting this to false. That way you should get the entire sql resultset into a single list and then write the entire list to a single file.

how to pass Map in camel context in jbpm component

look at the location JBPMComponentIntegrationTest
.setHeader(JBPMConstants.PARAMETERS, constant(map))
so them map is passed as map in java route.
If i want to pass the same via xml is there a way ?
<camel:setHeader headerName="CamelJBPMParameters">
<camel:constant>????</camel:constant>
</camel:setHeader>
I could not find any example over internet.
If you are using a spring context you could simply initialize the map then reference it in your constant file. I am not 100% familiar with the xml camel constructs but it should look similar to this:
<camelContext id="myContext" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<setHeader headerName="theHeader">
<!-- not sure if the ref keyword is valid might have to adjust syntax -->
<constant ref="maps" />
</setHeader>
<to uri="mock:result"/>
</route>
</camelContext>
<property name="maps">
<map>
<entry key="Key 1" value="1" />
<entry key="Key 2" value="2" />
<entry key="Key 3" value="3" />
</map>
</property>

Exchange edifact message with smooks - error during response

I need to write osgi bundle to simple unmarshal edifact message (invoice), and persist invoice in database. I receive an exception during response
My environment:
- ServiceMix 5.0.0
I have following camel route
<route>
<from uri="mina2:tcp://localhost:9999?textline=true&encoding=utf-8" />
<to uri="smooks://smooks-config.xml" />
</route>
<route>
<from uri="direct:invoice" />
<process ref="invoiceProcessor" />
</route>
where smooks-config.xml is:
<?xml version="1.0"?>
<smooks-resource-list
xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd"
xmlns:edi="http://www.milyn.org/xsd/smooks/edi-1.2.xsd"
xmlns:unedifact="http://www.milyn.org/xsd/smooks/unedifact-1.4.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"
xmlns:camel="http://www.milyn.org/xsd/smooks/camel-1.4.xsd"
>
<unedifact:reader mappingModel="urn:org.milyn.edi.unedifact:d96a-mapping:1.4" ignoreNewLines="true" />
<import file="/org/milyn/edi/unedifact/d96a/message-bindingconfig.xml" />
<camel:route beanId="INVOIC">
<camel:to endpoint="direct:invoice"/>
</camel:route>
<core:exports>
<core:result type="org.milyn.payload.StringResult"/>
</core:exports>
</smooks-resource-list>
As you can see I am using standard d96a binding and mapping.
When I call netcat to send sample edifact message, all processing is going to be fine except response. During response I receive:
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.milyn.SmooksException: Failed to filter source.
at org.milyn.delivery.sax.SmooksSAXFilter.doFilter(SmooksSAXFilter.java:97)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.delivery.sax.SmooksSAXFilter.doFilter(SmooksSAXFilter.java:64)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.Smooks._filter(Smooks.java:526)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.Smooks.filterSource(Smooks.java:477)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.smooks.camel.processor.SmooksProcessor.process(SmooksProcessor.java:107)[221:org.milyn.smooks.osgi:1.5.2]
at org.apache.camel.impl.ProcessorEndpoint.onExchange(ProcessorEndpoint.java:103)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.impl.ProcessorEndpoint$1.process(ProcessorEndpoint.java:71)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:110)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:105)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:87)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.component.mina2.Mina2Consumer$ReceiveHandler.messageReceived(Mina2Consumer.java:339)[186:org.apache.camel.camel-mina2:2.12.3]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:690)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:417)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:47)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:765)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:74)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:769)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:761)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:703)[26:org.apache.mina.core:2.0.7]
at java.lang.Thread.run(Thread.java:744)[:1.7.0_51]
Caused by: java.util.EmptyStackException
at java.util.Stack.peek(Stack.java:102)[:1.7.0_51]
at java.util.Stack.pop(Stack.java:84)[:1.7.0_51]
at org.milyn.namespace.NamespaceDeclarationStack.popNamespaces(NamespaceDeclarationStack.java:132)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.edisax.unedifact.UNEdifactInterchangeParser.parse(UNEdifactInterchangeParser.java:125)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.smooks.edi.unedifact.UNEdifactReader.parse(UNEdifactReader.java:75)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.delivery.sax.SAXParser.parse(SAXParser.java:76)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.delivery.sax.SmooksSAXFilter.doFilter(SmooksSAXFilter.java:86)[221:org.milyn.smooks.osgi:1.5.2]
... 27 more
Im new with smooks and i dont know the source of the problem, but you can change your camel route for this:
<route>
<from uri="mina2:tcp://localhost:9999?textline=true&encoding=utf-8" />
<to uri="smooks://smooks-config.xml" />
<to uri="direct:invoice" />
</route>
<route>
<from uri="direct:invoice" />
<process ref="invoiceProcessor" />
</route>
And change your smooks-config.xml
<?xml version="1.0"?>
<smooks-resource-list
xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:unedifact="http://www.milyn.org/xsd/smooks/unedifact-1.4.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"
>
<unedifact:reader mappingModel="urn:org.milyn.edi.unedifact:d96a-mapping:1.4" ignoreNewLines="true" />
<core:exports>
<core:result type="org.milyn.payload.StringResult"/>
</core:exports>
</smooks-resource-list>
This is the same that they do in their example : https://github.com/smooks/smooks/tree/67b38b41a17510758ea9f00b7d6fad189916b2ed/smooks-examples/camel/camel-unedifact-to-xml

How to scale Apache Camel route?

I have the following routes:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<threadPoolProfile id="defaultProfile"
defaultProfile="true" poolSize="100" maxPoolSize="200" />
<route>
<from uri="amq:example.MyQueue" />
<setHeader headerName="myRoutingSlipHeader">
<constant>amq:one#amq:two#amq:three#amq:four</constant>
</setHeader>
<log message="Makan" />
<setExchangePattern pattern="InOut" />
<routingSlip uriDelimiter="#">
<header>myRoutingSlipHeader</header>
</routingSlip>
<setExchangePattern pattern="InOnly" />
<log message="End: ${body}" />
</route>
<route>
<from uri="amq:one" />
<to uri="bean:helloBean?method=stepOne" />
</route>
<route>
<from uri="amq:two" />
<to uri="bean:helloBean?method=stepTwo" />
</route>
<route>
<from uri="amq:three" />
<to uri="bean:helloBean?method=stepThree" />
</route>
<route>
<from uri="amq:four" />
<to uri="bean:helloBean?method=stepFour" />
</route>
</camelContext>
<bean id="amq" class="org.apache.activemq.camel.component.ActiveMQComponent"
p:brokerURL="tcp://localhost:61616" p:transacted="true"
p:cacheLevelName="CACHE_CONSUMER" p:concurrentConsumers="20"
p:maxConcurrentConsumers="500" p:idleConsumerLimit="10"
/>
Given that example.MyQueue is preloaded with 1000 messages, and each hello bean's step* methods takes 250ms, when I do camel:run, the performance is still bad. It prints "End: ..." each 1 secs in sequence not parallel. What would be the problem here?
In the following much simple case, I see a strange behavior. When there is no JMS producer putting messages into the queue, the printings happen in sequence. But when there is, the printings happen in parallel. What's the explanation?
<threadPoolProfile id="defaultProfile"
defaultProfile="true" poolSize="100" maxPoolSize="200" />
<route>
<from uri="amq:example.MyQueue" />
<delay>
<constant>1000</constant>
</delay>
<log message="End: ${body}" />
</route>
<bean id="amq" class="org.apache.activemq.camel.component.ActiveMQComponent"
p:brokerURL="tcp://localhost:61616" p:transacted="true"
p:cacheLevelName="CACHE_CONSUMER" p:concurrentConsumers="20"
p:maxConcurrentConsumers="500" p:idleConsumerLimit="10"
/>
Try replacing
<from uri="amq:example.MyQueue" />
with
<from uri="amq:example.MyQueue?concurrentConsumers=200&maxConcurrentConsumers=500" />
The routing slip runs in sequence, and yo do request/reply over JMS (eg the MEP is InOut) so processing one message would take
call amq:one = 250 millis (request/reply)
call amq:two = 250 millis (request/reply)
call amq:three = 250 millis (request/reply)
call amq:four = 250 millis (request/reply)
A total of 1 sec per message.
The AMQ route in < from > could process messages in parallel though. But each message would still take 1 second to process.
I guess routingSlip pattern is synchronized. you need something asynchronized component to handle this. Please check this:http://camel.apache.org/async.html
Just one question, why you need set the ExchangePattern?

Resources