I need to write osgi bundle to simple unmarshal edifact message (invoice), and persist invoice in database. I receive an exception during response
My environment:
- ServiceMix 5.0.0
I have following camel route
<route>
<from uri="mina2:tcp://localhost:9999?textline=true&encoding=utf-8" />
<to uri="smooks://smooks-config.xml" />
</route>
<route>
<from uri="direct:invoice" />
<process ref="invoiceProcessor" />
</route>
where smooks-config.xml is:
<?xml version="1.0"?>
<smooks-resource-list
xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd"
xmlns:edi="http://www.milyn.org/xsd/smooks/edi-1.2.xsd"
xmlns:unedifact="http://www.milyn.org/xsd/smooks/unedifact-1.4.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"
xmlns:camel="http://www.milyn.org/xsd/smooks/camel-1.4.xsd"
>
<unedifact:reader mappingModel="urn:org.milyn.edi.unedifact:d96a-mapping:1.4" ignoreNewLines="true" />
<import file="/org/milyn/edi/unedifact/d96a/message-bindingconfig.xml" />
<camel:route beanId="INVOIC">
<camel:to endpoint="direct:invoice"/>
</camel:route>
<core:exports>
<core:result type="org.milyn.payload.StringResult"/>
</core:exports>
</smooks-resource-list>
As you can see I am using standard d96a binding and mapping.
When I call netcat to send sample edifact message, all processing is going to be fine except response. During response I receive:
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.milyn.SmooksException: Failed to filter source.
at org.milyn.delivery.sax.SmooksSAXFilter.doFilter(SmooksSAXFilter.java:97)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.delivery.sax.SmooksSAXFilter.doFilter(SmooksSAXFilter.java:64)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.Smooks._filter(Smooks.java:526)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.Smooks.filterSource(Smooks.java:477)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.smooks.camel.processor.SmooksProcessor.process(SmooksProcessor.java:107)[221:org.milyn.smooks.osgi:1.5.2]
at org.apache.camel.impl.ProcessorEndpoint.onExchange(ProcessorEndpoint.java:103)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.impl.ProcessorEndpoint$1.process(ProcessorEndpoint.java:71)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:110)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:105)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:87)[98:org.apache.camel.camel-core:2.12.3]
at org.apache.camel.component.mina2.Mina2Consumer$ReceiveHandler.messageReceived(Mina2Consumer.java:339)[186:org.apache.camel.camel-mina2:2.12.3]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:690)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:417)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:47)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:765)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:74)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:769)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:761)[26:org.apache.mina.core:2.0.7]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:703)[26:org.apache.mina.core:2.0.7]
at java.lang.Thread.run(Thread.java:744)[:1.7.0_51]
Caused by: java.util.EmptyStackException
at java.util.Stack.peek(Stack.java:102)[:1.7.0_51]
at java.util.Stack.pop(Stack.java:84)[:1.7.0_51]
at org.milyn.namespace.NamespaceDeclarationStack.popNamespaces(NamespaceDeclarationStack.java:132)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.edisax.unedifact.UNEdifactInterchangeParser.parse(UNEdifactInterchangeParser.java:125)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.smooks.edi.unedifact.UNEdifactReader.parse(UNEdifactReader.java:75)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.delivery.sax.SAXParser.parse(SAXParser.java:76)[221:org.milyn.smooks.osgi:1.5.2]
at org.milyn.delivery.sax.SmooksSAXFilter.doFilter(SmooksSAXFilter.java:86)[221:org.milyn.smooks.osgi:1.5.2]
... 27 more
Im new with smooks and i dont know the source of the problem, but you can change your camel route for this:
<route>
<from uri="mina2:tcp://localhost:9999?textline=true&encoding=utf-8" />
<to uri="smooks://smooks-config.xml" />
<to uri="direct:invoice" />
</route>
<route>
<from uri="direct:invoice" />
<process ref="invoiceProcessor" />
</route>
And change your smooks-config.xml
<?xml version="1.0"?>
<smooks-resource-list
xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:unedifact="http://www.milyn.org/xsd/smooks/unedifact-1.4.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"
>
<unedifact:reader mappingModel="urn:org.milyn.edi.unedifact:d96a-mapping:1.4" ignoreNewLines="true" />
<core:exports>
<core:result type="org.milyn.payload.StringResult"/>
</core:exports>
</smooks-resource-list>
This is the same that they do in their example : https://github.com/smooks/smooks/tree/67b38b41a17510758ea9f00b7d6fad189916b2ed/smooks-examples/camel/camel-unedifact-to-xml
Related
Camel / CXF versions used:
<camel-version>2.18.1</camel-version>
<cxf-version>3.1.7</cxf-version>
Using Spring DSL to define routes. Sample code:
<cxf:cxfEndpoint address="{{ws.url}}" id="wsCallCxfBean"
serviceClass="com.some.package.GeneratedClassEndpoint">
<cxf:properties>
<entry key="username" value="${ws.username}" />
<entry key="password" value="${ws.password}" />
</cxf:properties>
<cxf:outInterceptors>
<ref bean="logOutbound" />
</cxf:outInterceptors>
</cxf:cxfEndpoint>
Camel Context:
<camel:errorHandler id="deadLetterErrorHandler" type="DeadLetterChannel" deadLetterUri="log:dead" />
<camelContext id="camelCxtId" trace="{{trace}}" handleFault="true" errorHandlerRef="deadLetterErrorHandler"
xmlns="http://camel.apache.org/schema/spring">
<onException id="onCContextException_Common">
<exception>java.lang.Exception</exception>
<redeliveryPolicy maximumRedeliveries="2" />
<!-- mark this as handled -->
<handled>
<constant>true</constant>
</handled>
<to id="onCContextExceptionHandling_Common" pattern="OutOnly" uri="vm:onRouteErrorOutMsgCaught" />
</onException>
.....
Route:
<route id="wsCall_RouteID" handleFault="true">
<description>Call the web service</description>
<from id="vmFrom_id" uri="vm:fromPOJOVM" />
<to id="wsCall_id" uri="cxf:bean:wsCallCxfBean" />
<onException id="onRouteException_WS">
<exception>java.lang.Exception</exception>
<redeliveryPolicy maximumRedeliveries="2" />
<handled>
<constant>true</constant>
</handled>
<to id="onRouteExceptionHandling_WS" pattern="OutOnly"
uri="vm:onRouteErrorOutMsgCaught" />
</onException>
</route>
Issue that I am facing:
Whenever exception is thrown in CXF, it is not handled by the Camel onException.
I can see the exception in the log. Sample exception from log (truncated):
WARN PhaseInterceptorChain:449 - Interceptor for {http://someaddress}WSEndpointService#
{http://someaddress}WSOperation has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: Marshalling Error: Connection timed out: connect
at org.apache.cxf.jaxb.JAXBEncoderDecoder.marshall(JAXBEncoderDecoder.java:266)
...
Caused by: javax.xml.bind.MarshalException
- with linked exception:
[java.net.ConnectException: Connection timed out: connect]
at com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:328)
... 20 more
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
...
at com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:323)
I have tried adding handleFault="true" in my camel context and Route - but that did not help.
How can I catch the exception thrown in CXF (for eg: the timeout, unknown host etc) in my camel route?
Thanks
I'm try to convert a application/json from rest service to a list of POJO.
But I can't
My input is a list of Event in camel:put I cannot refer a list.
<dataFormats>
<json id="eventJsonList" prettyPrint="true" library="Jackson" useList="true"
unmarshalTypeName="example.model.Event" />
</dataFormats>
<camel:rest path="events" consumes="application/json" produces="application/json">
<camel:put
uri="/save">
<to uri="direct:save-events" />
</camel:put>
</camel:rest>
using Camel 2.22
I think you need to use camel:unmarshall to tell the route to use your data format.
<camel:rest path="events" consumes="application/json" produces="application/json">
<camel:put uri="/save">
<camel:route>
<camel:unmarshal ref="eventJsonList" />
<camel:to uri="direct:save-events" />
</camel:route>
</camel:put>
</camel:rest>
Property bindingMode="off" must be disabled and the body converted to String:
<dataFormats>
<json id="eventJsonList" prettyPrint="true" library="Jackson" useList="true"
unmarshalTypeName="example.model.Event" />
</dataFormats>
<camel:rest path="events" bindingMode="off" consumes="application/json" produces="application/json">
<camel:put
uri="/save">
<to uri="direct:save-events" />
</camel:put>
</camel:rest>
<camel:route id="save-events">
<from uri="direct:save-events" />
<camel:convertBodyTo type="java.lang.String"/>
<camel:unmarshal ref="eventJsonList"/>
<to uri="mock:result"/>
</camel:route>
I want to set timeout for Apache Camel CXF consumer component.
My code looks something like:
<route>
<from uri="direct:NDS/getUserInformation" />
<to uri="freemarker:file:/application/DT/adapter/NDSLookupService.ftl" />
<bean ref="ndsServiceLogger" method="logNDSRequest" />
<setHeader headerName="SOAPAction">
<simple>getLookUpServiceDetails</simple>
</setHeader>
<bean ref="ndsServiceLogger" method="logNDSServiceStartTime" />
<toD uri="${headers.nds_url}?wsdlURL=/application/DT/adapter/NDSLookupService.wsdl&serviceName={http://webservices.lookup.sdp.bharti.ibm.com}NDSLookupServiceService&portName={http://webservices.lookup.sdp.bharti.ibm.com}NDSLookupService&dataFormat=MESSAGE" />
<bean ref="ndsServiceLogger" method="logNDSServiceEndTime" />
<bean ref="ndsServiceLogger" method="logNDSResponse" />
<convertBodyTo type="java.lang.String" />
</route>
How about to try properties http.connection.timeout and
http.receive.timeout?
UPD: it does not work... I can just give you ideas where to go...
I use CXF bus for endpoints and Asynchronous transport (doc at Asynchronous Client HTTP Transport .
It is possible to set timeouts there like below.
<cxf-core:bus bus="bus-common-outbound">
<cxf-core:properties>
<spring:entry key="use.async.http.conduit" value="true" />
<spring:entry key="org.apache.cxf.transport.http.async.usePolicy" value="ALWAYS" />
<spring:entry key="org.apache.cxf.transport.http.async.SO_TIMEOUT" value="45000" />
</cxf-core:properties>
</cxf-core:bus>
and endpoint has property "bus" pointed to that by its "bus" property like: bus=bus-common-outbound
maybe you can use something like that or dig deeper for default synchronous transport...
My xml is given below
<camelContext trace="false" xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder id="placeholder" location="classpath:application.properties" />
<!--Route:1 for POLLUX Data Processing -->
<route id="processPolluxData-Route" startupOrder="1">
<from uri="{{POLLUX_INPUT_PATH}}?noop=true"/>
<unmarshal ref="csvBindyDataformatForPolluxData"/>
<camel:bean ref="polluxDataController" method="processPolluxData"/>
<camel:log message="Line:${body}" loggingLevel="INFO"/>
<to uri="sqlComponent:{{sql.insertPolluxData}}?batch=true" />
</route>
<!-- Route:2 for RSI Data Processing -->
<route id="processRsiData-Route" startupOrder="2">
<from uri="{{RSI_INPUT_PATH}}?noop=true"/>
<unmarshal ref="csvBindyDataformatForRsiData"/>
<camel:bean ref="rsiDataController" method="processRsiData"/>
<camel:log message="Line:${body}" loggingLevel="INFO"/>
<to uri="sqlComponent:{{sql.insertRsiData}}?batch=true" />
</route>
<!-- Route for Global Data Processing -->
<route id="processGlobalData-Route" >
<from uri="sqlComponent:{{sql.selectOrder}}?consumer.useIterator=false" />
<camel:bean ref="globalDataController" method="processGlobalData" />
<marshal>
<csv delimiter=","/>
</marshal>
<log message="${body}" />
<setHeader headerName="camelFilename">
<constant>result.csv</constant>
</setHeader>
<to uri="{{GLOBAL_OUTPUT_PATH}}?fileExist=Append" />
</route>
My sql statement is
sql.selectOrder=select STID,CLLTR,SOURCE from GSI_DEVL.POLLUX_DATA
bean class for processing result set is
public class GlobalDataController {
List<Map<String, Object>> globalStationProccessedList = new ArrayList<Map<String, Object>>();
List<Map<String, Object>> globalStationMap = new ArrayList<Map<String, Object>>();
#SuppressWarnings("unchecked")
public List<Map<String, Object>> processGlobalData(Exchange exchange) throws Exception {
// System.out.println("Processing " + exchange.getIn().getBody());
globalStationMap = (List<Map<String, Object>>) exchange.getIn().getBody();
globalStationProccessedList.addAll(globalStationMap);
return globalStationProccessedList;
}
}
Problem now is Route 1 data is transffered to csv file with exact number of rows in database.But no data in the route 2 is append to the csv file
I am using camel 2.16
If the problem only in large number of files (not in the file format), then here is the solution:
<route id="processOrder-route">
<from uri="sqlComponent:{{sql.selectOrder}}"/>
<camel:bean ref="controllerformarshalling" method="processGlobalData" />
<marshal >
<csv delimiter="," useMaps="true" > </csv>
</marshal>
<log message="${body}"/>
<setHeader headerName="CamelFileName">
<constant>result.csv</constant>
</setHeader>
<to uri="file://D://cameltest//output&fileExist=Append" />
</route>
For next pool you can set another file name, based on current time, maybe.
Have you tried setting this parameter on the sql component?
consumer.useIterator boolean true
Camel 2.11: SQL consumer only: If true each row returned when
polling will be processed individually. If false the entire
java.util.List of data is set as the IN body.
Try setting this to false. That way you should get the entire sql resultset into a single list and then write the entire list to a single file.
I have the following routes:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<threadPoolProfile id="defaultProfile"
defaultProfile="true" poolSize="100" maxPoolSize="200" />
<route>
<from uri="amq:example.MyQueue" />
<setHeader headerName="myRoutingSlipHeader">
<constant>amq:one#amq:two#amq:three#amq:four</constant>
</setHeader>
<log message="Makan" />
<setExchangePattern pattern="InOut" />
<routingSlip uriDelimiter="#">
<header>myRoutingSlipHeader</header>
</routingSlip>
<setExchangePattern pattern="InOnly" />
<log message="End: ${body}" />
</route>
<route>
<from uri="amq:one" />
<to uri="bean:helloBean?method=stepOne" />
</route>
<route>
<from uri="amq:two" />
<to uri="bean:helloBean?method=stepTwo" />
</route>
<route>
<from uri="amq:three" />
<to uri="bean:helloBean?method=stepThree" />
</route>
<route>
<from uri="amq:four" />
<to uri="bean:helloBean?method=stepFour" />
</route>
</camelContext>
<bean id="amq" class="org.apache.activemq.camel.component.ActiveMQComponent"
p:brokerURL="tcp://localhost:61616" p:transacted="true"
p:cacheLevelName="CACHE_CONSUMER" p:concurrentConsumers="20"
p:maxConcurrentConsumers="500" p:idleConsumerLimit="10"
/>
Given that example.MyQueue is preloaded with 1000 messages, and each hello bean's step* methods takes 250ms, when I do camel:run, the performance is still bad. It prints "End: ..." each 1 secs in sequence not parallel. What would be the problem here?
In the following much simple case, I see a strange behavior. When there is no JMS producer putting messages into the queue, the printings happen in sequence. But when there is, the printings happen in parallel. What's the explanation?
<threadPoolProfile id="defaultProfile"
defaultProfile="true" poolSize="100" maxPoolSize="200" />
<route>
<from uri="amq:example.MyQueue" />
<delay>
<constant>1000</constant>
</delay>
<log message="End: ${body}" />
</route>
<bean id="amq" class="org.apache.activemq.camel.component.ActiveMQComponent"
p:brokerURL="tcp://localhost:61616" p:transacted="true"
p:cacheLevelName="CACHE_CONSUMER" p:concurrentConsumers="20"
p:maxConcurrentConsumers="500" p:idleConsumerLimit="10"
/>
Try replacing
<from uri="amq:example.MyQueue" />
with
<from uri="amq:example.MyQueue?concurrentConsumers=200&maxConcurrentConsumers=500" />
The routing slip runs in sequence, and yo do request/reply over JMS (eg the MEP is InOut) so processing one message would take
call amq:one = 250 millis (request/reply)
call amq:two = 250 millis (request/reply)
call amq:three = 250 millis (request/reply)
call amq:four = 250 millis (request/reply)
A total of 1 sec per message.
The AMQ route in < from > could process messages in parallel though. But each message would still take 1 second to process.
I guess routingSlip pattern is synchronized. you need something asynchronized component to handle this. Please check this:http://camel.apache.org/async.html
Just one question, why you need set the ExchangePattern?