Camel , catch error and move file in different folder - apache-camel

I am trying to doing ETL job (read,convert, store an excel) .
This route work fine :
from("file:src/data?move=processed&moveFailed=error&idempotent=true")
.bean(ExcelTransformer.class,"process")
.to("jpa:cie.service.receiver.DatiTelefonici?entityType=java.util.ArrayList");
but I need a customization in case of Exception : I need to move file in other folder , then applied :
onException(Throwable.class).maximumRedeliveries(0).to("file:src/data?move=error");
But file component can't move the file because is locket by first file comp instance.
Then I am tryinf to use try/catch but it doesn't work (probably the move operation inside catch is unaware of correct file name ? )
from("file:src/data?noop=true")
.doTry()
.bean(ExcelTransformer.class,"process")
.to("jpa:cie.service.receiver.DatiTelefonici?entityType=java.util.ArrayList")
.to("file:src/data?move=processed")
.doCatch(Throwable.class)
.to("file:src/data?move=error")
.end();
tanks
After many comments my current code looks like :
from("file:src/data?noop=false&delete=true")
.doTry()
.bean(ExcelTransformer.class,"process") .to("jpa:cie.service.receiver.DatiTelefonici?entityType=java.util.ArrayList")
.to("file:src/data/processed")
.doCatch(Throwable.class)
.to("file:src/data/error")
/*
.doFinally()
.to("file:src/data:delete=true")
*/
.end();
It move correctly the file in processed and error folder but the file remain in main folder and is preocessed more ,recursively

If I understood your question well then you need remove the idempotent=true from the parameters, then it should work:
from("file:src/data?move=processed&moveFailed=error")
.bean(ExcelTransformer.class,"process")
.to("jpa:cie.service.receiver.DatiTelefonici?entityType=java.util.ArrayList");
The previous route moves the file to the processed folder if the routing was successful otherwise it moves the file to the error folder (if any exception happens). The filename won't be changed.
Other solution with try-catch
from("file://src/data?delete=true")
.doTry()
.bean(ExcelTransformer.class,"process")
.to("jpa:cie.service.receiver.DatiTelefonici?entityType=java.util.ArrayList")
.to("file://src/data/processed")
.doCatch(Throwable.class)
.to("file://src/data/error")
.end();

Related

Execute route only when direct component is called

I want to unzip files whenever a "direct" route is called. The fileName I am getting from some other direct route.
from("direct:unZipFile")
.from("file:C:\\MYFILES\\File\\Unzipped\\?fileName=${header.fileName}&idempotent=true")
.split(new ZipSplitter())
.streaming()
.to("file:C:\\MYFILES\\File\\Unzipped\\")
.split(body().convertToString().tokenize("\n"))
.transform()
.simple("${in.body}")
.end();
Now it is working when I call the direct component but it also keeps on scanning the directory and processes the same file. I understand that the above code allows the trigger from direct as well as file component but I just want it to execute from "direct" component and I can't remove the "file" component as from that only I am reading the file.
You can also try to use pollenrich
from("direct:unZipFile")
.pollEnrich.simple("file:C:\\MYFILES\\File\\Unzipped\\?fileName=${header.fileName}")
.split(new ZipSplitter())
.streaming()
...
You can use consumerTemplate
from("direct:unZipFile")
.process(exchange -> {
Exchange recvFiles = exchange.getContext().createConsumerTemplate().receive("file:C:\\MYFILES\\File\\Unzipped\\?fileName=${header.fileName}&idempotent=true");
exchange.getIn().setBody(recvFiles.getIn().getBody());
exchange.getIn().getHeaders().putAll(recvFiles.getIn().getHeaders());
})
.split(new ZipSplitter())
.streaming()
.to("file:C:\\MYFILES\\File\\Unzipped\\")
.split(body().convertToString().tokenize("\n"))
.transform()
.simple("${in.body}")
.end();
You can also just set the message body with a java.io.File where you compute the file name in a few lines of Java code, but yeah otherwise the pollEnrich is the EIP pattern solution for this.

Apache Camel route with no "to" endpoint

I am using Apache Camel to assist with capturing message data emitted by a third party software package. In this particular instance, I only need to capture what is produced by the software, there is no receiver on the other end (really no "end" to go to).
So, I tried to set up a route with just the "from" endpoint and no "to" endpoint. Apparently this is incorrect usage as I received the following exception:
[2018-08-15 11:08:03.205] ERROR: string.Launcher:191 - Exception
org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> From[mina:udp://localhost:9877?sync=false] <<< in route: Route(route1)[[From[mina:udp://localhost:9877?sync=false]] -... because of Route route1 has no output processors. You need to add outputs to the route such as to("log:foo").
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:1063)
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:196)
at org.apache.camel.impl.DefaultCamelContext.startRoute(DefaultCamelContext.java:974)
at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:3301)
at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3024)
at org.apache.camel.impl.DefaultCamelContext.access$000(DefaultCamelContext.java:175)
at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:2854)
at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:2850)
at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:2873)
at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:2850)
at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:2819)
at {removed}.Launcher.startCamel(Launcher.java:189)
at {removed}.Launcher.main(Launcher.java:125)
Caused by: java.lang.IllegalArgumentException: Route route1 has no output processors. You need to add outputs to the route such as to("log:foo").
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:1061)
... 13 more
How do I set up a camel route that allows me to intercept (capture) the message traffic coming from the source, and not send it "to" anything? There is no need for a receiver. What would be an appropriate "to" endpoint that just drops everything it receives?
The exception suggestion of to("log:foo"). What does this do?
You can see if the Stub component can help
http://camel.apache.org/stub.html
Example:
from("...")
.to("stub:nowhere");
The exception suggestion of to("log:foo"). What does this do?
It sends your route messages to an endpoint with a component of type log:
(http://camel.apache.org/log.html) - component which basically dumps message contents (body and/or headers and/or properties) to your log file using appropriate log category.
If you just want to drop everything received, it's a good choice:
to("log:com.company.camel.sample?level=TRACE&showAll=true&multiline=true")
Apparently if you're under Linux or Unix, you can also redirect to /dev/null like in this example:
to( "file:/dev?fileName=null")
I am not sure it can be used on Windows but I don't think so.
Note that the syntax: to( "file:/dev/null") does not work as it point to a directory called null but with the fileName option it will work.

How move file in the event of catch exception (Mule 3.2 )

In my principal flow move file from "IN" to "BackUp" but in case occurs exception I want move the file from (I don't know
) to "Error".
How move file from "my cath exception" to "Error" in Mule 3.2?
Use catch exception strategy and use a file outbound endpoint there.
You should be able to send the file to the desired location
<flow>
<inbound-endpoint/><!-- Fill in your details for this component -->
<outbound-endpoint/><!-- Fill in your details for this component -->
<catch-exception-strategy>
<logger message="#[payload]" level="ERROR' />
<outbound-endpoint/><!-- Fill in your details for this component -->
</catch-exception-strategy>
<flow>
On any exception that happens on your flow will be captured in teh exception strategy and the logic or flow you write there will be executed. In this case the paylod is written to a file in the folder that you configure within
</catch-exception-strategy>

Camel SFTP Route fails to continue onException

I've fairly simple looking route
sftp://hostname:22//incoming/folder/location/?username=username&password=xxxxx
&localWorkDirectory=/tmp&readLock=changed&readLockCheckInterval=2000
&move=processed/$simple{date:now:yyyy}/$simple{date:now:MM}/$simple{date:now:dd}${file:name}
&consumer.delay=450000&stepwise=false&streamDownload=true&disconnect=true
I also have an onException clause
onException(ValidationException.class)
.handled(true)
.logStackTrace(true)
.filter(header("VALIDATION_ERROR").isEqualTo(true))
.choice()
.when(header("CamelFileName").contains("Param1"))
.to(sftp://hostname:22//One/error/folder?password=xxxxxx&username=username)
.when(header("CamelFileName").contains("Param2"))
.to(sftp://hostname:22//Two/error/folder?password=xxxxxx&username=username)
.endChoice();
When I have single file, the route seems to work as expected. When more than one file and exception occurs, I get many different exceptions like
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot list directory: incoming/folder/location
Caused by: java.lang.IndexOutOfBoundsException
I tried using all the attributes mentioned in the route viz. streamDownload, stepwise, readLock, localWorkDirectory et al. However, the error handling while multiple files is not working. I see the first file getting processed. However, it doesn't move to the processed folder once the exception occurs and then incoming/folder/location becomes non listable. I tried using continued(true) as well instead of handled(true)
The problem was with multiple files being handled in the same exchange. On exception, the route was trying to FTP back the error file on the same server. The solution was to split body into multiple exchanges so that each file has its own exchange and process them separately.
from(sftp://hostname:22//incoming/folder/location/?username=username&password=xxxxx
&localWorkDirectory=/tmp&readLock=changed&disconnect=true&stepwise=false
&move=processed/$simple{date:now:yyyy}/$simple{date:now:MM}/$simple{date:now:dd}${file:name}
&consumer.delay=450000).split(body()).processRef("incomingProcessor").end();

Camel routing issue

I have created some routes. Following is the code which is having issues.
Following is the expected behavior:
Exchange at first gets processed at hourlyFeedParts queue and then passed to dailyProcessor.
In dailyProcessor a property currHour is being checked if it is 23 or not. If not, it just passes on.
If currHour == 23, code inside it shall be processed. This part again has following functionality,
If property feedsleft is not zero, all code inside the choice currHour==23 is executed. This is fine.
if property feedsLeft is zero, code inside it processed. Code within looks for any further messages. If yes, they are send to hourlyFeedParts. Here comes the issue: If there is any message to be processed the code beyond to("direct:hourlyFeedParts") is not executed. Though, if nothing is returned, the code works fine.
I guess the issue could be code ends at to. So, what shall be the alternative?
from("direct:dailyProcessor")
.choice()
.when(simple("${property.currHour} == 23"))
.choice()
.when(simple("${property.feedsLeft} == 0"))
.split(beanExpression(APNProcessor.class, "recheckFeeds"))
.to("direct:hourlyFeedParts")
.endChoice()
.end()
.split(beanExpression(new S3FileKeyProcessorFactory(), "setAPNS3Header"))
.parallelProcessing()
.id("APN Daily PreProcessor / S3 key generator ")
.log("Uploading file ${file:name}")
.to("{{apn.destination}}")
.id("APN Daily S3 > uploader")
.log("Uploaded file ${file:name} to S3")
.endChoice()
.end()
I believe that the issue is the nested choice().
try to extract the inner choice to a seperate route e.g.:
from("direct:dailyProcessor")
.choice()
.when(simple("${property.currHour} == 23"))
.to("direct:inner")
.split(beanExpression(new S3FileKeyProcessorFactory(), "setAPNS3Header"))
.parallelProcessing()
.id("APN Daily PreProcessor / S3 key generator ")
.log("Uploading file ${file:name}")
.to("{{apn.destination}}")
.id("APN Daily S3 > uploader")
.log("Uploaded file ${file:name} to S3");
from("direct:inner")
.choice()
.when(simple("${property.feedsLeft} == 0"))
.split(beanExpression(APNProcessor.class, "recheckFeeds"))
.to("direct:hourlyFeedParts");
I haven't tested it, but I guess you get the point.

Resources