I have a route which looks like this, the first route reads from the file
from("file:verylargefile.csv")
.multicast()
.parallelProcessing()
.to(directEndpoint)).end()
from(directEndpoint)
.routeId(routeId)
.routePolicy(routePolicy)
.process(proxyFieldProcessor)
.marshal().json(JsonLibrary.Jackson)
.to(http-endpoint);
I need to pause the file processing if there are too many bad responses from rest endpoints. i am using custom route policy on the second route, this does suspend the second route but the first route keeps sending messages to second route which results in org.apache.camel.component.direct.DirectConsumerNotAvailableException
is there any other way to pause file processing in camel?
One approach to exactly do that can be to implement an error handler in the 2nd route, that uses the ControlBus component to suspend the 1st one:
controlbus:route?routeId=foo&action=suspend
Related
I'm using Camel to implement the saga pattern using its Saga EIP, one of my requirements is having the route body in the compensation routes, to achieve this the documentation advises to use the option method:
from("direct:example")
.routeId("example")
.enrich("direct:getData", dataAggregationStrategy)
.process(somethingProcessor)
.setHeader("Body", body())
.saga()
.option("OptionBody", body())
.completion("direct:exampleDone")
.compensation("direct:exampleCompensated")
.to("direct:doSomething");
In my code I'm doing some enrich and process steps to manipulate the body and all is well in the routes called by to which receive the correct processed body.
In the compensation routes though I'm receiving an header OptionBody containing the original body passed to the direct:example route and not the processed one.
I tried to set an header with setHeader and the body is the correct version (the processed one).
I'm not sure why this is happening, is it because of lazy evaluation of the expressions?
I have requirement to poll two ftp folders say "output" and "error" for file. Actual file could come in either of two folders(not both). I tried multicast (even used completionSize of 1) but process keeps waiting as camel waits for both output and error endpoint routes to complete.
from("direct:got_filename_to_poll")
.multicast()
.parallelProcessing(true)
.to("ftp:output", "ftp:error")
.end()
.to("direct:process_extracted_file")
Is there any way to interrupt "ftp:error" sub-route if I get response from "ftp:output" sub-route(or vice versa) or is there any other option to solve this problem without compromising response time, example adding timeout on slow response will slow down overall response time.
I want to write a camel route which will take input from multiple file destination and process them after aggregating.
is it possible to take input from multiple files for a single route?
Yes you can use poll-enrich to call consumer-endpoints like file to enrich the message. This works for many other consumer-endpoints as well like SFTP or message queues.
If you need to read same file multiple times it can get trickier as you'll likely have to set noop=true and possibly use something like dummy idempotent repository to get around camels default behavior.
Note that calling pollEnrich seems to clear headers / create new message so use exchange properties to persist data between pollEnrich calls.
from("file:someDirectory")
.setProperty("file1").body()
.pollEnrich("file:otherDirectory", 3000)
.setProperty("file2").body()
.pollEnrich("file:yetAnotherDirectory", 3000)
.setProperty("file3").body();
I have an issue consuming a Chunked HTTP RSVP Stream using Camel.
The stream is here and you can find more information about it at the end of this page
I have created a simple route such:
from("http4://stream.meetup.com/2/rsvps").log(org.apache.camel.LoggingLevel.INFO, "MeetupElasticSearch.Log", "JSON RSVP ${body}")
But nothing happens no message are consumed. I tried by adding a camel timer before because I am not sure you can use http4 component directly in the from but result is the same.
Can you help me please?
You cannot use the http4 component in a from() directive.
However, there is several ways to call an URL and to retrieve the result:
One is to create a timer and call the http4 component from a to() like this:
from("quartz2://HttpPoll/myTimer?trigger.repeatCount=0")
.to("http4://stream.meetup.com/2/rsvps")
.log("Body is: ${body}")
.to(mockDestination);
Another way is to use the content enricher pattern if you want to for example aggegates results into one another stucture.
Note as well, if your URL is dynamic you must use recipientList() instead to().
UPDATE:
Another way to consume a stream from the Internet is to use another component than http4: the camel-stream.
This one you can declare it in a from()directive:
// Read stream from the given URL... an publish in a queue
from("stream:url?url=http://stream.meetup.com/2/rsvps&scanStream=true&retry=true").
to("seda:processingQueue");
// Read records from the processing queue
// and do something with them.
from("seda:processingQueue").to("log:out")
You can see more sample in this site.
What I want to be able to do is following:
from(...)
.replyWith()
.from(...)
.end()
So that the response to my producer is taken from the consumption of another endpoint, an example would be something like a REST endpoint for a queue.
Is there an idiomatic way in Camel to be able to do something like the above without grabbing an Endpoint from a CamelContext instance and manually retrieving the contents and setting them into the Exchange?
Can you explain a bit more?
You dont want to just do
from A
to B
from B
to C
So that a message send to A will be send to B. And B is processed in another route, and the response from this will be send back to the first route, which will be used as reply to any client invoking A in the first place.
Also if you want something with dynamic endpoints, then you can use the Recipient List EIP pattern
http://camel.apache.org/recipient-list.html
from("http://0.0.0.0:9001/getResultsFromQueue")
.pollEnrich("activemq:queue:myQueue")
.to("log:test?level=DEBUG");