I have about 10K messages in a CSV file. Each message has an associated timestamp for it. When the time is reached, I want the message delivered to an MQ. The timestamps are not uniformly spread. Is this possible with Apache Camel?
As far as I know Apache Camel by default has no consumer endpoint components that you could configure to trigger with specific messages at specified times.
There is however a timer component that you can setup to trigger for example once a second. Then in the route you could use a processor to check if a list contains any messages that should be send at the given time and send them to the MQ.
You could also trigger route from java code using a ProducerTemplate that you can create using the CamelContext.
The list could be populated using your csv file and ordered by timestamp so you could use it like a stack and only check the first few entries instead of going through all 10K every second.
The real problem here would be persistence i.e to figure out which of the messages listed on the csv have already been sent if the application closes before all 10K messages have been sent.
Related
I am using Apache camel to trigger various actions based on timer events. For timer events to trigger, I am using camel-quartz. And a sample trigger looks like this:
from("quartz://job_timers/customer_activation?cron=0+0+0+*+*+?+*&trigger.timeZone=America/Chicago")
.routeId("TrigCustActivation")
.log(LoggingLevel.INFO, "Triggered Customer Activation Job")
.to("direct://set/headers/activation")
.to("direct://customer/activation");
Over the period of time, the number of such triggers have increased a lot. And they will be increasing even more. So, instead of keeping a hard-copy of when which job will trigger, I was planning to create a web-route, so that when I perform a GET request, it would fetch all the active routes, and print their RouteIds (I will update all the RouteIds to have time as well. Say, the routeId for above flow will become "TrigCustActivation_00_00_CST_Daily").
I was able to print all the route at application startup, but failed to fetch those dynamically through a GET request.
Can we access other RouteIds from a running route? Is my approach feasible?
Yes there is API on CamelContext where you can fetch all the current routes, and you can then get their IDs etc. See the javadoc of this class.
http://static.javadoc.io/org.apache.camel/camel-core/2.21.0/org/apache/camel/CamelContext.html
I am trying to get time taken for a message just to be delivered to the IBM MQ queues in Apache Camel. Below is the route configuration,
from("direct:deliver-route-2").process("mySampleProcessor").split().method("messageSplitterBean", "splitMessage").shareUnitOfWork().stopOnException().toD("mqDeliverJms:${headers.Deliver}?preserveMessageQos=true");
I have a splitter in place to split the list of messages and dynamically deliver (toD) it using the messages header "Deliver".
I used the MessageHistory suggestion given in one of the questions, but then I don't see the elapsed time for the (toD) delivery of the messages to IBM MQ.
String MessageHist = org.apache.camel.util.MessageHelper.dumpMessageHistoryStacktrace(exchange,
new DefaultExchangeFormatter(), false);
Here is the message history,
You can use the 'EventNotifier' to capture the ExchangeSent events where you have the time, eg there is little example in the Camel docs at: http://camel.apache.org/eventnotifier-to-log-details-about-all-sent-exchanges.html
I’m stuck at one particular requirement in camel , I’m processing a set of files and when any exception occurs i send a message to an exception queue , and from the exception queue im sending an email about the failure , the functionality is working fine but I end up sending many mails, like if 10 files fail im sending 10 emails, is there a way to send only one mail , like I would want to wait for the entire route to finish , then go look the exception queue and send a single mail stating what has failed (by processing the exception queue )
I'm open for suggestions.
I had to do this scenario once (inversed though - mail on success). I had a handy MySQL database configured and ready, so I just added each event from the queue to the database. Then once every now and then, extracted all info (and deleted it)- simply select * from events; delete from events; from the database and created a mail.
You could process the error queue with the aggregator pattern, it is very nice for these tasks. http://camel.apache.org/aggregate-example.html . You still need to know WHEN the aggregator should fire off a message. If you can, trigger a "finish, send mail" event such as in the example in the link above.
The most simple way would be to time schedule these mail notifications. Take a look at: http://camel.apache.org/simplescheduledroutepolicy.html . You can set it to run your route for some good choice of time, then when it fires off, you set the aggreator to complete upon timeout, and make the timeout good enough to empty any reasonable queue size of errors, but not too large.
At least that's my suggestions to your issue
I'm writing an application that synchronize emails (inbox only) from an IMAP server. For that I'm using javamail and I have performance issue. When I want to refresh my emails, I fetch all messages in inbox, and it takes several minutes. :(
So I would like to fetch only messages that were modified since the last time I refreshed emails. I found how to fetch messages received or sent since a date, but what I want to do is slightly different. It is possible to change the state of a very old message (unread to read). In this case the modification date is recent but received or send date is old.
Any idea ?
Regards,
Quentin
You can't change the content of an old message, but you can change the flags. You can fetch all the flags for all messages and compare them with your cached copy of the flags. (There are IMAP extensions that help with this, but many IMAP servers don't support them and JavaMail doesn't support them.)
Use the Folder.fetch method to fetch all the flags in one operation, then iterate over the Message objects and compare the flags.
How is the best way to handle messages through a server to multiple devices?
Scenario
It will be an app capable of running on multiple mobile platforms including online in a web browser. A type of instant messenger. The messages will be directed through a server to another mobile device.
The back-end structure/concept must be basically the same as WhatsApp. Sending messages to one-another like that.
What I think
Have the device send it to the web-server.
Server saves it in a queue table in a database.
When receiver device checks for new message (every second) it finds it in the queue.
Remove it from queue and put message in history table.
Final
What would be a efficient way to structure/handle such an app to get similar results as WhatsApp?
You may want to push messages instead of pull them every second. This has two big advantages:
Less bandwidth usage.
You can skip the database part if the sender and the receiver are both connected when the message is sent. Only queue the messages in the database if the receiver isn't connected.
So it's a huge performance boost if you use push.
If you have a web app using JavaScript you can use a JSON stream or, for new browsers, JavaScript WebSokets.