I'm trying to post a large message (JSON Format with +210k characters) on a Artemis Queue trough a endpoint Rest with Camel.
When I add a Camel Component with ActiveMQ Connection Factory org.apache.activemq.ActiveMQConnectionFactory - Version: 5.15.6) I'm able to post the message with success.
But when I use the Artemis Connection Factory ( org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory - Version: 2.6.2) the entire message isn't post in the Queue. The message was cut off reaming only 106725 characters.
Repository where I've create the examples: https://github.com/vitorvr/camel-amq
Thanks.
You should check out this Artemis documentation page for large message support in Artemis.
There is an attribute minLargeMessageSize that is by default 100 kB (roughly the remaining message size you mention). That setting means that Artemis treats all messages that are more than 100 kB as large message and therefore handles them differently.
Related
I am using camel version 2.24.1. I am reading a CSV and sending those records to the HTTP endpoint with the jetty client. the problem is on hawtio when I look at how many exchanges are in flight, it always shows a negative number. Does anyone why this is happening
Bit of a Camel newbie but here goes.
I have the following route:
from("activemq:queue:outputQueue").inputType(HelloWorld.class)
.to("log:stream")
.marshal().json(JsonLibrary.Jackson, HelloWorld.class)
.to("http:localhost:5000/messageForYouSir?bridgeEndpoint=true");
This retrieves messages from the queue and sends them to the HTTP endpoint as JSON. Fine.
But what if there is an error? Say a HTTP error code of 400? Then I want the message to stay on the queue. I have tried looking into not acknowledging the message, but have not been able to make it work.
Also I have made an Exception handler
onException(HttpOperationFailedException.class)
.handled(false)
.setBody().constant("Vi fekk ein feil");
But still the messages are gone from the queue. Is there some magic spell that can make Camel not acknowledge the messages when there is some error?
You have to consume transacted from the queue to be able to do a rollback. This is configured on the connection configuration to the broker.
Take a look at the Camel JMS docs (the ActiveMQ component extends the JMS component), especially the sections about cache levels and transacted consumption.
The most simple setup is using broker transactions by simply set transacted = true and lazyCreateTransactionManager = false on the JmsConfiguration. This way no Spring TX manager is required.
If transacted consumption is in place and the HTTP server returns an error (basically if an Exception occurs in the Camel Route), Camel does automatically a rollback (if you don't catch the error).
I have a application which places message in JMS Queue.The message size can be upto 150 mb. JMS message contains byte array data with additional metadata as xml,once the message is placed in queue the camel route application picks the message and calls the external java api with the message.i would like to stream the message from the JMS Queue using camel jms component.
Can you please tell me if i can achieve it using camel.
Currently i am facing heap error issue.
Sample JMS message is as below.
<request>
<esb-metadata>
<source-transaction-id>1234567890</source-transaction-id>
<environment>Dev</environment>
<message-type>REQ</message-type>
<source-id>12345678</source-id>
<master-id>mst-id1234</master-id>
</esb-metadata>
<message-body>
<parameters>
<info>
<document-class>Document</document-class>
<document-guid>idd_C0A5DF59-0000-C51C-BCDA-E76E9476D5C2</document-guid>
</info>
</parameters>
<document>
<document-mime-type>std/text</document-mime-type>
<document-content>SGVhZGluZzEgQ2hhbmdlZCB0b2RheSBKYW4gMTEgc2FtcGxlIHRleHQgYmVsb3csDQpPbmNlIEkg
cmVjZWl2ZSB0aGlzIGluZm9ybWF0aW9uLCBJIHdpbGwgdGhlbiBjb250YWN0IHlvdSBpbW1lZGlh
dGVseSB0byBkaXNjdXNzIHRoZSByZXNvbHV0aW9uIG9mIHlvdXIgY2xhaW0uDQoNCg0KSGVhZGlu
ZzIgc2FtcGxlIHRleHQsDQpJZiBwb3NzaWJsZSwgSSBuZWVkIHRvIHJlY2VpdmUgdGhpcyBpbmZv
cm1hdGlvbiBmcm9tIHlvdSB3aXRoaW4gMzAgRGF5cy4NCg0KMQkxCTENCjIJMgkyDQozCTMJMw0K
NAk0CTQNCg0KSGVhZGluZzMgc2FtcGxlIHRleHQsDQpJIHdpbGwgYXNzdW1lIHlvdSBkbyBub3Qg
d2lzaCB0byBwdXJzdWUgdGhpcyBjbGFpbSBhbmQgSSB3aWxsIGNsb3NlIG15IGZpbGUuDQpBRERF
RCBUSElTIE5FVyBMSU5FIFRPIFNFRSBJRiBJVCBJUyBWSVNJQkxFDQoNCnRhYmxlIHN0YXJ0cyBo
ZXJlDQpwb2xpY3kgbnVtYmVyCWNsYWltIG51bWJlcglpbnN1cmVkDQpwMTIzCWMxMjMJYm9zY2gN
CnAzMjEJYzMyMQlibXcNCnRhYmxlIGVuZHMgaGVyZQ0KDQomIzk2Nzk7CUJ1bGxldCBwb2ludCAx
DQomIzk2Nzk7CUJ1bGxldCBwb2ludCAyDQomIzk2Nzk7CUJ1bGxldCBwb2ludCAzDQoNCjEuCU51
bWJlcmVkIHBvaW50IDENCjIuCU51bWJlcmVkIHBvaW50IDINCjMuCU51bWJlcmVkIHBvaW50IDMN
Cg== </document-content>
</document>
</message-body>
</request>
HornetQ has support for streaming large messages so that clients don't have to worry about hitting memory limits. This support works in the HornetQ JMS implementation, but the JMS API doesn't directly support streaming large messages so a standard JMS component (e.g. one that Camel provides) likely won't work for this use-case. You'd need to use a component written to support the HornetQ implementation specifically. You can read about how HornetQ JMS clients can stream large messages in the documentation.
The MuleSoft HTTP ListenerConnector transforms an HTTP request into a Mule Message. It is my understanding that, as a consequence, the message is stored in a queue - therefore it can be guaranteed that the message will be delivered (once the request reached the integration platform).
In contrast, apache camel does not come with a messaging system. Therefore, I assume, the message is not stored anywhere and is lost when the destination of the message is temporarily not available.
Is that right? When using camel together with some messaging system, can the mule behavior be implemented easily?
Yes, that is true Camel does not come with a messaging or runtime system. You need to do decide on those yourself. I have used Camel with Karaf and messaging system such as ActiveMQ and RabbitMQ. You can easily ensure delivery use both of those together with the correct configuration. Both support persistent messaging, persistent queues and redelivery. Off course the main to do decide is, what happens if the host running your eventbus dies. In that case you need to consider clustering. The same applies for Mule.
What's the best strategy to send SMS via SMPP with Camel ? Should I use the ProducerTemplate ?
I'm new to camel so I'm not confident if my strategy is the best.
In my application upon reception of an SMS, I have to send back an other SMS with some computed content.
I created a
route smsIn that looks like this
from "uri=smpp ..."
unmarshal ref="bindyDataFormat"
to "uri=bean:myBean
and a route smsOut with
from "uri=direct:smsOut"
to "uri=smpp ..."
The smsIn route, receives the sms, transforms its conent (csv data) in a pojo and send that pojo to myBean.
In myBean I do some processing and then call a ProducerTemplate which send my computed message to the endpoint "direct:smsOut".
The reason I use the producerTemplate is that I have to set some info from my pojo in the header (CamelSmppDestAddr) and the body of the Exchange.
I have tested with the logica SMSC simulator, this seems to work fine, but would like to have your opinion about this solution ?
What about reliability , transaction ?
Should I store my message before trying to send it to the SMSC ?
Should I store it in a database, post it to a queue ?
I'm not sure why you have a producer template, you could just build up the route instead (given that you return something from your bean or takes an Exchange as paramter).
<from uri="smpp: ..."/>
<bean ref="bean:myBean"/>
<to uri="jms:queue:myQueue"/>
then not use direct, but use a JMS queue that is transactional and persistent. Say your smpp call fails, the message would have been gone. Using a queue like this and make sure its transactional, you can make sure not to lose data in this stage of the route.
<from uri="jms:queue:myQueue"/>
<transactional/>
<to uri="smpp.."/>
I suggest using Apache ActiveMQ as JMS middleware. Actually, if you download ActiveMQ, you get camel bundled, so you could actually run your Camel routes from ActiveMQ.
You might want to tweak how retries and error handling occurs dependent on what you want to happen (retry every second forever?, retry five times, then put to error queue? etc).
Read this page: Transaction Error handling in Camel
For deeper info and more tweaks, you might also want to read this:
Transactional Client