when sending a message through a JMS topic, I often get the following warning:
2016-07-22 14:32:02,418 WARN [org.apache.activemq.artemis.jms.client] (Finalizer) AMQ122000: I''m closing a JMS connection you left open. Please make sure you close all JMS connections explicitly before letting them go out of scope! see stacktrace to find out where it was created: java.lang.Exception
at org.apache.activemq.artemis.jms.client.ActiveMQConnection.<init>(ActiveMQConnection.java:155)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:750)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createContext(ActiveMQConnectionFactory.java:255)
at org.wildfly.extension.messaging.activemq.deployment.JMSContextProducer$JMSContextWrapper.create(JMSContextProducer.java:195)
at org.wildfly.extension.messaging.activemq.deployment.JMSContextProducer$JMSContextWrapper.getDelegate(JMSContextProducer.java:235)
at org.wildfly.extension.messaging.activemq.deployment.JMSContextProducer$JMSContextWrapper.createProducer(JMSContextProducer.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139)
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:395)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:202)
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
My code:
#Inject
#JMSConnectionFactory("java:/ConnectionFactory")
private JMSContext context;
#Resource(lookup = "java:/jms/topic/MyTopic", type = Long.class)
private Topic topic;
public void sendMessage(Event event) {
try {
context.createProducer().send(topic, 1L);
} catch (Exception e) {;
log.error("Error while sending message to the JMS topic. Will retry later.", e);
}
}
Any idea? I couldn't find anything on google yet.
According to JMSContext javadoc, the connection should be automatically closed when it's managed by the application server.
I think the message clearly explains the problem. Probably you just need to call close() or some sort on the producer you created.
Edit:
Actually, it is not the produced that needs closing, but the context.
From the doc (https://docs.oracle.com/javaee/7/api/javax/jms/JMSContext.html):
All these JMSContext objects are application-managed and must be
closed when no longer needed by calling their close method.
About the producer, it says (https://docs.oracle.com/javaee/7/api/javax/jms/JMSProducer.html):
Instances of JMSProducer are intended to be lightweight objects which
can be created freely and which do not consume significant resources.
This interface therefore does not provide a close method.
Related
I'm struggling with a problem in an application based on Apache Camel when connecting to ActiveMQ Artemis via JMS. At the end of one of the Camel routes, messages are stored in an Artemis JMS queue. A legacy component running in the same application picks them up from there periodically using a ConsumerTemplate.
This works fine for Camel messages with plain text bodies, but causes errors when using byte array bodies: It seems Artemis treats any message with byte body as a "large message", which are streamed instead of kept in memory. Receiving via the ConsumerTemplate works, but as soon as the body or headers are accessed, an exception as follows is raised:
org.apache.camel.RuntimeCamelException: Failed to extract body due to: javax.jms.IllegalStateException: AMQ119023: The large message lost connection with its session, either because of a rollback or a closed session. Message: ActiveMQMessage[ID:90c4d1d5-3233-11ea-b0cc-44032c68a56f]:PERSISTENT/ClientLargeMessageImpl[messageID=2974, durable=true, address=mytest,userID=90c4d1d5-3233-11ea-b0cc-44032c68a56f,properties=TypedProperties[firedTime=Wed Jan 08 17:26:03 CET 2020,__AMQ_CID=90b4f34e-3233-11ea-b0cc-44032c68a56f,breadcrumbId=ID-NB045-evolit-co-at-1578500762151-0-1,_AMQ_ROUTING_TYPE=1,_AMQ_LARGE_SIZE=3]]
at org.apache.camel.component.jms.JmsBinding.extractBodyFromJms(JmsBinding.java:172) ~[camel-jms-2.22.1.jar:2.22.1]
at org.apache.camel.component.jms.JmsMessage.createBody(JmsMessage.java:221) ~[camel-jms-2.22.1.jar:2.22.1]
at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:54) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.example.cdi.JmsPoller.someMethod(JmsPoller.java:36) ~[classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_171]
at org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:481) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.bean.MethodInfo$1.doProceed(MethodInfo.java:300) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:273) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.bean.AbstractBeanProcessor.process(AbstractBeanProcessor.java:188) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.bean.BeanProcessor.process(BeanProcessor.java:53) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.bean.BeanProducer.process(BeanProducer.java:41) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:197) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:79) [camel-core-2.22.1.jar:2.22.1]
at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_171]
at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_171]
Caused by: javax.jms.IllegalStateException: AMQ119023: The large message lost connection with its session, either because of a rollback or a closed session
at org.apache.activemq.artemis.core.client.impl.LargeMessageControllerImpl.saveBuffer(LargeMessageControllerImpl.java:273) ~[artemis-core-client-2.6.2.jar:2.6.2]
at org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.saveToOutputStream(ClientLargeMessageImpl.java:115) ~[artemis-core-client-2.6.2.jar:2.6.2]
at org.apache.activemq.artemis.jms.client.ActiveMQMessage.saveToOutputStream(ActiveMQMessage.java:853) ~[artemis-jms-client-2.6.2.jar:2.6.2]
at org.apache.activemq.artemis.jms.client.ActiveMQMessage.setObjectProperty(ActiveMQMessage.java:693) ~[artemis-jms-client-2.6.2.jar:2.6.2]
at org.apache.camel.component.jms.JmsBinding.createByteArrayFromBytesMessage(JmsBinding.java:251) ~[camel-jms-2.22.1.jar:2.22.1]
at org.apache.camel.component.jms.JmsBinding.extractBodyFromJms(JmsBinding.java:163) ~[camel-jms-2.22.1.jar:2.22.1]
... 21 more
Caused by: org.apache.activemq.artemis.api.core.ActiveMQIllegalStateException: AMQ119023: The large message lost connection with its session, either because of a rollback or a closed session
at org.apache.activemq.artemis.core.client.impl.LargeMessageControllerImpl.saveBuffer(LargeMessageControllerImpl.java:273) ~[artemis-core-client-2.6.2.jar:2.6.2]
at org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.saveToOutputStream(ClientLargeMessageImpl.java:115) ~[artemis-core-client-2.6.2.jar:2.6.2]
at org.apache.activemq.artemis.jms.client.ActiveMQMessage.saveToOutputStream(ActiveMQMessage.java:853) ~[artemis-jms-client-2.6.2.jar:2.6.2]
at org.apache.activemq.artemis.jms.client.ActiveMQMessage.setObjectProperty(ActiveMQMessage.java:693) ~[artemis-jms-client-2.6.2.jar:2.6.2]
at org.apache.camel.component.jms.JmsBinding.createByteArrayFromBytesMessage(JmsBinding.java:251) ~[camel-jms-2.22.1.jar:2.22.1]
at org.apache.camel.component.jms.JmsBinding.extractBodyFromJms(JmsBinding.java:163) ~[camel-jms-2.22.1.jar:2.22.1]
... 21 more
The problem also occurs for messages that do not exceed the minLargeMessageSize of Artemis, in a test program even for 3 bytes.
Coincidentally, the same problem occurred in a standalone application used for testing the application. There, I was able to solve the issue by keeping the JMS session and receiver open until the JMS message body and headers were completely read. With Camel, that's abstracted away in the Spring JmsTemplate that Camel is based on.
I consulted the user documentation of the Camel JMS component to find configuration options that might help me. I've tried the following:
eagerLoadingOfProperties=true on consumer side: no effect, only seems to affect MessageListenerContainer. The documentation says:
It uses [...] Spring’s JmsTemplate for sending and a MessageListenerContainer for consuming.
However, while debugging it seemed that MessageListenerContainer is only used when consuming messages from an JMS endpoint in a Camel route. Using a ConsumerTemplate like in my case uses a JmsTemplate for consuming.
messageConverter and mapJmsMessage on consumer side: no effect, they are executed when the session has already been closed
alwaysCopyMessage on producer side: I thought maybe copying prevents use of streamed large messages, no effect
streamMessageTypeEnabled=false on producer side: no effect
jmsMessageType=Bytes on both producer and consumer side: no effect
transferExchange=true on both producer and consumer side: this does seem to solve my specific case, but it feels like a workaround. Documentation advises to use the option with caution.
So right now, transferExchange seems to be my best bet, assuming it really solves my issue in all test cases. Nevertheless, I'd be glad to get better understanding on the issue or different solutions:
Why does Artemis treat small byte array messages as large messages anyway?
Does Camel ConsumerTemplate support streamed large messages at all?
My versions are Camel 2.22.1 and Artemis 2.10.1.
I've been able to reproduce my problem by modifying the Camel Example camel-example-cdi from the release package of Camel to have the minimal classes shown below.
In addition I've added camel-jms and Artemis dependencies and started Artemis locally, both like described in the camel-example-artemis-large-messages example.
public class MyRoutes extends RouteBuilder {
#Override
public void configure() {
setupJmsComponent();
from("timer:writeTimer?period=6000")
.log("writing to JMS")
.setBody(() -> new byte[]{0,1,2})
.to(JmsPoller.ENDPOINT);
from("timer:pollTimer?period=3000")
.to("bean:jmsPoller");
}
private void setupJmsComponent() {
ActiveMQJMSConnectionFactory connectionFactory = new ActiveMQJMSConnectionFactory("tcp://localhost:61616");
JmsComponent jmsComponent = new JmsComponent();
jmsComponent.setConnectionFactory(connectionFactory);
getContext().addComponent("jms", jmsComponent);
}
}
#Singleton
#Named("jmsPoller")
public class JmsPoller {
static final String ENDPOINT = "jms:queue:mytest";
#Inject
private ConsumerTemplate consumerTemplate;
public void someMethod(String body) {
Exchange exchange = consumerTemplate.receive(ENDPOINT, 1000L);
System.out.println("Received " + (exchange == null ? null : exchange.getIn().getBody()));
}
}
ActiveMQ Artemis doesn't treat just any message with a byte body as a "large" message. It's worth noting that the broker ultimately treats all message bodies as an array of bytes because that's exactly what they are. However, in order to be considered "large" the message has to exceed a certain size. The documentation states:
Any message larger than a certain size is considered a large message. Large messages will be split up and sent in fragments. This is determined by the URL parameter minLargeMessageSize.
Note:
Apache ActiveMQ Artemis messages are encoded using 2 bytes per character so if the message data is filled with ASCII characters (which are 1 byte) the size of the resulting Apache ActiveMQ Artemis message would roughly double. This is important when calculating the size of a "large" message as it may appear to be less than the minLargeMessageSize before it is sent, but it then turns into a "large" message once it is encoded.
The default value is 100KiB.
It looks like the application's use-case simply doesn't fit with the semantics of large message support in ActiveMQ Artemis since the session which the message came from is being closed before the message's body is fully received.
Therefore, I recommend that you either keep the session open until the body is read or increase the minLargeMessageSize on the URL of the application which is sending the message so that no messages are ever considered "large." The latter option may result in greater memory usage on the broker since the entire message body will be held in memory at once.
When I shutdown DB after flush() and before commit(), an exception is logged but not captured by the code:
#Stateless
#TransactionAttribute(TransactionAttributeType.NEVER)
public class OuterService {
#EJB InnerService innerService;
public String outerMethod() {
try {
innerService.innerMethod();
return "success";
} catch (Exception e) {
return "failure";
}
}
}
#Stateless
#TransactionAttribute(TransactionAttributeType.REQUIRED)
public class InnerService {
#PersistenceContext EntityManager em;
public void innerMethod() {
em.persist(new Entity());
em.flush();
} //put the breakpoint here
}
I run the code in debug mode and set a breakpoint after flush but before exiting the transactional method. When the execution is paused, I stop the db service and then resume the code.
An exception is logged with the following root cause:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:
Communications link failure during commit(). Transaction resolution
unknown.
but it is not captured by the outer try..catch block and the method successfully completes. It seems that JTA implementation suffocates the exception. How can I be notified of an error?
I already tried BMT and CDI events but none worked. Plain JDBC and JPA (Hibernate, with built-in and C3p0 pools) in Java SE environment, however, do work.
My setup: Ubuntu 17.10, Wildfly 10, MySQL 5.7.20, Connector/J 5.1.44
Here is the log (some lines removed because of character limit):
2018-01-07 12:38:44,980 INFO [stdout] (default task-1) Hibernate: insert into Entity values ( )
2018-01-07 12:39:06,027 WARN [org.jboss.jca.core.connectionmanager.listener.TxConnectionListener] (default task-1) IJ000305: Connection error occured: org.jboss.jca.core.connectionmanager.listener.TxConnectionListener#f0b0aed[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection#327b7fd0 connection handles=0 lastReturned=1515316110106 lastValidated=1515316098805 lastCheckedOut=1515316124981 trackByTx=true pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool#a4e7bad mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool#42037075[pool=TestDS] xaResource=LocalXAResourceImpl#306327f6[connectionListener=f0b0aed connectionManager=6110d60 warned=false currentXid=null productName=MySQL productVersion=5.7.20-0ubuntu0.17.10.1 jndiName=java:/datasources/TestDS] txSync=TransactionSynchronization#360457732{tx=TransactionImple < ac, BasicAction: 0:ffff7f000101:-7fd727eb:5a51e37d:1d status: ActionStatus.COMMITTING > wasTrackByTx=true enlisted=true cancel=false}]: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.Util.getInstance(Util.java:408)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1552)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnection.commit(LocalManagedConnection.java:96)
at org.jboss.jca.core.tx.jbossts.LocalXAResourceImpl.commit(LocalXAResourceImpl.java:172)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239)
at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636)
at InnerService$$$view26.innerMethod(Unknown Source)
at OuterService.outerMethod(OuterService.java:23)
at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
at OuterService$$$view33.outerMethod(Unknown Source)
at RestManager.test(RestManager.java:112)
2018-01-07 12:39:06,032 WARN [com.arjuna.ats.jta] (default task-1) ARJUNA016039: onePhaseCommit on < formatId=131077, gtrid_length=47, bqual_length=36, tx_uid=0:ffff7f000101:-7fd727eb:5a51e37d:1d, node_name=mypc, branch_uid=0:ffff7f000101:-7fd727eb:5a51e37d:20, subordinatenodename=null, eis_name=java:/datasources/TestDS > (LocalXAResourceImpl#306327f6[connectionListener=f0b0aed connectionManager=6110d60 warned=false currentXid=null productName=MySQL productVersion=5.7.20-0ubuntu0.17.10.1 jndiName=java:/datasources/TestDS]) failed with exception XAException.XAER_RMFAIL: org.jboss.jca.core.spi.transaction.local.LocalXAException: IJ001156: Could not commit local transaction
at org.jboss.jca.core.tx.jbossts.LocalXAResourceImpl.commit(LocalXAResourceImpl.java:177)
at com.arjuna.ats.internal.jta.resources.arjunacore.XAOnePhaseResource.commit(XAOnePhaseResource.java:120)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91)
at InnerService$$$view26.innerMethod(Unknown Source)
at OuterService.outerMethod(OuterService.java:23)
at OuterService$$$view33.outerMethod(Unknown Source)
at RestManager.test(RestManager.java:112)
at RestManager$Proxy$_$$_Weld$EnterpriseProxy$.test(Unknown Source)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.jboss.jca.core.spi.transaction.local.LocalResourceException: Communications link failure during commit(). Transaction resolution unknown.
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnection.commit(LocalManagedConnection.java:103)
at org.jboss.jca.core.tx.jbossts.LocalXAResourceImpl.commit(LocalXAResourceImpl.java:172)
... 248 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:860)
at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1552)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnection.commit(LocalManagedConnection.java:96)
... 249 more
I investigated on this and it sounds to me as an issue. I created the jira here: https://issues.jboss.org/browse/JBTM-2983. Feel free to follow discussion there if it's confirmed so.
I expect the caller should be informed that there was an exception during commit. Btw. in case I of some further investigation I created a small test project based of your issue: https://github.com/ochaloup/catch-ejb-exception-test.git
I'm not sure. There could be several causes.
First I would try is to increase the net_write_timeout property in your mysql configuration.
Also your Exception is caused by error code XAException.XAER_RMFAIL
Jboss Javadoc: XAException:
Error code indicating that the resource manager has failed and is not
available.
So it seems that PersistenceContext is broken or invalid because of break point interruption.
I have an app engine java project and am using objectify. I get a stack trace sporadically in the "stack driver error reporting" view of the app engine web console related to putting an item into memcache. This is the code:
try {
TestItem t = new TestItem(...);
ofy().save().entity(t).now();
} catch (Exception e) {
}
and this is the error I'll see sporadically:
com.googlecode.objectify.cache.MemcacheServiceRetryProxy invoke: Memcache operation failed, giving up
java.lang.reflect.InvocationTargetException
at com.google.appengine.runtime.Request.process-i4dx9s2kED3CVcPe(Request.java)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:44)
at com.googlecode.objectify.cache.MemcacheServiceRetryProxy.invoke(MemcacheServiceRetryProxy.java:68)
at com.sun.proxy.$Proxy9.putAll(Unknown Source)
at com.googlecode.objectify.cache.KeyMemcacheService.putAll(KeyMemcacheService.java:91)
at com.googlecode.objectify.cache.EntityMemcache.empty(EntityMemcache.java:319)
at com.googlecode.objectify.cache.CachingAsyncDatastoreService$5.trigger(CachingAsyncDatastoreService.java:445)
at com.googlecode.objectify.cache.TriggerFuture.isDone(TriggerFuture.java:87)
at com.googlecode.objectify.cache.TriggerFuture.get(TriggerFuture.java:102)
at com.googlecode.objectify.impl.ResultAdapter.now(ResultAdapter.java:34)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:22)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:10)
at com.googlecode.objectify.util.ResultTranslator.nowUncached(ResultTranslator.java:21)
at com.googlecode.objectify.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:22)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:10)
at com.googlecode.objectify.util.ResultTranslator.nowUncached(ResultTranslator.java:21)
at com.googlecode.objectify.util.ResultCache.now(ResultCache.java:30)
at com.me.test.Test.putSomethinInMemcache(Test.java:13)
...
Caused by: com.google.appengine.api.memcache.MemcacheServiceException: Memcache putAll: Unknown exception setting 1 keys
at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.handleApiProxyException(MemcacheServiceApiHelper.java:69)
at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$RpcResponseHandlerForPut.handleApiProxyException(AsyncMemcacheServiceImpl.java:349)
at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.absorbParentException(MemcacheServiceApiHelper.java:111)
at com.google.appengine.api.utils.FutureWrapper.handleParentException(FutureWrapper.java:52)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:91)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:89)
at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:26)
at com.google.appengine.api.memcache.MemcacheServiceImpl.putAll(MemcacheServiceImpl.java:115)
... 52 more
It doesn't appear to be caught in the try-statement. I just see it in that admin console mentioned earlier.
Does anyone know what this means, or how I can catch it? My main worry is that there could be an old copy of the object stuck in memcache after this operation fails.
Using objectify 5.1.10.
Thanks
This is a get() operation. If memcache is unavailable during a get() operation, Objectify just reads from the datastore. The error is logged and performance suffers somewhat but the app marches on.
It is technically possible to have errors during write operations (any save() clears the cache entry; reads will repopulate the cache). This could in theory leave stale info in the cache. There's nothing that can really be done about this - if you can't clear the cache entry, it's going to be stuck there. My advice is that if you have sensitive data but want it cached, put a reasonable timeout on the cache entry (#Cache(expirationSeconds=60) or whatnot).
So well, I am trying to get a MQXAQueueConnectionFactory to work, I have created a extended class from the JmsComponent to handle username and password when sending data to the queue.
It does get/put messages on the queue, but in my case I've created a router to test the XA such as
from("wmq:queue:incomingQueue")
.process(new Processor(){
... Thread.sleep(20000)
})
.to("wmq:queue:outgoingQueue")
while being in sleep, I shut down the queuemanager. However when trying to get uncommited messages from the queue
DISPLAY QSTATUS('qChainQueue') i get CURDEPTH(0), while it should be 1 as I understand the XA part.
Am I doing this totally wrong?
How can it be tested?
HelpClass to handle WMQ:
public class WMQComponent extends JmsComponent {
private final String username;
private final String password;
public WMQComponent(String hostname, int port, String username, String password,
String queueManager, String channel) throws JMSException {
super();
this.username = username;
this.password = password;
MQXAQueueConnectionFactory connectionFactory = new MQXAQueueConnectionFactory();
connectionFactory.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
connectionFactory.setFailIfQuiesce(1);
connectionFactory.setHostName(hostname);
connectionFactory.setPort(port);
connectionFactory.setQueueManager(queueManager);
connectionFactory.setChannel(channel);
setConnectionFactory(connectionFactory);
}
#Override
public Endpoint createEndpoint(String uri) throws Exception {
if (uri.contains("username") || uri.contains("password")) {
throw new IllegalStateException("Username and password is set by the component");
}
if (uri.contains("?")) {
return super.createEndpoint(uri + "&username=" + username + "&password=" + password);
} else {
return super.createEndpoint(uri + "?username=" + username + "&password=" + password);
}
}
}
With the following errors:
2015-03-25 14:01:12,077 [ #2 - Multicast] INFO dest_chain_ldap - org.springframework.jms.IllegalStateException: JMSWMQ0018: Failed to connect to queue manager 'QMBATCHESB' with connection mode 'Client' and host name 'hostname.com'.; nested exception is com.ibm.msg.client.jms.DetailedIllegalStateException: JMSWMQ0018: Failed to connect to queue manager 'QMBATCHESB' with connection mode 'Client' and host name 'hostname.com'. Check the queue manager is started and if running in client mode, check there is a listener running. Please see the linked exception for more information.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2059' ('MQRC_Q_MGR_NOT_AVAILABLE').
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:279)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.send(JmsConfiguration.java:228)
at org.apache.camel.component.jms.JmsProducer.doSend(JmsProducer.java:431)
at org.apache.camel.component.jms.JmsProducer.processInOnly(JmsProducer.java:385)
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:153)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:120)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:163)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:416)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:51)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:120)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:163)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:416)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:105)
at org.apache.camel.processor.MulticastProcessor.doProcessParallel(MulticastProcessor.java:732)
at org.apache.camel.processor.MulticastProcessor.access$200(MulticastProcessor.java:82)
at org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:303)
at org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:288)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: com.ibm.msg.client.jms.DetailedIllegalStateException: JMSWMQ0018: Failed to connect to queue manager 'QMBATCHESB' with connection mode 'Client' and host name 'hostname.com'. Check the queue manager is started and if running in client mode, check there is a listener running. Please see the linked exception for more information.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:496)
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:236)
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:430)
at com.ibm.msg.client.wmq.internal.WMQXAConnection.<init>(WMQXAConnection.java:70)
at com.ibm.msg.client.wmq.factories.WMQXAConnectionFactory.createV7ProviderConnection(WMQXAConnectionFactory.java:190)
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createProviderConnection(WMQConnectionFactory.java:6210)
at com.ibm.msg.client.jms.admin.JmsConnectionFactoryImpl.createConnection(JmsConnectionFactoryImpl.java:278)
at com.ibm.mq.jms.MQConnectionFactory.createCommonConnection(MQConnectionFactory.java:6155)
at com.ibm.mq.jms.MQQueueConnectionFactory.createQueueConnection(MQQueueConnectionFactory.java:144)
at com.ibm.mq.jms.MQQueueConnectionFactory.createConnection(MQQueueConnectionFactory.java:223)
at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.doCreateConnection(UserCredentialsConnectionFactoryAdapter.java:175)
at org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter.createConnection(UserCredentialsConnectionFactoryAdapter.java:150)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:456)
... 29 more
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2059' ('MQRC_Q_MGR_NOT_AVAILABLE').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:223)
... 41 more
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2059;AMQ9204: Connection to host 'hostname.com(1514)' rejected. [1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2059;AMQ9213: A communications error for occurred. [1=java.net.ConnectException[Connection refused: connect],3=hostname.com]],3=hostname.com(1514),5=RemoteTCPConnection.connnectUsingLocalAddress]
at com.ibm.mq.jmqi.remote.internal.RemoteFAP.jmqiConnect(RemoteFAP.java:1831)
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:345)
... 40 more
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2059;AMQ9213: A communications error for occurred. [1=java.net.ConnectException[Connection refused: connect],3=hostname.com]
at com.ibm.mq.jmqi.remote.internal.RemoteTCPConnection.connnectUsingLocalAddress(RemoteTCPConnection.java:612)
at com.ibm.mq.jmqi.remote.internal.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:940)
at com.ibm.mq.jmqi.remote.internal.system.RemoteConnection.connect(RemoteConnection.java:1097)
at com.ibm.mq.jmqi.remote.internal.system.RemoteConnectionPool.getConnection(RemoteConnectionPool.java:348)
at com.ibm.mq.jmqi.remote.internal.RemoteFAP.jmqiConnect(RemoteFAP.java:1503)
... 41 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:69)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:157)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at com.ibm.mq.jmqi.remote.internal.RemoteTCPConnection$2.run(RemoteTCPConnection.java:597)
at java.security.AccessController.doPrivileged(Native Method)
at com.ibm.mq.jmqi.remote.internal.RemoteTCPConnection.connnectUsingLocalAddress(RemoteTCPConnection.java:588)
... 45 more
The reason code 2059 and various errors stating that the connection was refused suggest either a mechanical issue (i.e. Listener not running) or an auths issue.
If I were attempting to debug this, the first thing I'd do is to enable authorization events, channel events, and any others you would normally enable. If you use MQ Explorer, also install the MS0P Plugin which will allow you to view the event messages in human-readable text.
Next, I would use the MQ sample programs to test. Since I always install the full client rather than grabbing the jar files, I have amqsputc available. However, the Java classes have IVT (initial verification test) programs. These ensure that the listener is running, the channel is configured and available, etc. As of v7.1 this also ensures that the CHLAUTH rules are set to allow the access. As of v8.0, or if you had the Capitalware exit installed, this also lets us test the user ID and password authentication.
The queue manager's error log and the event messages should provide good diagnostics, assuming the connection request makes it to MQ. Be sure to look both in the QMgr-specific error logs and the installation-global error logs.
Once I had confirmed that basic connectivity is in place, I'd reconcile my client-side configuration parameters for host, port, channel and if it is specified [shudder!] the QMgr name. Assuming these are correct and having proven basic connectivity works, it is now possible to test the app with some confidence.
The same method applies. First make sure the app's connection request makes it to the QMgr. If it does and is refused, the event messages and error logs will note this and why. If there is no indication of a failure in these places, the app isn't getting to the QMgr. The 2059 can indicate that the socket was refused, that the listener is up but the QMgr is not, that the channel instances have maxed out, or that after provisionally starting the channel it was closed by the QMgr, often due to a CHLAUTH rule. In any case, the event messages and error logs will have a detailed explanation as to why.
So I had done this a bit wrong, it was not enough to use the MQXAConnectionFactory but I had to create the JmsComponent as transacted.
Have tried to stop the queue manager while running the application and stop the application while handling a message and it seems to do the rollback as expected.
Ended up with
public static JmsComponent mqXAComponentTransacted(String hostname, int port, String username, String password,
String queueManager, String channel) throws JMSException {
MQXAQueueConnectionFactory connectionFactory = new MQXAQueueConnectionFactory();
connectionFactory.setTransportType(JMSC.MQJMS_TP_CLIENT_MQ_TCPIP);
connectionFactory.setFailIfQuiesce(1);
connectionFactory.setHostName(hostname);
connectionFactory.setPort(port);
connectionFactory.setQueueManager(queueManager);
connectionFactory.setChannel(channel);
UserCredentialsConnectionFactoryAdapter connectionFactoryAdapter=new UserCredentialsConnectionFactoryAdapter();
connectionFactoryAdapter.setTargetConnectionFactory(connectionFactory);
connectionFactoryAdapter.setUsername(username);
connectionFactoryAdapter.setPassword(password);
return JmsComponent.jmsComponentTransacted(connectionFactoryAdapter);
}
Also using the UserCredentialsConnectionFactoryAdapter, I didn't want to use Spring components but since the Jms package is already dependent of it, it was easier to use it than my previous solution to handle credentials.
We are using camel 2.13.2 - I have a multicast route with an AggregationStrategy.
And in each multicast branch, we have a custom camel component that returns huge data (around 4 MB) and writes to Stream Cache (Cached Output Stream) and we need to aggregate the data in the multicast (Aggregation Strategy).
In the Aggregation strategy, I need to do XPath evaluation using camel XPathBuilder.
Hence, I try to read the body and convert from StreamCache to byte[] to avoid 'Error during type conversion from type: org.apache.camel.converter.stream.InputStreamCache.' in the XPathBuilder.
When I try to read the body in the beginning of the Aggregation Strategy, I get the following error.
*/tmp/camel/camel-tmp-4e00bf8a-4a42-463a-b046-5ea2d7fc8161/cos6047774870387520936.tmp (No such file or directory), cause: FileNotFoundException:/tmp/camel/camel-tmp-4e00bf8a-4a42-463a-b046-5ea2d7fc8161/cos6047774870387520936.tmp (No such file or directory).
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.(FileInputStream.java:138)
at org.apache.camel.converter.stream.FileInputStreamCache.createInputStream(FileInputStreamCache.java:123) at org.apache.camel.converter.stream.FileInputStreamCache.getInputStream(FileInputStreamCache.java:117)
at org.apache.camel.converter.stream.FileInputStreamCache.writeTo(FileInputStreamCache.java:93)
at org.apache.camel.converter.stream.StreamCacheConverter.convertToByteArray(StreamCacheConverter.java:102)
at com.sap.it.rt.camel.aggregate.strategies.MergeAtXPathAggregationStrategy.convertToByteArray(MergeAtXPathAggregationStrategy.java:169)
at com.sap.it.rt.camel.aggregate.strategies.MergeAtXPathAggregationStrategy.convertToXpathCompatibleType(MergeAtXPathAggregationStrategy.java:161)
*
Following is the line of code where it is throwing an error:
Object body = exchange.getIn().getBody();
if( body instanceof StreamCache){
StreamCache cache = (StreamCache)body;
xml = new String(convertToByteArray(cache,exchange));
exchange.getIn().setBody(xml);
}
By disabling stream cache to write to file by setting a threshold of 10MB in multicast related routes, we were able to work with the aggregation strategy. But we do not want to do that, as we may have incoming data that maybe bigger.
<camel:camelContext id="multicast_xml_1" streamCache="true">
<camel:properties>
<camel:property key="CamelCachedOutputStreamCipherTransformation" value="RC4"/>
<camel:property key="CamelCachedOutputStreamThreshold" value="100000000"/>
</camel:properties>
....
</camel:camelContext>
Note: The FileNotFound issue does not appear if we have the StreamCache based camel component in the route with other processors, but without Multicast + Aggregation.
After debugging, I could understand the issue with aggregating huge data from StreamCache with MulticastProcessor.
In MulticastProcessor.java: doProcessParallel() is called and on completion of the branch exchange of multicast, the CachedOutputStream deletes / cleans up the temporary file.
This happens even before the multicast branch exchange reaches the aggregation Strategy, which tries to read the data from the branch exchange. In case of huge data in StreamCache, the temporary file is already deleted, leading to FileNotFound issues.
public CachedOutputStream(Exchange exchange, boolean closedOnCompletion) {
this.strategy = exchange.getContext().getStreamCachingStrategy();
currentStream = new CachedByteArrayOutputStream(strategy.getBufferSize());
if (closedOnCompletion) {
// add on completion so we can cleanup after the exchange is done such as deleting temporary files
exchange.addOnCompletion(new SynchronizationAdapter() {
#Override
public void onDone(Exchange exchange) {
try {
if (fileInputStreamCache != null) {
fileInputStreamCache.close();
}
close();
} catch (Exception e) {
LOG.warn("Error deleting temporary cache file: " + tempFile, e);
}
}
#Override
public String toString() {
return "OnCompletion[CachedOutputStream]";
}
});
}
}
public void close() throws IOException {
currentStream.close();
cleanUpTempFile();
}
I was able to circumvent the issue, if I try to set closedOnCompletion= false, while writing to CachedOutputStream in any component in any Multicast branch.
But this is a leaky solution, because the streamcache temporary file(s) may then never get cleaned up... hence I try to close + clean up the cachestream, after reading the data in the AggregationStrategy.
Can the MulticastProcessor be adjusted so that the multicast branch exchanges reach 'completion' status only, after they have been aggregated at the end of multicast?
Please help / advise on the issue, as I am new to using camel Multicast.
Thanks,
Lakshmi
I have similar exception thrown when trying to send larger than 1MB JSON response to Restlet request (yes, I know 1MB JSON is too big):
java.io.FileNotFoundException: C:\Users\me\AppData\Local\Temp\camel\camel-tmp-7ad6e098-538d-4d4c-9357-2b7addb1f19d\cos6725022584818060586.tmp (The system cannot find the file specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at org.apache.camel.converter.stream.FileInputStreamCache.createInputStream(FileInputStreamCache.java:123)
at org.apache.camel.converter.stream.FileInputStreamCache.getInputStream(FileInputStreamCache.java:117)
at org.apache.camel.converter.stream.FileInputStreamCache.read(FileInputStreamCache.java:112)
at java.io.InputStream.read(InputStream.java:170)
at java.io.InputStream.read(InputStream.java:101)
at org.restlet.engine.io.BioUtils.copy(BioUtils.java:81)
at org.restlet.representation.InputRepresentation.write(InputRepresentation.java:148)
at org.restlet.engine.adapter.ServerCall.writeResponseBody(ServerCall.java:510)
at org.restlet.engine.adapter.ServerCall.sendResponse(ServerCall.java:454)
at org.restlet.ext.servlet.internal.ServletCall.sendResponse(ServletCall.java:426)
at org.restlet.engine.adapter.ServerAdapter.commit(ServerAdapter.java:196)
at org.restlet.engine.adapter.HttpServerHelper.handle(HttpServerHelper.java:153)
at org.restlet.ext.servlet.ServerServlet.service(ServerServlet.java:1089)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1496)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:102)
Same workaround works for me:
getContext().getProperties().put(CachedOutputStream.THRESHOLD, "" + THREE_MEGABYTE_TRESHOLD_BEFORE_FILE_CACHE);
I don't use multicast in this route, just plain
restlet request -> Service -> Jackson marshall => error
I use Camel 2.14.0 & Restlet 2.2.2 with JDK 7 and Spring-boot 1.0.2 / Jetty
This Camel reverse proxy - no response stream caching might be related to my issue.