Long times on get pooled connection from datasource - database

Lately we have been having a lot of lag with getting pooled connections from datasource. Is this a db issue ? Would slow DB be a cause of this or should we check application code for a fix for this. This happens intermittently though. Would increasing pool size work or is there any other fix or checks.
Thanks
<service interface="javax.sql.DataSource">
<service-properties>
<entry key="osgi.jndi.service.name" value="jdbc/Idn/pfile/DataSource" />
</service-properties>
<bean class="com.ibm.db2.jcc.DB2DataSource">
<property name="serverName" value="${host}" />
<property name="portNumber" value="${port}" />
<property name="databaseName" value="${dbName}" />
<property name="currentSchema" value="${schema}" />
<property name="user" value="${user}" />
<property name="password" value="${password}" />
<property name="driverType" value="${drivertype}" />
<property name="dataSourceName" value="${dataSource.name}" />
</bean>
</service>


try using the DB2ConnectionPoolDataSource or other JDBC connection pool (DBCP, etc)

Related

PooledConnectionFactory - The session has already been closed

I face an exception and see the following getting logged when using the org.apache.activemq.pool.PooledConnectionFactory for producing messages to an AMQ Queue -
Caused by: java.lang.IllegalStateException: The session has already been closed
at org.apache.activemq.jms.pool.PooledSession.safeGetSessionHolder(PooledSession.java:482) ~[activemq-jms-pool-5.14.3.jar:5.14.3]
at org.apache.activemq.jms.pool.PooledSession.getInternalSession(PooledSession.java:382) ~[activemq-jms-pool-5.14.3.jar:5.14.3]
at org.apache.activemq.jms.pool.PooledSession.createQueue(PooledSession.java:197) ~[activemq-jms-pool-5.14.3.jar:5.14.3]
It also shows up as :
Caused by: javax.jms.IllegalStateException: The Session is closed
at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:771) ~[activemq-client-5.14.3.jar:5.14.3]
at org.apache.activemq.ActiveMQSession.createQueue(ActiveMQSession.java:1329) ~[activemq-client-5.14.3.jar:5.14.3]
at org.apache.activemq.jms.pool.PooledSession.createQueue(PooledSession.java:197) ~[activemq-jms-pool-5.14.3.jar:5.14.3]
A few points to note:
The parameter timeBetweenExpirationCheckMillis was added recently to the bean and the exceptions started showing up only after this change.
My code uses the same connection pool for all consumers and producers.
For AMQ interactions, I also use concurrent consumers in several places.
The beans for the connection and the pool are as follows:
<bean id="jmsPooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop" >
<property name="maxConnections" value="40" />
<property name="connectionFactory" ref="smiConnectionFactory" />
<property name="timeBetweenExpirationCheckMillis" value="300000" />
</bean>
<bean id="smiConnectionFactory" class="org.apache.activemq.ActiveMQSslConnectionFactory">
<property name="brokerURL" value="<ssl_protocol_broker>" />
<property name="userName" value="***********" />
<property name="password" value="***********" />
<property name="trustStore" value="************"/>
<property name="trustStorePassword" value="***********" />
<property name="keyStore" value="***********"/>
<property name="keyStorePassword" value="***********" />
<property name="optimizeAcknowledge" value="true" />
<property name="dispatchAsync" value="true" />
<property name="alwaysSessionAsync" value="true" />
<property name="useAsyncSend" value="true" />
<property name="watchTopicAdvisories" value="false" />
<property name="prefetchPolicy" ref="prefetchPolicy" />
</bean>
<bean id="prefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy">
<property name="durableTopicPrefetch" value="2000" />
<property name="topicPrefetch" value="32766" />
<property name="queuePrefetch" value="0" />
</bean>
**UPDATE 1:**
Active MQ version : 5.14.5

Spring batch, unable to write on oracle database

this is my xml file configuration.
<batch:job id="pdgRecallJob">
<batch:step id="stepTmpRecall">
<batch:tasklet>
<batch:chunk reader="readFileRecall" processor="" writer="tmpRecallWriter" commit-interval="10000">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="tmpRecall" class="it.mef.pdg.batch.model.TmpRecall" scope="prototype" />
<bean id="readFileRecall" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="strict" value="false" />
<property name="resource" value="classpath:/properties/RECALLPDG_*.txt"/>
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value=";"/>
<property name="names" value="appNomeFile, appCodFisc, appRata, appImporto, appIscrizione, appNumTitolo,
appProgQuota, appAnnoEmissione, appCodEnte, appEsitoMef, appStatoFile, appDpt, appPostazione, appNLotto,
appDLotto, appNomeFileOrigine" />
</bean>
</property>
<property name="fieldSetMapper">
<bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="tmpRecall" />
</bean>
</property>
</bean>
</property>
</bean>
<bean id="tmpRecallWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="dataSource" ref="dataSource" />
<property name="sql">
<value>
<![CDATA[
INSERT INTO APP_TMP_RECALL(APP_NOME_FILE,APP_COD_FISC,APP_RATA,APP_IMPORTO,APP_ISCRIZIONE,APP_NUM_TITOLO,APP_PROGR_QUOTA,APP_ANNO_EMISSIONE,APP_COD_ENTE,APP_ESITO_MEF,APP_STATO_FILE,APP_DPT,APP_POSTAZIONE,APP_NLOTTO,APP_DLOTTO,APP_NOME_FILE_ORIGINE)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
]]>
</value>
</property>
<property name="itemPreparedStatementSetter">
<bean class="it.mef.pdg.batch.items.TmpRecallPreparedStatementSetter" />
</property>
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="${db.url}" />
<property name="username" value="${db.user}" />
<property name="password" value="${db.pass}" />
</bean>
Then my itemPreparedStatementSetter Class:
final class TmpRecallPreparedStatementSetter implements ItemPreparedStatementSetter<TmpRecall>{
public void setValues(TmpRecall tmp, PreparedStatement ps) throws SQLException {
ps.setString(1, tmp.getAppNomeFile());
ps.setString(2, tmp.getAppCodFisc());
ps.setInt(3, tmp.getAppRata());
ps.setDouble(4, tmp.getAppImporto());
ps.setString(5, tmp.getAppIscrizione());
ps.setInt(6, tmp.getAppNumTitolo());
ps.setString(7, tmp.getAppProgQuota());
ps.setInt(8, tmp.getAppAnnoEmissione());
ps.setString(9, tmp.getAppCodEnte());
ps.setString(10, tmp.getAppEsitoMef());
ps.setString(11, tmp.getAppStatoFile());
ps.setString(12, tmp.getAppDpt());
ps.setInt(13, tmp.getAppPostazione());
ps.setInt(14, tmp.getAppNLotto());
ps.setDate(15, tmp.getAppDLotto());
ps.setInt(16, tmp.getAppNLotto());
}
}
My batch seems to work then I have final status COMPLETED, but it doesn't write no record inside my table.
I'm using jdk 1.5, ojdbc5-11.2.0.3.jar.
Either is there something I'm missing in my xml conf or any bean is not ok?
I was thinking that might be even a mismatch in Date field in table.

You don't currently have an item processor, which (I think) is needed to pass records from the reader to the writer. Since you are not doing any processing (that is, your reader produces the same type that your writer consumes), I think you could use a PassThroughItemProcessor here.

Spring : How to manually reset connection pool's connection?

I am trying to validate the database health manually using #Aspect.
And if something happened in DatabaseA (in my case is DatabaseA have no more write permission) the application will switch to DatabaseB.
How can I manually re-initialize the connection pool so that I can ensure all connection is going to DatabaseB?
#Aspect
#Component
#Order(value=1)
public class PostServiceAspect extends BaseService {
#Before("execution(* com.some.class.execute(..))")
public void databaseHealthCheck() {
try {
databaseDao.excuteHealthCheckSQL(); // execute "UPDATE table SET field = 1;"
} catch (Exception ex) {
databaseDao.switchTargetDatabase(); // switch to databaseB
// How to re-initialize the connection pool
}
}
//DataSource config
<bean id="dataSource" class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://testinghost:3306/devgpayment" />
<property name="username" value="root" />
<property name="password" value="password" />
<property name="defaultAutoCommit" value="false" />
<property name="maxTotal" value="40" />
<property name="maxIdle" value="40" />
<property name="maxWaitMillis" value="10000" />
<property name="validationQuery" value="SELECT 1" />
<property name="testWhileIdle" value="true" />
<property name="timeBetweenEvictionRunsMillis" value="5000" />
</bean>

camel jms not getting messages from IBM MQ

I have several camel microservices deployed to tomcat as individual war files. Each war file contains a unique camel route. One of the services accepts soap requests and puts a message onto a queue on IBM MQ. This works without a problem. The services which get these messages don't appear to pull the messages off the queue and forward them on. Each war file has the following as its IBM MQ configuration
<!-- Configure IBM MQ connection factory -->
<bean id="ibmMqConnectionFactory" class="com.ibm.mq.jms.MQConnectionFactory">
<property name="transportType" value="1"/>
<property name="hostName" value="${ibm.mq.host}"/>
<property name="port" value="${ibm.mq.port}"/>
<property name="channel" value="${ibm.mq.channel}"/>
<property name="queueManager" value="${ibm.qm.name}"/>
</bean>
<bean id="ibmMqPooledConnectionFactory" class="org.apache.activemq.jms.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="connectionFactory" ref="ibmMqConnectionFactory"/>
<property name="maxConnections" value="8"/>
<property name="maximumActiveSessionPerConnection" value="1"/>
<property name="expiryTimeout" value="30000"/>
<property name="idleTimeout" value="30000"/>
<property name="blockIfSessionPoolIsFull" value="false"/>
</bean>
<bean id="ibmMqConnectionFactorySecured" class="org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter">
<property name="targetConnectionFactory" ref="ibmMqPooledConnectionFactory"/>
<property name="username" value="${ibm.mq.username}"/>
<property name="password" value="${ibm.mq.password}"/>
</bean>
<bean id="ibmMqTransactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="ibmMqConnectionFactorySecured" />
</bean>
<bean id="ibmMqJmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="ibmMqConnectionFactorySecured"/>
<property name="concurrentConsumers" value="4"/>
<property name="transacted" value="true"/>
<property name="transactionManager" ref="ibmMqTransactionManager" />
<property name="cacheLevelName" value="CACHE_NONE" />
</bean>
<bean id="ibmMq" class="org.apache.camel.component.jms.JmsComponent">
<property name="configuration" ref="ibmMqJmsConfig"/>
</bean>
An example of one of the camel routes which pulls a message of a queue looks like
<route id="cancellation-service-route" errorHandlerRef="deadLetterErrorHandler">
<from uri="{{cancellationService.queue}}"/>
<unmarshal ref="requestCancellationRequest"/>
<to uri="bean:cancellationTranslator?method=transform"/>
...
</route>
The route starts but does not pull messages off the queue.

I resolved the truncation issue by doing <convertBodyTo type="String" /> before sending the message to the queue. I suspect it is because the message is an UTF-8 ecoded xml message.

How to use Camel to get message from remote WMQ and place message in ActiveMQ?

I have been looking at these for days and still haven't have mush success.
I have install and run activeMQ on a server. Created a queue called "testUpdate". I also have another queue on another server, let call this "forward" and this is on the IBM MQ (WMQ).
So we have testUpdate on ActiveMQ and forward on WMQ. I want the message on forward queue to be placed into testUpdate queue. Trying to use Camel for this process.
So the setup in ActiveMQ has an XML(activemq.xml) that I can place spring beans and configure to do the routing. In this xml, I added http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd to the xmlns and then added
import resource="camel.xml"
In this new xml, I have the following:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<description>Example Camel Route</description>
<from uri="mqseries:forward"/>
<to uri="activemq:testUpdate"/>
</route>
</camelContext>
<bean id="mqseries" class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="transportType" value="1"/>
<property name="hostName" value="172.00.12.21/>
<property name="port" value="xyza"/>
<property name="queueManager" value="manager"/>
<property name="channel" value="srvcChannel"/>
</bean>
<!--
Lets configure some Camel endpoints
http://camel.apache.org/components.html
-->
<!-- configure the camel activemq component to use the current broker -->
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" >
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://amq-broker?create=false"/>
<property name="userName" value="${activemq.username}"/>
<property name="password" value="${activemq.password}"/>
</bean>
</property>
</bean>
</beans>
I added the jar file com.ibm.mq.allclient.jar into the lib folder of ActiveMQ. When running the program, I am getting this exception:
java.lang.ClassNotFoundException: javax.jms.JMSRuntimeException
It sounds like I am missing the websphere jar file? Is this correct?

I use both IBM MQ and ActiveMQ on a regular basis. This example below will show you some sample configuration options. Please make sure you configure these to your own use cases.
//ActiveMQ connection factory
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" destroy-method="doStop">
<property name="configuration">
<bean class="org.apache.camel.component.jms.JmsConfiguration">
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
<property name="acceptMessagesWhileStopping" value="true" />
<property name="acknowledgementModeName" value="CLIENT_ACKNOWLEDGE" />
<property name="cacheLevelName" value="CACHE_CONSUMER" />
<property name="connectionFactory">
<bean class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="1" />
<property name="MaximumActiveSessionPerConnection" value="500" />
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${activemq1.brokerUrl}" />
<property name="userName" value="${activemq1.username}" />
<property name="password" value="${activemq1.password}" />
<property name="redeliveryPolicy">
<bean class="org.apache.activemq.RedeliveryPolicy">
<property name="maximumRedeliveries" value="-1" />
</bean>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</property>
</bean>
//IBM MQ connection factory
<bean id="ibmmq" class="org.apache.camel.component.jms.JmsComponent" destroy-method="doStop">
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
<property name="connectionFactory">
<bean class="org.springframework.jms.connection.SingleConnectionFactory" destroy-method="destroy">
<constructor-arg>
<bean class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="transportType" value="1" />
<property name="channel" value="${channel}" />
<property name="hostName" value="${hostname}" />
<property name="port" value="${port}" />
<property name="queueManager" value="${queueManager}" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>

You can use jms component to consume the message from IBM MQ and use activemq component to post it to ActiveMQ
have a look at the following links
http://camel.apache.org/jms.html
http://camel.apache.org/activemq.html
Thanks,
Gibu

You can use users camel extra component https://github.com/camel-extra/camel-extra/blob/master/components/camel-wmq/README.md to get messages from wmq without using the JMS wrapping and then send messages to ActiveMQ with custom camel component https://camel.apache.org/components/latest/activemq-component.html. See also a similar topic Apache Camel with IBM MQ.

Resources