this is my xml file configuration.
<batch:job id="pdgRecallJob">
<batch:step id="stepTmpRecall">
<batch:tasklet>
<batch:chunk reader="readFileRecall" processor="" writer="tmpRecallWriter" commit-interval="10000">
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="tmpRecall" class="it.mef.pdg.batch.model.TmpRecall" scope="prototype" />
<bean id="readFileRecall" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="strict" value="false" />
<property name="resource" value="classpath:/properties/RECALLPDG_*.txt"/>
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value=";"/>
<property name="names" value="appNomeFile, appCodFisc, appRata, appImporto, appIscrizione, appNumTitolo,
appProgQuota, appAnnoEmissione, appCodEnte, appEsitoMef, appStatoFile, appDpt, appPostazione, appNLotto,
appDLotto, appNomeFileOrigine" />
</bean>
</property>
<property name="fieldSetMapper">
<bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
<property name="prototypeBeanName" value="tmpRecall" />
</bean>
</property>
</bean>
</property>
</bean>
<bean id="tmpRecallWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="dataSource" ref="dataSource" />
<property name="sql">
<value>
<![CDATA[
INSERT INTO APP_TMP_RECALL(APP_NOME_FILE,APP_COD_FISC,APP_RATA,APP_IMPORTO,APP_ISCRIZIONE,APP_NUM_TITOLO,APP_PROGR_QUOTA,APP_ANNO_EMISSIONE,APP_COD_ENTE,APP_ESITO_MEF,APP_STATO_FILE,APP_DPT,APP_POSTAZIONE,APP_NLOTTO,APP_DLOTTO,APP_NOME_FILE_ORIGINE)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
]]>
</value>
</property>
<property name="itemPreparedStatementSetter">
<bean class="it.mef.pdg.batch.items.TmpRecallPreparedStatementSetter" />
</property>
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="${db.url}" />
<property name="username" value="${db.user}" />
<property name="password" value="${db.pass}" />
</bean>
Then my itemPreparedStatementSetter Class:
final class TmpRecallPreparedStatementSetter implements ItemPreparedStatementSetter<TmpRecall>{
public void setValues(TmpRecall tmp, PreparedStatement ps) throws SQLException {
ps.setString(1, tmp.getAppNomeFile());
ps.setString(2, tmp.getAppCodFisc());
ps.setInt(3, tmp.getAppRata());
ps.setDouble(4, tmp.getAppImporto());
ps.setString(5, tmp.getAppIscrizione());
ps.setInt(6, tmp.getAppNumTitolo());
ps.setString(7, tmp.getAppProgQuota());
ps.setInt(8, tmp.getAppAnnoEmissione());
ps.setString(9, tmp.getAppCodEnte());
ps.setString(10, tmp.getAppEsitoMef());
ps.setString(11, tmp.getAppStatoFile());
ps.setString(12, tmp.getAppDpt());
ps.setInt(13, tmp.getAppPostazione());
ps.setInt(14, tmp.getAppNLotto());
ps.setDate(15, tmp.getAppDLotto());
ps.setInt(16, tmp.getAppNLotto());
}
}
My batch seems to work then I have final status COMPLETED, but it doesn't write no record inside my table.
I'm using jdk 1.5, ojdbc5-11.2.0.3.jar.
Either is there something I'm missing in my xml conf or any bean is not ok?
I was thinking that might be even a mismatch in Date field in table.
You don't currently have an item processor, which (I think) is needed to pass records from the reader to the writer. Since you are not doing any processing (that is, your reader produces the same type that your writer consumes), I think you could use a PassThroughItemProcessor here.
Just need to check, If there is the the way to hide Database connectivity details like username or password from hibernate configuration file:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"></property>
<property name="url" value="jdbc:oracle:thin:#localhost:1521:xe"></property>
<property name="username" value="system"></property>
<property name="password" value="tiger"></property>
</bean>
You have 2 options
1) Use JNDI
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jee="http://www.springframework.org/schema/jee" xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee.xsd">
<jee:jndi-lookup id="dbDataSource" jndi-name="jdbc/DatabaseName" expected-type="javax.sql.DataSource" />
2) Use a property placeholder
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location">
<value>classpath:database.properties</value>
<!-- this example use the system parameter configlocation, add -Dconfiglocation -->
<value>file:${configlocation}/database.properties</value>
</property>
</bean>
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"></property>
<property name="url" value="jdbc:oracle:thin:#localhost:1521:xe"></property>
<property name="username" value="${jdbc.user}"></property>
<property name="password" value="${jdbc.password}"></property>
</bean>
The password can be in the file database.properties which is only in the classpath for the running application and not visible to all.
I have several camel microservices deployed to tomcat as individual war files. Each war file contains a unique camel route. One of the services accepts soap requests and puts a message onto a queue on IBM MQ. This works without a problem. The services which get these messages don't appear to pull the messages off the queue and forward them on. Each war file has the following as its IBM MQ configuration
<!-- Configure IBM MQ connection factory -->
<bean id="ibmMqConnectionFactory" class="com.ibm.mq.jms.MQConnectionFactory">
<property name="transportType" value="1"/>
<property name="hostName" value="${ibm.mq.host}"/>
<property name="port" value="${ibm.mq.port}"/>
<property name="channel" value="${ibm.mq.channel}"/>
<property name="queueManager" value="${ibm.qm.name}"/>
</bean>
<bean id="ibmMqPooledConnectionFactory" class="org.apache.activemq.jms.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="connectionFactory" ref="ibmMqConnectionFactory"/>
<property name="maxConnections" value="8"/>
<property name="maximumActiveSessionPerConnection" value="1"/>
<property name="expiryTimeout" value="30000"/>
<property name="idleTimeout" value="30000"/>
<property name="blockIfSessionPoolIsFull" value="false"/>
</bean>
<bean id="ibmMqConnectionFactorySecured" class="org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter">
<property name="targetConnectionFactory" ref="ibmMqPooledConnectionFactory"/>
<property name="username" value="${ibm.mq.username}"/>
<property name="password" value="${ibm.mq.password}"/>
</bean>
<bean id="ibmMqTransactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="ibmMqConnectionFactorySecured" />
</bean>
<bean id="ibmMqJmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="ibmMqConnectionFactorySecured"/>
<property name="concurrentConsumers" value="4"/>
<property name="transacted" value="true"/>
<property name="transactionManager" ref="ibmMqTransactionManager" />
<property name="cacheLevelName" value="CACHE_NONE" />
</bean>
<bean id="ibmMq" class="org.apache.camel.component.jms.JmsComponent">
<property name="configuration" ref="ibmMqJmsConfig"/>
</bean>
An example of one of the camel routes which pulls a message of a queue looks like
<route id="cancellation-service-route" errorHandlerRef="deadLetterErrorHandler">
<from uri="{{cancellationService.queue}}"/>
<unmarshal ref="requestCancellationRequest"/>
<to uri="bean:cancellationTranslator?method=transform"/>
...
</route>
The route starts but does not pull messages off the queue.
I resolved the truncation issue by doing <convertBodyTo type="String" /> before sending the message to the queue. I suspect it is because the message is an UTF-8 ecoded xml message.
I have been looking at these for days and still haven't have mush success.
I have install and run activeMQ on a server. Created a queue called "testUpdate". I also have another queue on another server, let call this "forward" and this is on the IBM MQ (WMQ).
So we have testUpdate on ActiveMQ and forward on WMQ. I want the message on forward queue to be placed into testUpdate queue. Trying to use Camel for this process.
So the setup in ActiveMQ has an XML(activemq.xml) that I can place spring beans and configure to do the routing. In this xml, I added http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd to the xmlns and then added
import resource="camel.xml"
In this new xml, I have the following:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<description>Example Camel Route</description>
<from uri="mqseries:forward"/>
<to uri="activemq:testUpdate"/>
</route>
</camelContext>
<bean id="mqseries" class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="transportType" value="1"/>
<property name="hostName" value="172.00.12.21/>
<property name="port" value="xyza"/>
<property name="queueManager" value="manager"/>
<property name="channel" value="srvcChannel"/>
</bean>
<!--
Lets configure some Camel endpoints
http://camel.apache.org/components.html
-->
<!-- configure the camel activemq component to use the current broker -->
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" >
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://amq-broker?create=false"/>
<property name="userName" value="${activemq.username}"/>
<property name="password" value="${activemq.password}"/>
</bean>
</property>
</bean>
</beans>
I added the jar file com.ibm.mq.allclient.jar into the lib folder of ActiveMQ. When running the program, I am getting this exception:
java.lang.ClassNotFoundException: javax.jms.JMSRuntimeException
It sounds like I am missing the websphere jar file? Is this correct?
I use both IBM MQ and ActiveMQ on a regular basis. This example below will show you some sample configuration options. Please make sure you configure these to your own use cases.
//ActiveMQ connection factory
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" destroy-method="doStop">
<property name="configuration">
<bean class="org.apache.camel.component.jms.JmsConfiguration">
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
<property name="acceptMessagesWhileStopping" value="true" />
<property name="acknowledgementModeName" value="CLIENT_ACKNOWLEDGE" />
<property name="cacheLevelName" value="CACHE_CONSUMER" />
<property name="connectionFactory">
<bean class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="1" />
<property name="MaximumActiveSessionPerConnection" value="500" />
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${activemq1.brokerUrl}" />
<property name="userName" value="${activemq1.username}" />
<property name="password" value="${activemq1.password}" />
<property name="redeliveryPolicy">
<bean class="org.apache.activemq.RedeliveryPolicy">
<property name="maximumRedeliveries" value="-1" />
</bean>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</property>
</bean>
//IBM MQ connection factory
<bean id="ibmmq" class="org.apache.camel.component.jms.JmsComponent" destroy-method="doStop">
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
<property name="connectionFactory">
<bean class="org.springframework.jms.connection.SingleConnectionFactory" destroy-method="destroy">
<constructor-arg>
<bean class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="transportType" value="1" />
<property name="channel" value="${channel}" />
<property name="hostName" value="${hostname}" />
<property name="port" value="${port}" />
<property name="queueManager" value="${queueManager}" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>
You can use jms component to consume the message from IBM MQ and use activemq component to post it to ActiveMQ
have a look at the following links
http://camel.apache.org/jms.html
http://camel.apache.org/activemq.html
Thanks,
Gibu
You can use users camel extra component https://github.com/camel-extra/camel-extra/blob/master/components/camel-wmq/README.md to get messages from wmq without using the JMS wrapping and then send messages to ActiveMQ with custom camel component https://camel.apache.org/components/latest/activemq-component.html. See also a similar topic Apache Camel with IBM MQ.
I have quartz setup with clustering. Each of the quartz cluster node servers have two distinct entries in the qrtz_scheduler_state table. I can't understand it. Has anyone seen this?
More info:
Original configuration that has issue
Spring 3.0.3
Quartz 1.8.5
Weblogic 10.3.5
SQL Server 2005
New configuration that also has the issue
Spring 3.1.1
Quartz 2.1.5
Weblogic 10.3.5
SQL Server 2005
quarts-config.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jee="http://www.springframework.org/schema/jee"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/jee
http://www.springframework.org/schema/jee/spring-jee-2.0.xsd">
<!-- Defining this bean starts the triggers -->
<bean id="scheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<ref bean="pendingTransactionJob" />
<ref bean="assetJob" />
</list>
</property>
<property name="dataSource" ref="wfbisDataSource" />
<property name="transactionManager" ref="wfbisTransactionManager" />
<property name="overwriteExistingJobs" value="true" />
<property name="autoStartup" value="false" />
<property name="schedulerContextAsMap">
<map>
<entry key="pendingTransactionService" value-ref="pendingTransactionService" />
<entry key="assetService" value-ref="assetService" />
</map>
</property>
<property name="quartzProperties">
<props>
<prop key="org.quartz.scheduler.instanceName">ClusteredScheduler</prop>
<prop key="org.quartz.scheduler.instanceId">AUTO</prop>
<prop key="org.quartz.threadPool.class">org.quartz.simpl.SimpleThreadPool</prop>
<prop key="org.quartz.threadPool.threadCount">6</prop>
<prop key="org.quartz.jobStore.clusterCheckinInterval">300000</prop>
<prop key="org.quartz.threadPool.threadPriority">5</prop>
<prop key="org.quartz.jobStore.misfireThreshold">60000</prop>
<prop key="org.quartz.jobStore.class">org.quartz.impl.jdbcjobstore.JobStoreTX</prop>
<prop key="org.quartz.jobStore.driverDelegateClass">org.quartz.impl.jdbcjobstore.MSSQLDelegate</prop>
<prop key="org.quartz.jobStore.tablePrefix">QRTZ_</prop>
<prop key="org.quartz.jobStore.isClustered">true</prop>
<prop key="org.quartz.jobStore.useProperties">false</prop>
</props>
</property>
</bean>
<bean id="pendingTransactionJob" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="pendingTransactionJobDetail" />
<property name="cronExpression" value="0 0/5 * * * ?" />
</bean>
<bean name="pendingTransactionJobDetail" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="com.wellsfargo.trust.message.ScheduledJob" />
<property name="jobDataAsMap">
<map>
<entry key="serviceName" value="pendingTransactionService" />
</map>
</property>
</bean>
<bean id="assetJob" class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="assetJobDetail" />
<property name="cronExpression" value="0 0/5 * * * ?" />
</bean>
<bean id="assetJobDetail" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="com.wellsfargo.trust.message.ScheduledJob" />
<property name="jobDataAsMap">
<map>
<entry key="serviceName" value="assetService" />
</map>
</property>
</bean>
Delete table entries, restart the cluster and see if double entries are created again. It could happen that those entries were created but not deleted by some errors.