Camel pollenrich on mybatis-select looping endlessly - apache-camel

I'm using a quartz trigger quite sporadically to invoke a mybatis select
from( "quartz:'once a minute' )
.pollEnrich( "mybatis://selectItems?statementType=SelectList" )
.process ( ...
the select appears correct and runs in a sql-debugger pad correctly. (currently returns no results because table is empty).
When the quartz trigger invokes it starts the poll-enrich, reports no error but just keeps continually re-running the select on the database.
There seems no end it it, it just keeps polling and polling as fast as it can. It never reaches the process point. I have no idea what could be going wrong with it or a way around this problem. Why it is running the select more than once ... it is as though something fails and it comes tries again, but it doesnt give any error message and normally mybatis is very verbose with errors
I have no idea how to fix this...
Here is a dump of just a small section of the logs
DEBUG 2013-10-30 14:10:28,913 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:28,913 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:28,913 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,428 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:29,428 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,428 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,944 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:29,944 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,944 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:30,460 [DBIM] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:30,460 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:30,460 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]

Use enrich instead of pollEnrich

Related

Excpetion while configuring camel jmx in jboss

I am deploying came jmxAgent in Jboss 6.3.0. Follwing is the code snippet:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation=" http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd ">
<context:annotation-config />
<!-- start : Files import to camel context-->
<!-- Active MQ message broker -->
<!-- <import resource="classpath:META-INF/spring/message-borker/integration-message-broker.xml" /> -->
<import resource="file:${FIS_HOME}/COMMON/database-context.xml" />
<import resource="file:${FIS_HOME}/CA/job-scheduer-route-context.xml" />
<import resource="file:${FIS_HOME}/CA/jobs-process-route-context.xml" />
<import resource="file:${FIS_HOME}/CA/pci-jobs-process-route-context.xml" />
<import resource="file:${FIS_HOME}/CA/archiving_reports.xml" />
<import resource="file:${FIS_HOME}/COMMON/powercard-sqlFetcher-route-context.xml" />
<import resource="file:${FIS_HOME}/COMMON/rest/rest-context.xml" />
<!-- End : Files import to camel context-->
<camelContext xmlns="http://camel.apache.org/schema/spring" trace="true" streamCache="true"
lazyLoadTypeConverters="true" id="main">
<properties>
<property key="CamelLogEipName" value="com.hp.rcu.rtsp.filemanager.route"/>
</properties>
<propertyPlaceholder id="properties"
location="file:${FIS_HOME}/COMMON/routes.properties"
xmlns="http://camel.apache.org/schema/spring"/>
<jmxAgent id="agent" createConnector="true"/>
</camelContext>
An although the deployment is successful i am able to see the below exception in the logs:
13:50:47,163 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) Exception in thread "Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://205.239.212.91:1099/jmxrmi/camel" java.lang.UnsupportedOperationException: JBAS011859: Naming context is read-only
13:50:47,164 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at org.jboss.as.naming.WritableServiceBasedNamingStore.requireOwner(WritableServiceBasedNamingStore.java:155)
13:50:47,165 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at org.jboss.as.naming.WritableServiceBasedNamingStore.bind(WritableServiceBasedNamingStore.java:63)
13:50:47,165 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at org.jboss.as.naming.NamingContext.bind(NamingContext.java:248)
13:50:47,165 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at org.jboss.as.naming.InitialContext$DefaultInitialContext.bind(InitialContext.java:268)
13:50:47,167 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at org.jboss.as.naming.NamingContext.bind(NamingContext.java:257)
13:50:47,167 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at javax.naming.InitialContext.bind(InitialContext.java:425)
13:50:47,168 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at javax.naming.InitialContext.bind(InitialContext.java:425)
13:50:47,168 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at javax.management.remote.rmi.RMIConnectorServer.bind(RMIConnectorServer.java:644)
13:50:47,168 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:427)
13:50:47,169 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at org.apache.camel.management.DefaultManagementAgent$1.run(DefaultManagementAgent.java:570)
13:50:47,170 ERROR [stderr] (Camel Thread #2 - Camel Thread #1 - JMXConnector: service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel) at java.lang.Thread.run(Thread.java:748)
Also I am not able to connect to the above url from jconsole.
Note: The jboss container is not in my local and in another server. So the JMX url is service:jmx:rmi:///jndi/rmi://uspnsvulx785.elabs.svcs.lxp.com:1099/jmxrmi/camel
Camel version: 2.23.2
You can check this - http://camel.apache.org/camel-jmx.html.
There is a section for JBoss here which will help you configure it.
Ok for now i have a workaround. Rather than implementing jmx connector in Camel, i have used the connector of Jboss itself. You can see all the mbeans of jboss within it. You will find all the mbeans of camel under org.apache.camel.
It will not fix the above exception but you can monitor camel routes and processors.
Also you can remove the jmxAgent line given below from the camel-context.xml if using the jboss connector.
<jmxAgent id="agent" createConnector="true"/>

Java client fails to connect to local Flink cluster

I am trying out a small program with a local Flink cluster, setup according to the instructions here. The sample wordcount program runs fine, but when I attempt to run my own program, it stalls and fails while connecting to the job manager. This is Flink 1.5 with JDK 1.8
The relevant part of the code is
FlinkPipelineOptions options = PipelineOptionsFactory.as(FlinkPipelineOptions.class);
options.setStreaming(true);
options.setFlinkMaster("localhost:6123");
options.setRunner(FlinkRunner.class);
I start the cluster with start-cluster.sh, and I can see the two processes (job and task managers) are running. The logs on Flink don't have much. On the client side, after turning on debug, I can see the following
18:43:20.507 [flink-akka.actor.default-dispatcher-4] INFO akka.remote.Remoting - Remoting started; listening on addresses :[akka.tcp://flink#talonx:38183]
18:43:20.511 [main] INFO org.apache.flink.client.program.StandaloneClusterClient - Actor system started at akka.tcp://flink#talonx:38183
18:43:20.511 [main] INFO org.apache.flink.client.program.StandaloneClusterClient - Submitting job with JobID: dbf63281771465550fd3598b2b67b91f. Waiting for job completion.
Submitting job with JobID: dbf63281771465550fd3598b2b67b91f. Waiting for job completion.
18:43:20.521 [flink-akka.actor.default-dispatcher-4] INFO org.apache.flink.runtime.client.JobSubmissionClientActor - Received SubmitJobAndWait(JobGraph(jobId: dbf63281771465550fd3598b2b67b91f)) but there is no connection to a JobManager yet.
18:43:20.522 [flink-akka.actor.default-dispatcher-4] INFO org.apache.flink.runtime.client.JobSubmissionClientActor - Received job test-talonx-0618131319-b721a69a (dbf63281771465550fd3598b2b67b91f).
18:43:20.523 [flink-akka.actor.default-dispatcher-4] INFO org.apache.flink.runtime.client.JobSubmissionClientActor - Disconnect from JobManager null.
After a while, I get the following exception on the client
19:03:19.396 [main] ERROR org.apache.beam.runners.flink.FlinkRunner - Pipeline execution failed
org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Couldn't retrieve the JobExecutionResult from the JobManager.
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:492)
at org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:105)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:456)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:449)
at org.apache.flink.streaming.api.environment.RemoteStreamEnvironment.executeRemotely(RemoteStreamEnvironment.java:212)
at org.apache.flink.streaming.api.environment.RemoteStreamEnvironment.execute(RemoteStreamEnvironment.java:176)
at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:126)
at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:115)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)
at Test.main(Test.java:106)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Couldn't retrieve the JobExecutionResult from the JobManager.
at org.apache.flink.runtime.client.JobClient.awaitJobResult(JobClient.java:300)
at org.apache.flink.runtime.client.JobClient.submitJobAndWait(JobClient.java:387)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:481)
... 10 common frames omitted
Caused by: org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException: Lost connection to the JobManager.
at org.apache.flink.runtime.client.JobClientActor.handleMessage(JobClientActor.java:219)
at org.apache.flink.runtime.akka.FlinkUntypedActor.handleLeaderSessionID(FlinkUntypedActor.java:104)
at org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(FlinkUntypedActor.java:71)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
What might be missing here?

camel Atomikos XA continually reconecting

I have recently moved my routes over to use XA, I am not sure if this is normal behavior or a problem but I am seeing continual reconnecting in the log and the transaction logs are growing when there is no activity as such on the jms queue.
The logs show the following
16:14:19.777 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.datasource.xa.XAResourceTransaction-logInfo - XAResource.start ( 31302E3136322E3230382E3130322E746D30303030323030303032:31302E3136322E3230382E3130322E746D32 , XAResource.TMNOFLAGS ) on resource ConnectionFactoryTESTQM represented by XAResource instance com.ibm.mq.jmqi.JmqiXAResource#27520e8f
16:14:20.806 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: calling getTransacted on JMS driver session...
16:14:20.806 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsMessageConsumerProxy-logInfo - atomikos MessageConsumer proxy for com.ibm.mq.jms.MQQueueReceiver#690215c4: close...
16:14:20.822 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.datasource.xa.XAResourceTransaction-logInfo - XAResource.end ( 31302E3136322E3230382E3130322E746D30303030323030303032:31302E3136322E3230382E3130322E746D32 , XAResource.TMSUCCESS ) on resource ConnectionFactoryTESTQM represented by XAResource instance com.ibm.mq.jmqi.JmqiXAResource#27520e8f
16:14:20.838 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: closing session atomikos xa session proxy for resource ConnectionFactoryTESTQM - is terminated ? false
16:14:20.838 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: close()...
16:14:20.838 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.CompositeTransactionImp-logInfo - commit() done (by application) of transaction 10.162.208.102.tm0000200002
16:14:20.978 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.datasource.xa.XAResourceTransaction-logInfo - XAResource.commit ( 31302E3136322E3230382E3130322E746D30303030323030303032:31302E3136322E3230382E3130322E746D32 , true ) on resource ConnectionFactoryTESTQM represented by XAResource instance com.ibm.mq.jmqi.JmqiXAResource#27520e8f
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.BaseTransactionManager-logInfo - createCompositeTransaction ( 300000 ): created new ROOT transaction with id 10.162.208.102.tm0000300002
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosConnectionFactoryBean-logInfo - AtomikosConnectionFactoryBean 'ConnectionFactoryTESTQM': createConnection()...
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosConnectionFactoryBean-logInfo - AtomikosConnectionFactoryBean 'ConnectionFactoryTESTQM': init...
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: creating XA-capable session...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: calling hashCode on JMS driver...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: calling hashCode on JMS driver...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: calling start on JMS driver...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: calling createQueue on JMS driver session...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: calling createConsumer on JMS driver session com.ibm.mq.jms.MQXASession#41a4d0d0
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsMessageConsumerProxy-logInfo - atomikos MessageConsumer proxy for com.ibm.mq.jms.MQQueueReceiver#3cb81a5f: receive ( 1000 )...
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsMessageConsumerProxy-logInfo - atomikos MessageConsumer proxy for com.ibm.mq.jms.MQQueueReceiver#3cb81a5f: receive ( 1000 , null )...
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.CompositeTransactionImp-logInfo - registerSynchronization ( com.atomikos.jms.ConsumerProducerSupport$JmsRequeueSynchronization#4d70e77 ) for transaction 10.162.208.102.tm0000300002
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.CompositeTransactionImp-logInfo - addParticipant ( XAResourceTransaction: 31302E3136322E3230382E3130322E746D30303030333030303032:31302E3136322E3230382E3130322E746D33 ) for transaction 10.162.208.102.tm0000300002
If this is normal then I will just surpress the logging but I figured it worth checking.
thanks
Paul

org.apache.camel.component.file.GenericFileOperationFailedException - Cannot list directory:

Their is a similar posting that claims to have the answer but I'm still getting the error after putting step -> stepwise=false in my camel route. -> exception
14:35:33,649 WARN [org.apache.camel.component.file.remote.RemoteFilePollingConsumerPollStrategy] (Camel (camel-1) thread #13 - sftp://myftp:22) Trying to recover by discon
necting from remote server forcing a re-connect at next poll: sftp://myUser#myftp:22
14:35:33,654 WARN [org.apache.camel.component.file.remote.SftpConsumer] (Camel (camel-1) thread #13 - sftp://myftp:22) Consumer Consumer[sftp://myftp:22?delay
=1h&delete=true&doneFileName=done&password=xxxxxx&sortBy=ignoreCase%3Afile%3Aname&stepwise=false&username=myUser] failed polling endpoint: Endpoint[sftp://myftp:22?delay=1h
&delete=true&doneFileName=done&password=xxxxxx&sortBy=ignoreCase%3Afile%3Aname&stepwise=false&username=myUser]. Will try again at next poll. Caused by: [org.apache.camel.component.file.
GenericFileOperationFailedException - Cannot list directory: .]: org.apache.camel.component.file.GenericFileOperationFailedException: Cannot list directory: .
at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:583) [camel-ftp-2.13.2.jar:2.13.2]
at org.apache.camel.component.file.remote.SftpConsumer.doPollDirectory(SftpConsumer.java:90) [camel-ftp-2.13.2.jar:2.13.2]
at org.apache.camel.component.file.remote.SftpConsumer.pollDirectory(SftpConsumer.java:52) [camel-ftp-2.13.2.jar:2.13.2]
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:119) [camel-core-2.13.2.jar:2.13.2]
at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187) [camel-core-2.13.2.jar:2.13.2]
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114) [camel-core-2.13.2.jar:2.13.2]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [rt.jar:1.7.0_65]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [rt.jar:1.7.0_65]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [rt.jar:1.7.0_65]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65]
Caused by: 4:
at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1660) [jsch-0.1.49.jar:]
at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1466) [jsch-0.1.49.jar:]
at org.apache.camel.component.file.remote.SftpOperations.listFiles(SftpOperations.java:574) [camel-ftp-2.13.2.jar:2.13.2]
... 12 more
Caused by: java.io.IOException: Pipe closed
at java.io.PipedInputStream.read(PipedInputStream.java:308) [rt.jar:1.7.0_65]
at com.jcraft.jsch.Channel$MyPipedInputStream.updateReadSide(Channel.java:344) [jsch-0.1.49.jar:]
at com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1483) [jsch-0.1.49.jar:]
... 14 more
here is my route ->
sftp://myftp:22?delay
=1h&delete=true&doneFileName=done&password=xxxxxx&sortBy=ignoreCase%3Afile%3Aname&stepwise=false&username=myUser
Yeah, there is java.io.IOException: Pipe closed.
I just checked the code of camel-ftp, it has the code to check the connection, but it's hard to know if the connection is still opened if we don't send some bytes to server socket.
The solution could be force the ftp client send some ping message to check if the connection is still open.
If you want to set a very long delay for your ftp pulling, the work around way is setting disconnect option to be true to force ftp endpoint restart a new connection when it pull the directory again from the FTP server.
I just created a JIRA for it.

Camel sftp component start failing after upgrade

I upgraded camel from 2.8.0 to 2.10.3 version and my sftp component started failing.
Afaik, sftp component uses stepwise by default and had no trouble to change directories previous to upgrade.
Below is a log entry that I see on camel/jsch
20130113 15:28:54,509 INFO [Camel (camel-1) thread #0 -
file://../test_outbound/] RemoteFileProducer[193]: Connected and
logged in to:
Endpoint[sftp://sftp-user#localhost/incoming?disconnect=true&privateKeyFile=xyz.ppk&privateKeyFilePassphrase=**]
20130113 15:28:54,528 WARN [Camel (camel-1) thread #0 -
file://../test_outbound/] RemoteFileProducer[71]: Writing file failed
with: Cannot change directory to: home 20130113 15:28:54,528 DEBUG
[Camel (camel-1) thread #0 - file://../test_outbound/]
RemoteFileProducer[86]: Disconnecting from:
Endpoint[sftp://sftp-user#localhost/incoming?disconnect=true&privateKeyFile=xyz.ppk&privateKeyFilePassphrase=**]
20130113 15:28:54,529 INFO [Camel (camel-1) thread #0 -
file://../test_outbound/] SftpOperations$JSchLogger[259]: JSCH ->
Disconnecting from localhost port 22 20130113 15:28:54,529 INFO
[Connect thread localhost session] SftpOperations$JSchLogger[259]:
JSCH -> Caught an exception, leaving main loop due to socket closed
Not sure why this is happening? Any ideas appreciated, thanks.
cd works with v2.8.0
20130114 18:42:52,956 INFO [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] RemoteFileConsumer[133]: Connected and
logged in to: sftp://user#host:22 20130114 18:42:52,956 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpOperations[339]:
getCurrentDirectory() 20130114 18:42:52,959 TRACE [Camel (camel-1)
thread #0 - sftp://user#host/outgoing] SftpConsumer[68]:
doPollDirectory from absolutePath: outgoing, dirName: null 20130114
18:42:52,959 TRACE [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] SftpOperations[348]:
changeCurrentDirectory(outgoing) 20130114 18:42:52,959 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpOperations[389]:
Changing directory: outgoing 20130114 18:42:52,966 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpConsumer[81]:
Polling directory: outgoing 20130114 18:42:52,966 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpOperations[415]:
listFiles(.) 20130114 18:42:52,982 TRACE [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] SftpConsumer[94]: Found 2 in directory:
outgoing 20130114 18:42:52,982 TRACE [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] SftpOperations[348]:
changeCurrentDirectory(/home/sftp-user) 20130114 18:42:52,982 TRACE
[Camel (camel-1) thread #0 - sftp://user#host/outgoing]
SftpOperations[389]: Changing directory: / 20130114 18:42:52,988 TRACE
[Camel (camel-1) thread #0 - sftp://user#host/outgoing]
SftpOperations[389]: Changing directory: home 20130114 18:42:52,994
TRACE [Camel (camel-1) thread #0 - sftp://user#host/outgoing]
SftpOperations[389]: Changing directory: sftp-user 20130114
18:42:53,000 DEBUG [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] GenericFileConsumer[84]: Took 0.044 seconds
to poll: outgoing
and fails with 2.10.3
20130114 19:15:53,530 INFO [Camel (camel-1) thread #0 -
file://../test_outbound/] RemoteFileProducer[193]: Connected and
logged in to:
Endpoint[sftp://user#host/incoming?disconnect=true&knownHostsFile=known_hosts&privateKeyFile=sftp-user.ppk&privateKeyFilePassphrase=**&separator=Auto]
20130114 19:15:53,530 TRACE [Camel (camel-1) thread #0 -
file://../test_outbound/] SftpOperations[305]:
buildDirectory(incoming,false) 20130114 19:15:53,530 TRACE [Camel
(camel-1) thread #0 - file://../test_outbound/] SftpOperations[372]:
getCurrentDirectory() 20130114 19:15:53,670 TRACE [Camel (camel-1)
thread #0 - file://../test_outbound/] SftpOperations[381]:
changeCurrentDirectory(/home/sftp-user) 20130114 19:15:53,670 TRACE
[Camel (camel-1) thread #0 - file://../test_outbound/]
SftpOperations[372]: getCurrentDirectory() 20130114 19:15:53,670 TRACE
[Camel (camel-1) thread #0 - file://../test_outbound/]
SftpOperations[430]: Changing directory: \ 20130114 19:15:53,749 TRACE
[Camel (camel-1) thread #0 - file://../test_outbound/]
SftpOperations[430]: Changing directory: home 20130114 19:15:53,796
WARN [Camel (camel-1) thread #0 - file://../test_outbound/]
RemoteFileProducer[71]: Writing file failed with: Cannot change
directory to: home
Looks like separator is bad but I tried all 3 options (Windows,Unix,Auto)... none worked.
Accepted as a bug and fixed, see
https://issues.apache.org/jira/browse/CAMEL-5970

Resources