I have recently moved my routes over to use XA, I am not sure if this is normal behavior or a problem but I am seeing continual reconnecting in the log and the transaction logs are growing when there is no activity as such on the jms queue.
The logs show the following
16:14:19.777 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.datasource.xa.XAResourceTransaction-logInfo - XAResource.start ( 31302E3136322E3230382E3130322E746D30303030323030303032:31302E3136322E3230382E3130322E746D32 , XAResource.TMNOFLAGS ) on resource ConnectionFactoryTESTQM represented by XAResource instance com.ibm.mq.jmqi.JmqiXAResource#27520e8f
16:14:20.806 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: calling getTransacted on JMS driver session...
16:14:20.806 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsMessageConsumerProxy-logInfo - atomikos MessageConsumer proxy for com.ibm.mq.jms.MQQueueReceiver#690215c4: close...
16:14:20.822 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.datasource.xa.XAResourceTransaction-logInfo - XAResource.end ( 31302E3136322E3230382E3130322E746D30303030323030303032:31302E3136322E3230382E3130322E746D32 , XAResource.TMSUCCESS ) on resource ConnectionFactoryTESTQM represented by XAResource instance com.ibm.mq.jmqi.JmqiXAResource#27520e8f
16:14:20.838 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: closing session atomikos xa session proxy for resource ConnectionFactoryTESTQM - is terminated ? false
16:14:20.838 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: close()...
16:14:20.838 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.CompositeTransactionImp-logInfo - commit() done (by application) of transaction 10.162.208.102.tm0000200002
16:14:20.978 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.datasource.xa.XAResourceTransaction-logInfo - XAResource.commit ( 31302E3136322E3230382E3130322E746D30303030323030303032:31302E3136322E3230382E3130322E746D32 , true ) on resource ConnectionFactoryTESTQM represented by XAResource instance com.ibm.mq.jmqi.JmqiXAResource#27520e8f
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.BaseTransactionManager-logInfo - createCompositeTransaction ( 300000 ): created new ROOT transaction with id 10.162.208.102.tm0000300002
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosConnectionFactoryBean-logInfo - AtomikosConnectionFactoryBean 'ConnectionFactoryTESTQM': createConnection()...
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosConnectionFactoryBean-logInfo - AtomikosConnectionFactoryBean 'ConnectionFactoryTESTQM': init...
16:14:21.009 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: creating XA-capable session...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: calling hashCode on JMS driver...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: calling hashCode on JMS driver...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsConnectionProxy-logInfo - atomikos connection proxy for resource ConnectionFactoryTESTQM: calling start on JMS driver...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: calling createQueue on JMS driver session...
16:14:21.134 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsXaSessionProxy-logInfo - atomikos xa session proxy for resource ConnectionFactoryTESTQM: calling createConsumer on JMS driver session com.ibm.mq.jms.MQXASession#41a4d0d0
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsMessageConsumerProxy-logInfo - atomikos MessageConsumer proxy for com.ibm.mq.jms.MQQueueReceiver#3cb81a5f: receive ( 1000 )...
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.jms.AtomikosJmsMessageConsumerProxy-logInfo - atomikos MessageConsumer proxy for com.ibm.mq.jms.MQQueueReceiver#3cb81a5f: receive ( 1000 , null )...
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.CompositeTransactionImp-logInfo - registerSynchronization ( com.atomikos.jms.ConsumerProducerSupport$JmsRequeueSynchronization#4d70e77 ) for transaction 10.162.208.102.tm0000300002
16:14:21.150 [Camel (camel-1) thread #0 - JmsConsumer[AAA.TESTQ1]] INFO com.atomikos.icatch.imp.CompositeTransactionImp-logInfo - addParticipant ( XAResourceTransaction: 31302E3136322E3230382E3130322E746D30303030333030303032:31302E3136322E3230382E3130322E746D33 ) for transaction 10.162.208.102.tm0000300002
If this is normal then I will just surpress the logging but I figured it worth checking.
thanks
Paul
Related
I'm using apache camel to make a ftp client for downloading some files to some local directory. The program reads a xml file to get the name of the file that will be fetched from the ftp.The program seems to work except that the files downloaded are corrupted. Right now I'm trying to download some image files but the ones I get are 14.9Kb and corrupted, no error message shown.
This is my code:
main
public void main() throws FileNotFoundException {
BasicConfigurator.configure();
RutaFtp routeBuilder = new RutaFtp();
CamelContext ctx = new DefaultCamelContext();
try {
ctx.addRoutes(routeBuilder);
ctx.start();
Thread.sleep(10000);
ctx.stop();
}
catch (Exception e) {
e.printStackTrace();
}
}
camel route:
from("file:./?fileName=Datos.xml&noop=true")
.split(xpath("//Datos/imagen/text()"))
.setProperty("rutaArchivo", this.body())
.log(LoggingLevel.INFO, "imagen: ${body}")
.process(ExtraerNombre).to("direct:ftp").end();
from("direct:ftp")
.pollEnrich("ftp://"+user+"#"+ftp+"/?password="+password+"&recursive=true&passiveMode=true&fileName=${body}&delete="+borrado+"")
.to("file:C:/outputFolder?flatten=true").end();
}
I've tried using the streamDownload parameter but tha prevents the files to be downloaded (I don't know why)
.pollEnrich("ftp://"+user+"#"+ftp+"/?password="+password+"&recursive=true&passiveMode=true&streamDownload=true&fileName=${body}&delete="+borrado+"")
console log:
INFO | Apache Camel 2.15.1.redhat-621084 (CamelContext: camel-1) is
starting 0 [main] INFO org.apache.camel.impl.DefaultCamelContext -
Apache Camel 2.15.1.redhat-621084 (CamelContext: camel-1) is starting
INFO | JMX is enabled 10 [main] INFO
org.apache.camel.management.ManagedManagementStrategy - JMX is
enabled INFO | Loaded 185 type converters 208 [main] INFO
org.apache.camel.impl.converter.DefaultTypeConverter - Loaded 185
type converters INFO | AllowUseOriginalMessage is enabled. If access
to the original message is not needed, then its recommended to turn
this option off as it may improve performance. 395 [main] INFO
org.apache.camel.impl.DefaultCamelContext - AllowUseOriginalMessage
is enabled. If access to the original message is not needed, then its
recommended to turn this option off as it may improve performance.
INFO | StreamCaching is not in use. If using streams then its
recommended to enable stream caching. See more details at
http://camel.apache.org/stream-caching.html 395 [main] INFO
org.apache.camel.impl.DefaultCamelContext - StreamCaching is not in
use. If using streams then its recommended to enable stream caching.
See more details at http://camel.apache.org/stream-caching.html INFO
| Endpoint is configured with noop=true so forcing endpoint to be
idempotent as well 395 [main] INFO
org.apache.camel.component.file.FileEndpoint - Endpoint is configured
with noop=true so forcing endpoint to be idempotent as well INFO |
Using default memory based idempotent repository with cache max size:
1000 395 [main] INFO org.apache.camel.component.file.FileEndpoint -
Using default memory based idempotent repository with cache max size:
1000 INFO | Route: route1 started and consuming from:
Endpoint[file://./?fileName=Datos.xml&noop=true] 502 [main] INFO
org.apache.camel.impl.DefaultCamelContext - Route: route1 started and
consuming from: Endpoint[file://./?fileName=Datos.xml&noop=true] INFO
| Route: route2 started and consuming from: Endpoint[direct://ftp] 504
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route2
started and consuming from: Endpoint[direct://ftp] INFO | Total 2
routes, of which 2 is started. 504 [main] INFO
org.apache.camel.impl.DefaultCamelContext - Total 2 routes, of which
2 is started. INFO | Apache Camel 2.15.1.redhat-621084 (CamelContext:
camel-1) started in 0.504 seconds 507 [main] INFO
org.apache.camel.impl.DefaultCamelContext - Apache Camel
2.15.1.redhat-621084 (CamelContext: camel-1) started in 0.504 seconds INFO | Created default XPathFactory
com.sun.org.apache.xpath.internal.jaxp.XPathFactoryImpl#5434283f 1533
[Camel (camel-1) thread #0 - file://./] INFO
org.apache.camel.builder.xml.XPathBuilder - Created default
XPathFactory
com.sun.org.apache.xpath.internal.jaxp.XPathFactoryImpl#5434283f INFO
| imagen: ftp://190.0.56.190:8021/pruebasumman/conductor/71708375.jpg
1635 [Camel (camel-1) thread #0 - file://./] INFO route1 - imagen:
ftp://190.0.56.190:8021/pruebasumman/conductor/71708375.jpg INFO |
Apache Camel 2.15.1.redhat-621084 (CamelContext: camel-1) is shutting
down 10521 [main] INFO org.apache.camel.impl.DefaultCamelContext -
Apache Camel 2.15.1.redhat-621084 (CamelContext: camel-1) is shutting
down INFO | Starting to graceful shutdown 2 routes (timeout 300
seconds) 10524 [main] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Starting to graceful
shutdown 2 routes (timeout 300 seconds) INFO | Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 300
seconds. 10524 [Camel (camel-1) thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 300
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 299 seconds. 11525 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 299
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 298 seconds. 12528 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 298
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 297 seconds. 13529 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 297
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 296 seconds. 14540 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 296
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 295 seconds. 15555 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 295
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 294 seconds. 16568 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 294
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 293 seconds. 17569 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 293
seconds. INFO | Waiting as there are still 3 inflight and pending
exchanges to complete, timeout in 292 seconds. 18574 [Camel (camel-1)
thread #2 - ShutdownTask] INFO
org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are
still 3 inflight and pending exchanges to complete, timeout in 292
seconds.
Thanks in advance.
Download image file in binary mode
By default, Camel FTP is downloading file by ASCII mode.
Add binary=true into your ftp route will turn from ASCII mode to binary mode
I have multiple camelContexts created and would like each camelContext to use their own custom threadpool. However, the log shows all my camelContexts are using the same threadpool. What am I missing?
Main Camel Context
<camelContext id="routeLoader_route">
(no threadpool defined)
<route id="RouteCreator" >
<from uri="file://jsonFilePath" />
<bean ref="routeMonitor" method="loadJsontoCreateRoute" />
</route>
(some other routes defined)
</camelContext>
The method "loadJsontoCreateRoute" will read three json files and then create three routes to connect from
endpoint "file://xxx1/out" to endpoint "direct-vm:out.test"
endpoint "file://xxx2/out" to endpoint "direct-vm:out.test"
endpoint "file://xxx3/out" to endpoint "direct-vm:out.test"
Another camel context
<camelContext id="test_out_route">
<threadPoolProfile id="outTestThreadPoolProfile" defaultProfile="true" poolSize="1" maxPoolSize="1" maxQueueSize="1000" rejectedPolicy="CallerRuns"/>
<route id="outboundTestingRouter">
<from uri="direct-vm:out.test"/>
<doTry>
<log message="Outbound Test -- START" loggingLevel="INFO" />
<recipientList>
<method ref="outTestBean" method="dynamicRoute" />
</recipientList>
<doFinally>
<log message="Outbound Test-- END" loggingLevel="INFO" />
<stop/>
</doFinally>
</doTry>
</route>
(Some other routes defined)
</camelContext>
The dynamicRoute method will return a uri for ftp component
Log
20160623 09:48:04.297 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.524 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:04.526 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.527 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.634 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:04.636 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.652 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:04.653 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.749 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:04.749 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.827 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:04.827 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.890 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:04.937 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:04.937 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:04.999 [Camel (routeLoader_route) thread #24 - file://xxx3/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:05.140 [Camel (routeLoader_route) thread #5 - file://xxx1/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:05.358 [Camel (routeLoader_route) thread #5 - file://xxx1/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:05.358 [Camel (routeLoader_route) thread #5 - file://xxx1/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:05.469 [Camel (routeLoader_route) thread #5 - file://xxx1/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:05.471 [Camel (routeLoader_route) thread #5 - file://xxx1/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:05.593 [Camel (routeLoader_route) thread #5 - file://xxx1/out] INFO outboundTestingRouter - Outbound Test -- END
20160623 09:48:05.905 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- START
20160623 09:48:05.999 [Camel (routeLoader_route) thread #17 - file://xxx2/out] INFO outboundTestingRouter - Outbound Test -- END
Line 3 to line 10 of the log shows that there is more than 1 outboundTestingRouter route is concurrently running while my threadpool outTestThreadPoolProfile pool size is limit to 1
This indicates that the threadpool outTestThreadPoolProfile is not being used by outboundTestingRouter route
What I do want is to limit the max. number of concurrent usage of route outboundTestingRouter.
Hmm looks like that should work. What about the logging statement says your not using your threadpool?
Things to note:
1) Your route needs to be leveraging a thread pool to make off of the default template.
2) You can always manually assign a threadpool to a component typically with syntax similar to: executorServiceRef="outTestThreadPoolProfile" but make sure you check your component's documentation
3) If your default threadpool profile isn't working you can simply use the standard ThreadPool tag to make an isolated threadpool that can be assigned directly to a component.
My camel SFTP consumer connects to server every 10 seconds and during every connect it prints huge log messages as mentioned here.
02-05-2016 20:48:34,441 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> kex: client:
hmac-md5,hmac-sha1,hmac-sha1-96,hmac-md5-96
02-05-2016 20:48:34,441 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> kex: client:
none
02-05-2016 20:48:34,441 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> kex: client:
none
02-05-2016 20:48:34,441 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> kex: client:
02-05-2016 20:48:34,441 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> kex: client:
02-05-2016 20:48:34,441 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> kex:
server->client aes128-ctr hmac-md5 none
02-05-2016 20:48:34,441 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> kex:
client->server aes128-ctr hmac-md5 none
02-05-2016 20:48:34,443 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
SSH_MSG_KEXDH_INIT sent
02-05-2016 20:48:34,444 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> expecting
SSH_MSG_KEXDH_REPLY
02-05-2016 20:48:34,762 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
ssh_rsa_verify: signature true
02-05-2016 20:48:34,762 WARN
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> Permanently
added '11.11.11.11' (RSA) to the list of known hosts.
02-05-2016 20:48:34,763 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
SSH_MSG_NEWKEYS sent
02-05-2016 20:48:34,763 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
SSH_MSG_NEWKEYS received
02-05-2016 20:48:34,763 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
SSH_MSG_SERVICE_REQUEST sent
02-05-2016 20:48:35,079 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
SSH_MSG_SERVICE_ACCEPT received
02-05-2016 20:48:35,391 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
Authentications that can continue:
publickey,keyboard-interactive,password
02-05-2016 20:48:35,391 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> Next
authentication method: publickey
02-05-2016 20:48:35,391 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH ->
Authentications that can continue: password
02-05-2016 20:48:35,391 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> Next
authentication method: password
02-05-2016 20:48:35,707 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> Authentication
succeeded (password).
02-05-2016 20:48:36,662 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] Connected to
sftp://username#11.11.11.11:22
02-05-2016 20:48:36,662 INFO
(org.apache.camel.component.file.remote.SftpConsumer) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] Connected and logged
in to: sftp://username#11.11.11.11:22
02-05-2016 20:48:38,539 INFO
(org.apache.camel.component.file.remote.SftpOperations) [Camel
(camel-99) thread #117 -
sftp://username:******#11.11.11.11:22/download] JSCH -> Disconnecting
from 11.11.11.11 port 22
How can I stop these messages from appearing in the log?
If you are using a logger like log4j, you can simply turn off the JSCH logging with the following line in your logger config:
log4j.logger.org.apache.camel.component.file.remote.SftpOperations = OFF
As an alternative to change the logger config, the current Camel sFTP documentation mentions few options you can set on the endpoint:
jschLoggingLevel
readLockLoggingLevel
runLoggingLevel
For example you can set:
sftp://user#myserver.mydomain.net:9921/path/to/folder?binary=true&jschLoggingLevel=WARN&readLockLoggingLevel=INFO&runLoggingLevel=WARN
You can tweak each category to fit your needs directly in the endpoint without changing the configuration of your logger.
I'm using a quartz trigger quite sporadically to invoke a mybatis select
from( "quartz:'once a minute' )
.pollEnrich( "mybatis://selectItems?statementType=SelectList" )
.process ( ...
the select appears correct and runs in a sql-debugger pad correctly. (currently returns no results because table is empty).
When the quartz trigger invokes it starts the poll-enrich, reports no error but just keeps continually re-running the select on the database.
There seems no end it it, it just keeps polling and polling as fast as it can. It never reaches the process point. I have no idea what could be going wrong with it or a way around this problem. Why it is running the select more than once ... it is as though something fails and it comes tries again, but it doesnt give any error message and normally mybatis is very verbose with errors
I have no idea how to fix this...
Here is a dump of just a small section of the logs
DEBUG 2013-10-30 14:10:28,913 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:28,913 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:28,913 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,428 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:29,428 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,428 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,944 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:29,944 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:29,944 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:30,460 [DBIM] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Openning JDBC Connection
DEBUG 2013-10-30 14:10:30,460 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
DEBUG 2013-10-30 14:10:30,460 [EXEC] {Camel (camel-1) thread #7 - mybatis://selectItems} JdbcTransaction - Closing JDBC Connection [jdbc:oracle:thin:#X:1234:Y, UserName=A, Oracle JDBC driver]
Use enrich instead of pollEnrich
I upgraded camel from 2.8.0 to 2.10.3 version and my sftp component started failing.
Afaik, sftp component uses stepwise by default and had no trouble to change directories previous to upgrade.
Below is a log entry that I see on camel/jsch
20130113 15:28:54,509 INFO [Camel (camel-1) thread #0 -
file://../test_outbound/] RemoteFileProducer[193]: Connected and
logged in to:
Endpoint[sftp://sftp-user#localhost/incoming?disconnect=true&privateKeyFile=xyz.ppk&privateKeyFilePassphrase=**]
20130113 15:28:54,528 WARN [Camel (camel-1) thread #0 -
file://../test_outbound/] RemoteFileProducer[71]: Writing file failed
with: Cannot change directory to: home 20130113 15:28:54,528 DEBUG
[Camel (camel-1) thread #0 - file://../test_outbound/]
RemoteFileProducer[86]: Disconnecting from:
Endpoint[sftp://sftp-user#localhost/incoming?disconnect=true&privateKeyFile=xyz.ppk&privateKeyFilePassphrase=**]
20130113 15:28:54,529 INFO [Camel (camel-1) thread #0 -
file://../test_outbound/] SftpOperations$JSchLogger[259]: JSCH ->
Disconnecting from localhost port 22 20130113 15:28:54,529 INFO
[Connect thread localhost session] SftpOperations$JSchLogger[259]:
JSCH -> Caught an exception, leaving main loop due to socket closed
Not sure why this is happening? Any ideas appreciated, thanks.
cd works with v2.8.0
20130114 18:42:52,956 INFO [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] RemoteFileConsumer[133]: Connected and
logged in to: sftp://user#host:22 20130114 18:42:52,956 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpOperations[339]:
getCurrentDirectory() 20130114 18:42:52,959 TRACE [Camel (camel-1)
thread #0 - sftp://user#host/outgoing] SftpConsumer[68]:
doPollDirectory from absolutePath: outgoing, dirName: null 20130114
18:42:52,959 TRACE [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] SftpOperations[348]:
changeCurrentDirectory(outgoing) 20130114 18:42:52,959 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpOperations[389]:
Changing directory: outgoing 20130114 18:42:52,966 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpConsumer[81]:
Polling directory: outgoing 20130114 18:42:52,966 TRACE [Camel
(camel-1) thread #0 - sftp://user#host/outgoing] SftpOperations[415]:
listFiles(.) 20130114 18:42:52,982 TRACE [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] SftpConsumer[94]: Found 2 in directory:
outgoing 20130114 18:42:52,982 TRACE [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] SftpOperations[348]:
changeCurrentDirectory(/home/sftp-user) 20130114 18:42:52,982 TRACE
[Camel (camel-1) thread #0 - sftp://user#host/outgoing]
SftpOperations[389]: Changing directory: / 20130114 18:42:52,988 TRACE
[Camel (camel-1) thread #0 - sftp://user#host/outgoing]
SftpOperations[389]: Changing directory: home 20130114 18:42:52,994
TRACE [Camel (camel-1) thread #0 - sftp://user#host/outgoing]
SftpOperations[389]: Changing directory: sftp-user 20130114
18:42:53,000 DEBUG [Camel (camel-1) thread #0 -
sftp://user#host/outgoing] GenericFileConsumer[84]: Took 0.044 seconds
to poll: outgoing
and fails with 2.10.3
20130114 19:15:53,530 INFO [Camel (camel-1) thread #0 -
file://../test_outbound/] RemoteFileProducer[193]: Connected and
logged in to:
Endpoint[sftp://user#host/incoming?disconnect=true&knownHostsFile=known_hosts&privateKeyFile=sftp-user.ppk&privateKeyFilePassphrase=**&separator=Auto]
20130114 19:15:53,530 TRACE [Camel (camel-1) thread #0 -
file://../test_outbound/] SftpOperations[305]:
buildDirectory(incoming,false) 20130114 19:15:53,530 TRACE [Camel
(camel-1) thread #0 - file://../test_outbound/] SftpOperations[372]:
getCurrentDirectory() 20130114 19:15:53,670 TRACE [Camel (camel-1)
thread #0 - file://../test_outbound/] SftpOperations[381]:
changeCurrentDirectory(/home/sftp-user) 20130114 19:15:53,670 TRACE
[Camel (camel-1) thread #0 - file://../test_outbound/]
SftpOperations[372]: getCurrentDirectory() 20130114 19:15:53,670 TRACE
[Camel (camel-1) thread #0 - file://../test_outbound/]
SftpOperations[430]: Changing directory: \ 20130114 19:15:53,749 TRACE
[Camel (camel-1) thread #0 - file://../test_outbound/]
SftpOperations[430]: Changing directory: home 20130114 19:15:53,796
WARN [Camel (camel-1) thread #0 - file://../test_outbound/]
RemoteFileProducer[71]: Writing file failed with: Cannot change
directory to: home
Looks like separator is bad but I tried all 3 options (Windows,Unix,Auto)... none worked.
Accepted as a bug and fixed, see
https://issues.apache.org/jira/browse/CAMEL-5970