I have a SFTP inbound channel set up to poll a remote sftp server and copy files to a local directory. When it runs, it gives me a 'Permission denied' error, but in the log file it correctly mentions the file name. So it appears to be able to correctly list the contents of the remote path, but is unable to read the files.
I haven't been able to figure out what the access issue is exactly. When i fiddled with it on a test server i could see I would get the same issue if the sftp user had at least r-x access on the remote dir, but no access on the files themselves. However on the live server where i get the issue, the user does have this required level of access.
Running the sftp command copies the files without any issues:
/usr/bin/sftp -2 -i KEYFILE USER#SERVER:REMOTEDIR/FILEPATTERN* LOCALDIR
Here is how i have the SFTP channel in my Spring Integration config:
<int:poller default="true" fixed-rate="${fixed.rate}" />
<bean id="sftpClientFactory"
class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
<property name="host" value="${sftp.inbound.channel.host}" />
<property name="port" value="${sftp.inbound.channel.availableServerPort}" />
<property name="user" value="${sftp.inbound.channel.userid}" />
<property name="password" value="${sftp.inbound.channel.password}" />
<property name="privateKey" value="file:///${sftp.inbound.channel.server.key}"></property>
</bean>
<int-sftp:inbound-channel-adapter id="sftpInbound"
channel="sftpChannel" session-factory="sftpClientFactory"
filename-pattern="${input.file.format}" auto-create-local-directory="true"
delete-remote-files="false" remote-directory="${sftp.inbound.channel.remote.directory}"
local-directory="${sftp.inbound.channel.local.directory}">
</int-sftp:inbound-channel-adapter>
<int:channel id="sftpChannel">
<int:queue />
</int:channel>
The project is using Spring Integration version 4.0.4-RELEASE
This is the full exception trace. The file name gets correctly logged at the placeholder <FILENAME>
ERROR 9860 --- [ask-scheduler-2] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Problem occurred while synchronizing remote to local directory
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:209)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:192)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:149)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:298)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:292)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.springframework.messaging.MessagingException: Failure occurred while copying from remote to local directory
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.copyFileToLocalDirectory(AbstractInboundFileSynchronizer.java:238)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:177)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 20 more
Caused by: org.springframework.core.NestedIOException: failed to read file <FILENAME>; nested exception is 3: Permission denied
at org.springframework.integration.sftp.session.SftpSession.read(SftpSession.java:132)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.copyFileToLocalDirectory(AbstractInboundFileSynchronizer.java:231)
... 24 more
Caused by: 3: Permission denied
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2846)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1313)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1266)
at org.springframework.integration.sftp.session.SftpSession.read(SftpSession.java:128)
... 25 more
I'll appreciate if anyone can help me figure out what I may be missing.
Related
I'm configuring a datasource to use windows authentication with SQL Server.
I put the DLL sqljdbc_auth.dll in C:\Program Files\XXXX\wildfly-10.1.0.Final\bin
Stating Wildfly using standalone.bat works fine.
But using windows service I got this error:
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Cannot open database "Scope_Build36" requested by the login. The login failed. ClientConnectionId:831c2f7f-4352-4467-b54f-a6eb1369d6e9
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:217)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:251)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:81)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$1LogonProcessor.complete(SQLServerConnection.java:2825)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:3079)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:2360)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$100(SQLServerConnection.java:43)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:2346)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:6276)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1793)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1404)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:1068)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:904)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:451)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1014)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:321)
... 44 more
DataSource configuration:
<datasource jta="true" jndi-name="java:/datasources/PortalScopeWeb_Scope" pool-name="PortalScopeWeb_Scope" enabled="true" use-ccm="true">
<connection-url>jdbc:sqlserver://RI001421D\SQLEXPRESS:1433;databasename=Scope_Build36;integratedSecurity=true</connection-url>
<driver>sqlServer</driver>
<security>
<security-domain>securityDomainPortalScopeWeb_Scope</security-domain>
</security>
<validation>
<check-valid-connection-sql>SELECT 1</check-valid-connection-sql>
<validate-on-match>true</validate-on-match>
<background-validation>false</background-validation>
</validation>
</datasource>
Putting DDL in folder C:\Program Files\XXXX\wildfly-10.1.0.Final\bin\service\amd64 don't solved the problem.
you can use a .udl file to test and verify your connection.
In you desktop create a new .txt file, and rename it like "test.udl". Now, open it, and you can configure and test your connection. When you have the connection ready, you can open the .udl file with notepad and copy connection string.
here you have a reference: udl reference
hope this help!
I'm using Camel 2.15.2 with Apache Commons Net 3.3 on Java 8 deployed into a Tomcat container.
The issue is that consistently after processing just over 200 files (> 4000 files in the directory) the route stops, the FTP client disconnects and the following message is logged out:
[ogs.sharp-stream.com:21/root/] FtpConsumer WARN Error processing file RemoteFile[route/to/file] due to File operation failed: Connection closed without indication.. Code: 421. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - File operation failed: Connection closed without indication.. Code: 421]
org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: Connection closed without indication.. Code: 421
at org.apache.camel.component.file.remote.FtpOperations.getCurrentDirectory(FtpOperations.java:713)
at org.apache.camel.component.file.remote.FtpOperations.retrieveFileToFileInLocalWorkDirectory(FtpOperations.java:440)
at org.apache.camel.component.file.remote.FtpOperations.retrieveFile(FtpOperations.java:310)
at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:384)
at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137)
at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:211)
at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:175)
at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187)
at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.commons.net.ftp.FTPConnectionClosedException: Connection closed without indication.
at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:317)
at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:294)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:483)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:608)
at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:582)
at org.apache.commons.net.ftp.FTP.pwd(FTP.java:1454)
at org.apache.commons.net.ftp.FTPClient.printWorkingDirectory(FTPClient.java:2658)
at org.apache.camel.component.file.remote.FtpOperations.getCurrentDirectory(FtpOperations.java:709)
... 15 more
This is the URI used at the begining of the related route.
As you can tell from the URI I'm also using a FileIdempotentRepository. It's defined like this
<property name="fileStore" value="target/fileidempotent/.filestore1.dat" />
<property name="maxFileStoreSize" value="512000" />
<property name="cacheSize" value="250" />
</bean>
Any ideas why the connection might be closing before all files are processed?
You need to add following parameter to you ftp/ftps url
disconnect=true
This will result in disconnecting from your remote host after file(s) are transferred and a fresh connection being attempted whenever the route is triggered again .
(instead of using the stale one , which might have been closed by remote host/firewall)
I assume that your connection is closed from e.g. your fire wall.
Try to set the parameter maxMessagesPerPoll to e.g. 150.
I am trying to deploy a fairly basic Spring+Hibernate web app to my newly created app. (I am new to openshift).
I have followed this example from the knowledge base.
But after deploying the application i am getting
..
java.sql.SQLException: Illegal connection port value '${env.OPENSHIFT_MYSQL_DB_PORT}'
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2018)
..
I have defined the datasource in my applicationcontext.xml
This is the section in my applicationContext.xml that I define the deta shource (eli is the database name)
<bean id="dataSource"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"
p:driverClassName="${jdbc.driverClassName}"
p:url="jdbc:mysql://${env.OPENSHIFT_MYSQL_DB_HOST}:${env.OPENSHIFT_MYSQL_DB_PORT}/eli" p:username="${jdbc.username}"
p:password="${jdbc.password}"/>
Any idea what I am doing wrong?
Thanks
The error you're receiving seems to indicate that the property replacement strings in your Spring XML configuration aren't actually being processed by Spring. You can add a PropertyPlaceholderConfigurer to your configuration to enable processing from the system environment:
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="searchSystemEnvironment" value="true" />
...
</bean>
I am attempting to set up multiple datasources in my standalone.xml file. I have a connection to DB2 up and working fine. I am having trouble getting the SQL Server connection to work.
From standalone.xml:
<datasource jndi-name="java:/jdbc/syteline" pool-name="SYTELINEMSSDS" enabled="true" use-java-context="true" spy="true">
<connection-url>jdbc:sqlserver://SLSQL:1433;databaseName=npsl_pilot_app;</connection-url>
<driver-class>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class>
<driver>sqljdbc4.jar</driver>
<pool>
<min-pool-size>5</min-pool-size>
<max-pool-size>10</max-pool-size>
<prefill>true</prefill>
<use-strict-min>false</use-strict-min>
</pool>
<security>
<user-name>myUser</user-name>
<password>myPass</password>
</security>
<timeout>
<idle-timeout-minutes>15</idle-timeout-minutes>
</timeout>
</datasource>
I have the sqljdbc4.jar deployed in my standalone\deployments directory and it is deploying. I have tested the connection URL and credentials by copy/pasting it into my application and creating a JDBC connection within my code. That connection works properly. Setting the connection up as a datasource in standalone.xml gives me the following:
15:20:00,326 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-15) JBAS010400: Bound data source [java:/jdbc/syteline]
15:20:00,684 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (JCA PoolFiller) IJ000610: Unable to fill pool: javax.resource.ResourceException: Could not create connection
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:758) [ironjacamar-core-impl-1.0.7.Final.jar:1.0.7.Final]
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:703) [ironjacamar-core-impl-1.0.7.Final.jar:1.0.7.Final]
at org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97) [ironjacamar-core-impl-1.0.7.Final.jar:1.0.7.Final]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_23]
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'dna'. ClientConnectionId:4e4e0e7f-3e23-4a13-9c49-37e33aeb4613
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:254)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:84)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:2908)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:2234)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:41)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:2220)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1326)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:991)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:827)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1012)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
... 5 more
15:20:01,385 WARN [org.jboss.as.server.deployment] (MSC service thread 1-14) Class Path entry jcifs.jar in "/C:/jbossdir/jboss7/standalone/deployments/speed2.war/WEB-INF/lib/jtds-1.2.5.jar" does not point to a valid jar for a Class-Path reference.
I have not been able to figure out the Login Failed message.
Is the JBoss server on the same computer as the test application that can connect properly?
Just as a followup, I gave up on the Microsoft driver and used JTDS instead. Works fine.
I have two questions regarding Arquillian and Tomcat:
-My arquillian tests fail with the following error message:
org.jboss.jsfunit.example.hellojsf.HelloJSFTest Time elapsed: 0 sec
<<< ERROR! org.jboss.arquillian.container.spi.ConfigurationException:
Unable to connect to Tomcat manager. The server command
(/deploy?path=%2Ftest) failed with responseCode (401) and
responseMessage (Non-Autorisé). Please make sure that you provided
correct credentials to an user which is able to access Tomcat manager
application. These credentials can be specified in the Arquillian
container configuration as "user" and "pass" properties. The user must
have appripriate role specified in tomcat-users.xml file.
FYI my arquillian.xml file is as follows:
<arquillian xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://jboss.org/schema/arquillian
http://jboss.org/schema/arquillian/arquillian-1.0.xsd">
<engine>
<property name="deploymentExportPath">target/</property>
<property name="jmxPort">8099</property>
<property name="user">admin</property>
<property name="pass">admin75</property>
</engine>
<defaultProtocol type="Servlet 2.5" />
<container qualifier="tomcat-remote">
<configuration>
<property name="jmxPort">8099</property>
<property name="user">admin</property>
<property name="pass">admin75</property>
</configuration>
</container>
</arquillian>
I am trying to adapt the sample app for tomcat 6. Can anyone please help?
-When will Arquillian support tomcat 7?
Regards,
J.
tomcat-users.xml:
<tomcat-users>
<role rolename="manager"/>
<role rolename="tomcat"/>
<role rolename="admin"/>
<role rolename="role1"/>
<user username="tomcat" password="tomcat" roles="tomcat"/>
<user username="both" password="tomcat" roles="tomcat,role1"/>
<user username="admin" password="admin75" roles="manager,admin"/>
</tomcat-users>
This message
Unable to connect to Tomcat manager. The server command (/deploy?path=%2Ftest) failed with responseCode (401) and responseMessage (Non-Autorisé).
indicates that one of the following is true:
the tomcat-users.xml file used by your Tomcat installation does not have the admin user (that you've specified in arquillian.xml),
or the admin user is not mapped to the manager role in Tomcat 6, or the manager-script role in Tomcat 7.
When will Arquillian support tomcat 7?
Arquillian supports Tomcat 7, as an emebedded or a managed container. The documentation is not up to date (as of now), but the configuration parameters are more or less the same as the embedded and managed equivalents in Tomcat 6. The artifact Id to use for
a managed Tomcat 7 instance is org.jboss.arquillian.container:arquillian-tomcat-managed-7.
an embedded Tomcat 7 instance is org.jboss.arquillian.container:arquillian-tomcat-embedded-7.
As of today, 1.0.0.CR2 is the latest stable release. You can use 1.0.0.Final-SNAPSHOT, if you want to work against the development build.
Also, you can omit several redundant properties from your arquillian.xml file. A cleaner configuration would look like:
<arquillian xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://jboss.org/schema/arquillian
http://jboss.org/schema/arquillian/arquillian-1.0.xsd">
<engine>
<property name="deploymentExportPath">target/</property>
</engine>
<container qualifier="tomcat-remote">
<configuration>
<property name="jmxPort">8099</property>
<property name="user">admin</property>
<property name="pass">admin75</property>
</configuration>
</container>
</arquillian>