Camel file component for network disks - file

I need to save file to network drive using system service for Apache tomcat and get all the time error:
Exception: org.apache.camel.component.file.GenericFileOperationFailedException: Cannot store file: u:\xxx.txt
My route sample:
<route id="myRoute">
<from uri="quartz:myQuartz?cron=0+0+*+?+*+MON-FRI"/>
<camel:doTry>
<bean method="getData" ref="myService"/>
<marshal ref="bindyFixedDataformat"/>
<to uri="file:u:?fileName=xxx.txt&autoCreate=false&fileExist=Append"/>
<to uri="file://c:/XXX/files?fileName=xxx-${date:now:yyyyMMdd}.txt"/>
<camel:bean method="setProcessed" ref="myService"/>
<camel:doCatch>
<exception>java.io.IOException</exception>
<camel:log message="Network drive (U:) is not available, please renew connection!" loggingLevel="ERROR"/>
<camel:bean method="setFailed" ref="myService"/>
</camel:doCatch>
<camel:doCatch>
<exception>java.lang.Exception</exception>
<camel:log message="Unexpected error has occured!" loggingLevel="ERROR" />
<camel:bean method="setFailed" ref="myService"/>
</camel:doCatch>
</camel:doTry>
</route>
'U' disk is the mapped network disk. But the same route with my second partition disk 'D' file creates successfully.
I'm generating WAR file and run it with Local System Service by Apache Tomcat.
Interesting, that before service, when I run the same project with eclipse by maven - copying to U disk was happening successfully.

The local system service user probably can't see your mapped drive 'U'. Solution: use UNC paths.
Note that the user running Tomcat will need permission to access the network path - often a local system service user won't have. In that case you can run Tomcat as a user who does have permission, i.e. by changing the user in the 'Log on' tab of the service.

Related

Create route in Apache ServiceMix Bundle:List state is Grace Period

I create a route on apache servicemix and put it in deploy. When the server started and I check the bundle:list it says that my route is in Grace Period and it's not working.
Currently I'm routing ftp to file server and do this format.
<camelContext xmlns="http://camel.apache.org/schema/blueprint">
<route>
<from uri="FTP:username#//hostname /DB?password=password"/>
<to uri="file:\\storage\folder\anotherFolder\anotherFolder\anotherFolder\"/>
</route>
</camelContext>
I did not input the exact credentials I use but that is the format that I used.
Is there any problem with my code or it's on my apache service mix? Please help I doing this for a week.
P.S. It works when its file to file server
You can install ftp using feature:install camel-ftp .

Limit netty4 tcp endpoint to a single instance

I have a mock endpoint at 8001 that will echo anything provided to it.
I have an http endpoint that will submit the end of the URL to the mock endpoint, and provide a response from the endpoint's response.
That works fine.
The challenge is, I want the http route to use only 1 tcp connection to the 8001 endpoint.
I created a worker group as explained elsewhere, and set the worker count to 1. Looking through the source code, I'm thinking this approach should work.
However, when I do this bash command:
for a in {1..5}; do curl "http://localhost:8080/upstream/REQUESTNUM$a" > $a.txt & done;
I see multiple connections to 8001. I would have expected the http endpoint requests would have to share a single pool worker, but that doesn't seem to be the case.
Perhaps I am missing something, or perhaps there is another way to accomplish my goal of using only 1 back-end tcp connection for all the http requests.
How do I accomplish it?
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<camelContext
xmlns="http://camel.apache.org/schema/spring">
<route id="mockUpstream">
<from
uri="netty4:tcp://localhost:8001?sync=true&textline=true&keepAlive=true&disconnect=false&reuseChannel=true" />
<log message="Incoming to upstream: ${body}" />
<transform>
<simple>${body}</simple>
</transform>
</route>
<route id="httpServer">
<from
uri="netty4-http:http://0.0.0.0:8080/upstream?matchOnUriPrefix=true" />
<!-- optional just use CamelHttpQuery from header, for full query -->
<log
message="Incoming http command: ${in.headers[CamelHttpPath]}" />
<transform>
<simple>${in.headers[CamelHttpPath]}</simple>
</transform>
<to
uri="netty4:tcp://localhost:8001?workerGroup=#sharedPool&sync=true&textline=true&keepAlive=true&disconnect=false&reuseChannel=true" />
<transform>
<simple>${body}</simple>
</transform>
</route>
</camelContext>
<bean id="poolBuilder"
class="org.apache.camel.component.netty4.NettyWorkerPoolBuilder">
<property name="workerCount" value="1" />
</bean>
<bean id="sharedPool" class="io.netty.channel.EventLoopGroup"
factory-bean="poolBuilder" factory-method="build"
destroy-method="shutdown">
</bean>
</beans>
Looking at the logs, with TRACE level logging, I saw the NettyProducer's pool was indeed set to use 1 max active connection, but the NettyProducer was allowed 100 idle connections. I changed the following line and it is now behaving as expected:
<to
uri="netty4:tcp://localhost:8001?workerGroup=#sharedPool&sync=true&textline=true&keepAlive=true&disconnect=false&reuseChannel=true&producerPoolMaxActive=1&producerPoolMaxIdle=1" />
I assumed the "producer" settings were only good for the producer side (netty in mock host route) of the connection, but it looks like they can be used by the consumer end (netty in http route), too.
edit: I confused the terms producer and consumer and got that backwards above. Seeing that the "to" element is producing a request for something to consume, the producer* parameters make sense for (netty in http route). The (netty in mock host route) is the consumer of requests.

Spring Integration SFTP inbound adapter unable to copy files

I have a SFTP inbound channel set up to poll a remote sftp server and copy files to a local directory. When it runs, it gives me a 'Permission denied' error, but in the log file it correctly mentions the file name. So it appears to be able to correctly list the contents of the remote path, but is unable to read the files.
I haven't been able to figure out what the access issue is exactly. When i fiddled with it on a test server i could see I would get the same issue if the sftp user had at least r-x access on the remote dir, but no access on the files themselves. However on the live server where i get the issue, the user does have this required level of access.
Running the sftp command copies the files without any issues:
/usr/bin/sftp -2 -i KEYFILE USER#SERVER:REMOTEDIR/FILEPATTERN* LOCALDIR
Here is how i have the SFTP channel in my Spring Integration config:
<int:poller default="true" fixed-rate="${fixed.rate}" />
<bean id="sftpClientFactory"
class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
<property name="host" value="${sftp.inbound.channel.host}" />
<property name="port" value="${sftp.inbound.channel.availableServerPort}" />
<property name="user" value="${sftp.inbound.channel.userid}" />
<property name="password" value="${sftp.inbound.channel.password}" />
<property name="privateKey" value="file:///${sftp.inbound.channel.server.key}"></property>
</bean>
<int-sftp:inbound-channel-adapter id="sftpInbound"
channel="sftpChannel" session-factory="sftpClientFactory"
filename-pattern="${input.file.format}" auto-create-local-directory="true"
delete-remote-files="false" remote-directory="${sftp.inbound.channel.remote.directory}"
local-directory="${sftp.inbound.channel.local.directory}">
</int-sftp:inbound-channel-adapter>
<int:channel id="sftpChannel">
<int:queue />
</int:channel>
The project is using Spring Integration version 4.0.4-RELEASE
This is the full exception trace. The file name gets correctly logged at the placeholder <FILENAME>
ERROR 9860 --- [ask-scheduler-2] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Problem occurred while synchronizing remote to local directory
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:209)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:192)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:149)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:298)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:292)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.springframework.messaging.MessagingException: Failure occurred while copying from remote to local directory
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.copyFileToLocalDirectory(AbstractInboundFileSynchronizer.java:238)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:177)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 20 more
Caused by: org.springframework.core.NestedIOException: failed to read file <FILENAME>; nested exception is 3: Permission denied
at org.springframework.integration.sftp.session.SftpSession.read(SftpSession.java:132)
at org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.copyFileToLocalDirectory(AbstractInboundFileSynchronizer.java:231)
... 24 more
Caused by: 3: Permission denied
at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2846)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1313)
at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1266)
at org.springframework.integration.sftp.session.SftpSession.read(SftpSession.java:128)
... 25 more
I'll appreciate if anyone can help me figure out what I may be missing.

Rename file on remote scp failure launched by Camel ssh component

When using ssh component to launch a remote scp command I must set the body. The ssh component is using the body as the command to be executed remotely.
What I want to achieve is this:
Monitor folder for files. The "from" is using delete=true as I don't want to keep files once route is completed.
Copy the file on the DMZ server using sftp.
Launch a scp on the DMZ server using ssh.
In case of failure, scp does return code greater than 0, rename the file with ".failed".
Unfortunately using ssh require that I overwrite the body and I'm loosing the content of my file. I tried to use inOnly, sending to another route, but it does not make a copy of the file but probably a copy of the file pointer. I can't use wireTap because by the time it complete the remote execution, the route is completed and the file removed. I cannot use (I think) a temporary variable because the file can be up to gigabyte in size.
I'm using Camel version 2.12.1 running under Karaf 2.3.2. I'm trying to use blueprint XML only to avoid java coding as much as possible. Below a sample. The result, in case of non-zero exit code from ssh component, is the file contain my remote command.
<route id="RemoteTest1">
<!-- 1. Monitor for incoming files -->
<from uri="file:///data/karaf/tmp/RemoteTest1/?delete=true"/>
<!-- 2. Copy file on DMZ server -->
<to uri="sftp:username#myDmz.com//home/RemoteTest1/?privateKeyFile=myPrivateKey.pk"/>
<!-- 3. Execute scp remotely -->
<setHeader headerName="remoteCommand">
<simple>scp /home/RemoteTest1/${file:name} someuser#acme.com:${file:name}</simple>
</setHeader>
<doTry>
<inOnly uri="direct-vm:remoteExec"/>
<log message="Success"/>
<doCatch>
<exception>java.lang.Exception</exception>
<!-- 4. In case of failure rename the file by adding .failed -->
<to uri="file:?fileName=${file:absolute.path}.failed"/>
<log message="Failed"/>
</doCatch>
</doTry>
</route>
<route id="remoteExec">
<from uri="direct-vm:remoteExec"/>
<setBody>
<simple>${header.remoteCommand}</simple>
</setBody>
<to uri="ssh://username#myDmz.com?certResource=file:resources/keys/myPrivateKey.pk"/>
<!-- Throw exception on remote error -->
<choice>
<when>
<simple>${header.CamelSshExitValue} != '0'</simple>
<throwException ref="remoteExecException"/>
</when>
<otherwise>
<log message="scp completed normally"/>
</otherwise>
</choice>
</route>
<bean id="remoteExecException" class="java.lang.Exception">
<argument value="Failed remote execution" />
</bean>
Is there a solution? Any help or tip would be appreciated.
Thanks

To share a local host for go gae?

we are two people and i want my friend can use my local host but it is giving error
Runtime Error
Description: An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.
Details: To enable the details of this specific error message to be viewable on remote machines, please create a tag within a "web.config" configuration file located in the root directory of the current web application. This tag should then have its "mode" attribute set to "Off".
<configuration>
<system.web>
<customErrors mode="Off"/>
</system.web>
</configuration>
Notes: The current error page you are seeing can be replaced by a custom error page by modifying the "defaultRedirect" attribute of the application's configuration tag to point to a custom error page URL.
<!-- Web.Config Configuration File -->
<configuration>
<system.web>
<customErrors mode="RemoteOnly" defaultRedirect="mycustompage.htm"/>
</system.web>
</configuration>
needless to say
we are using python 2.7 and appcfg.py cmd to run app
In order to access your sites using local address, either use RemoteDeskTop, or remotedesktop shaing by chrome .
if your friend is in same network then he can browse your application using your LocalIp address.
http://192.168.10.20/YourApplication.aspx
If your Local Ip is not accessible in other computers of same network, then change your network to Home Network, and Add allow your IIS in Firewall.

Resources