I need to create a route in Camel in Spring DSL that waits for a certain file (done_.ctl) to exists on a SFTP and then download all other files by a defined pattern (.gz).
I tried using pollEnrich which did not work.
<route id="sftpDownloadCSVFiles">
<from uri="{{sftp-done-fromUri}}"/>
<log message="#### Found the signal file: '${header[CamelFileNameOnly]}'"/>
get all files from sftp://{{sftp-inbound-fromUri}}
<log message="All files downloaded."/>
</route>
I would like to download all *.gz files to the local disk and remove the done file.
Related
The Apache Camel File component not working properly in Windows7, where as its working in Linux without any Problem.
My Requirement:
After the file processing, the files must be deleted from the Directory. In Windows, because of .camelLock the files are not deleting properly.
After Multiple attempts, then only Apache camel can delete the file from the Directory. If the attemt failed to delete the file from the Directory then its throwing an exception.
If I have only a File in the Directory, it's working without any Problem, but if i have multiple Files then it's throwing an Exception.
Application Environment:
I deployed the Apache camel application in Tomcat Server.
Apache Camel version: 2.17.1
Apache Camel Route:
<from uri="file:///var/opt/irs/message?delete=true" />
<to uri="direct:file.storage.original" />
In Windows I am receiving following Error:
1|2017-11-14 17:56:34,828|11-01-41|default|WARN
|yes||o.a.c.c.f.GenericFileOnCompletion|file.analysis.input|Error
during commit. Exchange[ID-51741-1510678404569-9-22]. Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException -
Cannot delete file: GenericFile[C:\var\opt\irs\message\661.zip]]
org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot delete file: GenericFile[C:\var\opt\irs\message\661.zip] at
org.apache.camel.component.file.strategy.GenericFileDeleteProcessStrategy.commit(GenericFileDeleteProcessStrategy.java:89)
at
org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:127)
Similar Problems:
Camel 2.15 file locks ,
Camel 2.14.2 not deleting files on Windows ,deleting moving files
In this case, I forgot to close file input stream. so that's why I had a problem in Windows.
I am using InputStream unnecessarily for the Filecomponent. I removed it, everything working file.
InputStream input = CamelContextHelper.convertTo(context, InputStream.class, body);
If anybody has same problem, just close the file InputStream properly or Cross-check your code and try to replace it with other proper alternative solution.
Alternative Solution:
In case,if you are unable to find open streams in your code, then apply below parameters to Apache camel route. The performance would be better.
readLockCheckInterval=1&readLockTimeout=3
In my case Performance much better in windows.
Is there any technique to read a property file from a different jar file resource folder in Apache Camel
I have two project one is ESB-tracker and other is ESB-common. queue-config.properties is place under ESB-common/pk/com/herman/common/resources/.
ESB-common is a simple java project and ESB-tracker in a fuse-integration project in which im trying to read this property file using the below line.
<propertyPlaceholder id="properties" location="pk/com/herman/common/resources/queue-config.properties"/>
I have deployed the both jar files in jboss fuse but i got below excpetion
Caused by: java.io.FileNotFoundException: Properties file pk/com/herman/common/resources/queue-config.properties not found in classpath
I used FTP component of camel but it could not download the files that I want.
<camel:endpoint id="ftpNotificationDownload" uri="sftp:/11.1.1.1://app/as?username=aa&password=1111&fastExistsCheck=true&localWorkDirectory=C:\\asd&download=true&throwExceptionOnConnectFailed=true&delay=4000&useFixedDelay=true"/>
I want all the files and folders to be downloaded under my local folder so I did not put filename option.
When I give the host name as 11.1.1.1, it works but when I set directory after host name like 11.1.1.1:/app/directory, it does not work.
I have checked SFTP server and it is up.
There should not be a colon (:) after the host name in the uri. And probably not even one of the two slashes. On the other hand, you are missing one slash after the sftp:.
See Apache Camels URIs syntax.
sftp://[username#]hostName[:port]/directoryName[?options]
Try
sftp://11.1.1.1/app/as?...
Note that this is true for any URI, not just Camel.
I have a ZIP file with multiple files about 1GB each. Unzipping is done by one route (Route1) that polls the download directory and saves files to the extract directory. Another route (Route2) polls the extract directory to process the files.
These files are supposed to be processed in a certain sequence (Route2 uses sortBy when getting the files). Route2 immediately picks files that are still being unzipped before all the files are available.
How can i pause Route2 from processing until after Route1 is done unzipping the files.
Can you write a done-file in your first route once the unzip process has finished and then use the "doneFileName" URI option of the File2 component in your second route?
Take a look at readLock and readLockInterval parameters in the File2 component.
You can pickup one file at a time in route 1 with maxMessagesPerPoll=1, and then use the control bus component to stop the route, and then from the other route, start the route again.
http://camel.apache.org/controlbus.html
I'm trying to integrate apache solr4.5 with tomcat7.0.
Here is my prob, I make a directory named "solr" under $CATALINA_BASE/webapps, and create an context.xml under the path, which the directory structure looks like.
$CATALINA_BASE/webapps/solr/META-INF/context.xml
The context.xml looks like as follow:
<?xml version="1.0" encoding="utf-8" ?>
<Context docBase="/solr/home/root/solr.war">
<Environment name="solr/home" type="java.lang.String" value="/solr/home/root" />
</Context>
I was expecting when I visit "localhost:8080/solr/", the solr webapp should work fine, but what I got is some errors like this:
HTTP Status 404 - /solr/
type Status report
message /solr/
description The requested resource (/solr/) is not available.
Apache Tomcat/7.0.26
But according to the apache tomcat7.0 document,
Individual Context elements may be explicitly defined:
In an individual file at /META-INF/context.xml inside the application files. Optionally (based on the Host's copyXML
attribute) this may be copied to
$CATALINA_BASE/conf/[enginename]/[hostname]/ and renamed to
application's base file name plus a ".xml" extension.
In individual files (with a ".xml" extension) in the $CATALINA_BASE/conf/[enginename]/[hostname]/ directory. The context
path and version will be derived from the base name of the file (the
file name less the .xml extension). This file will always take
precedence over any context.xml file packaged in the web
application's META-INF directory.
Inside a Host element in the main conf/server.xml.
So, please anyone give me some clues, thanks.
If you check the catalina log file located in /logs, you will probably see some errors from Solr, one of which will be:
SEVERE: Error filterStart
This is due to a change in the way that logging in general is now implemented within Solr as of version 4.3. Please refer to SolrLogging - Using the example logging setup in containers other than Jetty for the necessary steps to setup logging within Tomcat.
This should resolve your issue.
Try placing your context.xml (for all webapps) or solr.xml (solr only) file in Catalina/localhost folder instead.
For instance:
<Context docBase="webapps/solr.war" allowLinking="true" reloadable="true">
<Environment name="solr/home" type="java.lang.String" value="/opt/solr" override="true" />
</Context>
Troubleshooting:
check your logs, e.g.:
cd /var/log/tomcat? && tail -f *.log *.out *.txt