Mule File Inbound - empty files are not triggered - file

I have a scenario wherein I need to read files from a particular folder. So I had a File inbound as below, its reading all non-empty files. But empty files are not read and sits in the same location as is.
<file:inbound-endpoint path="${file.path}" responseTimeout="10000" doc:name="File" moveToDirectory="${audit.location}">
<file:filename-regex-filter pattern="file.employee(.*).xml,file.client(.*).xml"
caseSensitive="true"/>
</file:inbound-endpoint>
I removed File filter, but still it doesn't read empty files.
Is there a way to enable file inbound to read empty files too?

According the the Mule File Connector documentation:
The File connector as inbound endpoint does not process empty (0 bytes) files.
So this behavior is expected. There is no documented way to process non empty file with the File Inbound Endpoint.
However you can still write your own connector to do this, or use a workaround such as fill your "empty" file with a single character (such as a space) to make it non-empty

If you want to read a file with the size of 0 KB, then you can`t achieve this with File Connector, but we can read a file by using MuleRequester in the flow. I will share sample snippet soon. Please let me know,If you need any help.
Regards,
Sreenivas B

Mule File connector does not process empty (0 bytes) files as inbound endpoint

As per my knowledge File Inbound connector will not process (0 KB) size files.

On the File Connector, the class org.mule.transport.file.FileMessageReceiver.java in method poll has :
if (file.length() == 0)
{
if (logger.isDebugEnabled())
{
logger.debug("Found empty file '" + file.getName() + "'. Skipping file.");
}
continue;
}
that prevents it from proccessing empty files
But you can create your own CustomFileMessageReceiver.java, create your package:
package com.mycompany.mule.transport.file;
And the class that extends AbstractPollingMessageReceiver
public class CustomFileMessageReceiver extends AbstractPollingMessageReceiver
Copy the original FileMessageReceiver.java methods but comment the above lines and change FileMessageReceiver to CustomFileMessageReceiver where needed.
The call fileConnector.move(file, workFile) is a protected method from the original package, commented and beware you cannot use workdir.
In the same package create a copy of org.mule.transport.file.ReceiverFileInputStream.java
Configure your connector:
<file:connector name="FILE" readFromDirectory="${incoming.directory}" autoDelete="true" streaming="false" recursive="true" validateConnections="true" doc:name="File" writeToDirectory="${processed.directory}">
<service-overrides messageReceiver="com.mycompany.mule.transport.file.CustomFileMessageReceiver" />
</file:connector>
Or you may implement your own file connector, as stated in the above answers.

Related

Putsftp is taking a wrong path in SFTP server in nifi

I have a flow to fetch file from SFTP server, rename it and put it back to server in same location.
My flow:
Listsftp-> fetchsftp-> updateAttribute-> putsftp
My file location is in d drive, I have mentioned that location in remote path property of putsftp but it taking the path like
c:/users/myname/d:/file/location
And of course it is giving me error.
Is there any solution for this?
Thanks in advance.
you can use the SFTP processor only if you are using a server with Host - Port etc.
If you want to get some files from your disk (C:/ for example) you can use the GETFILE processor
an example of flow could be this:
GETSFTP with the property Keep Source File to false
UpdateAttribute
new property -> filename -> new_file_test.example
PUTSFTP
you can use GETSFTP/GETFILE PUTSFTP/PUTFILE

Anybody know if OrcTableSource supports S3 file system?

I'm running into some troubles with using OrcTableSource to fetch Orc file from cloud Object storage(IBM COS), the code fragment is provided below:
OrcTableSource soORCTableSource = OrcTableSource.builder() // path to ORC
.path("s3://orders/so.orc") // s3://orders/so.csv
// schema of ORC files
.forOrcSchema(OrderHeaderORCSchema)
.withConfiguration(orcconfig)
.build();
seems this path is incorrect but anyone can help out? appreciate a lot!
Caused by: java.io.FileNotFoundException: File /so.orc does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:611)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:824)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:601)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:428)
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:142)
at
org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:346)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:768) at
org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:528)
at org.apache.orc.impl.ReaderImpl.(ReaderImpl.java:370) at
org.apache.orc.OrcFile.createReader(OrcFile.java:342) at
org.apache.flink.orc.OrcRowInputFormat.open(OrcRowInputFormat.java:225)
at
org.apache.flink.orc.OrcRowInputFormat.open(OrcRowInputFormat.java:63)
at
org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:170)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) at
java.lang.Thread.run(Thread.java:748)
By the way, I've already set up flink-s3-fs-presto-1.6.2 and had following code running correctly. The question is limited to OrcTableSource only.
DataSet<Tuple5<String, String, String, String, String>> orderinfoSet =
env.readCsvFile("s3://orders/so.csv")
.types(String.class, String.class, String.class
,String.class, String.class);
The problem is that Flink's OrcRowInputFormat uses two different file systems: One for generating the input splits and one for reading the actual input splits. For the former, it uses Flink's FileSystem abstraction and for the latter it uses Hadoop's FileSystem. Therefore, you need to configure Hadoop's configuration core-site.xml to contain the following snippet
<property>
<name>fs.s3.impl</name>
<value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
</property>
See this link for more information about setting up S3 for Hadoop.
This is a limitation of Flink's OrcRowInputFormat and should be fixed. I've created the corresponding issue.

Mule - Delete files with Files Connector

This is my main mule flow:
HTTP > Payload > File (Create) > Property (File Name) JMS
And that JMS calls to other flow that starts with JMS
JMS > JSON TO XML > File (Delete by file name)
Firstly, I create a file with the JSON message.
And finally, when JSON message is transformed to XML, I try to delete the file by name calling a property that tell me the name.
But the files doesn't delete. Its output this action by log:
Writing file to: C:\errors\91fa03a0-dd33-11e5-a0d9-408d5c4c1bf3.dat
How can I do to delete it?
I tried with the option autoDelete in the Connector Configuration but it doesn't delete.
<file:connector name="File1" outputPattern="#[message.inboundProperties.fileName]" autoDelete="true" streaming="false" validateConnections="true" doc:name="File"/>
You can't use this file outbound endpoint to delete a file.
If you would use something like S3 for example you can, but those connectors are 'operation-based'.
It's meant to write an new file or append data to an existing one.
The autoDelete option is for deleting the file once it has been read by a file inbound endpoint.
More info:
https://docs.mulesoft.com/mule-user-guide/v/3.7/file-transport-reference
If you want to delete and for your usecase you should use a piece of JAVA code and use methods from the File Class:
https://docs.oracle.com/javase/7/docs/api/java/io/File.html

Mule - how to get the name of the file created in an outbound endpoint

I have a Mule application that is writing a file in an outbound endpoint, with the config below:
<file:outbound-endpoint path="${Outbound}" outputPattern="outputFile_#[function:datestamp:yyyyMMddHHmmss].csv" doc:name="Output File"/>
Following this point in the flow I need to display a log message along the lines of "Successfully created file {filename}".
The issue I'm having is that I can't find a way of displaying the name of the file I have just created. I could put: Successfully created file outputFile_#[function:datestamp:yyyyMMddHHmmss].csv, but there is a chance that the datestamp may differ by one second.
Is there a way in Mule that I can display the name of the file I've just written?
UPDATE
Following the response from #til_b, I've achieved this using the following:
<set-variable value="outputFile_#[function:datestamp:yyyyMMddHHmmss].csv" variableName="Filename" doc:name="Variable"/>
<file:outbound-endpoint path="${Outbound}" outputPattern="#[variable:Filename]" doc:name="Output File"/>
<logger level="INFO" message="Successfully created file #[variable:Filename]" doc:name="Logger" />
I dont know about mule, but when i encounter such a problem while programming i store the generated filename in a variable, and then use that variable to actually create the file and display the message.
In Pseudocode:
var filename = #yyyymmddhhMMss.csv
create_file(filename)
log_message(filename + ' successfully created')

Generate and download file with jboss seam

I need to add an 'export' function to an existing web app using seam. The purpose is to export search results to a csv file. I have no problem generating a csv, but I do not know how the send the csv back to the user.
I do not want to store the csv on the server because that would be waisted storage space. How could I achieve this in jboss seam?
Use the Document Store Servlet provided by Seam.
Almost copying and pasting from the reference doc, declare the servlet in web.xml like this:
<servlet>
<servlet-name>Document Store Servlet</servlet-name>
<servlet-class>org.jboss.seam.document.DocumentStoreServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>Document Store Servlet</servlet-name>
<url-pattern>/seam/docstore/*</url-pattern>
</servlet-mapping>
Then create a export.xhtml file with only <s:resource> tag:
<s:resource xmlns="http://www.w3.org/1999/xhtml"
xmlns:s="http://jboss.com/products/seam/taglib"
data="#{myComponent.csvData}"
contentType="application/vnd.ms-excel"
fileName="#{myComponent.csvFileName}"/>
Generate link for downloading the file in your page with <s:download>:
<s:download src="/csv/export.xhtml">
<h:outputText value="Download CSV"/>
<f:param name="param1" value="somevalue"/>
<f:param name="param2" value="someOtherValue"/>
</s:download>
Finally, implement getCsvData() and getCsvFileName() methods in your component:
// could be byte[], File or InputStream
public InputStream getCsvData() {
// generate data to be downloaded
}
public String getCsvFileName() {
return "myfile.csv";
}
Note that <s:download> propagates conversation (unless you set propagation=none). If you propagate the conversation context probably you won't need to pass any parameter. For large data set it may be preferable to not propagate the conversation and pass parameter to select the data in a request scoped component.
There's a couple of ways:
1) Check the Seam docs for info on using Seam-Excel to programmatically generate your file and then write it out using a mime-type set for CSV - this is all detailed in the docs.
However, I could not get this to work in the latest version of Seam, as it requires a response object, which used to be available from the Seam context but now only returns null.
2) Code the CSV file you want as an Excel xhtml template (see the Seam docs and example projects) and simply render this as normal using an tag.
I do this regularly and it works well, bar the restriction that you cannot supply a filename.
HTH.

Resources