Problem: The file is not consumed from the server
I am using
from("test")
.routeId("test")
.pollEnrich()
.simple("smb://myUrl?password=test&fileName=${in.headers.test}")
.aggregationStrategy((Exchange oldExchange, Exchange newExchange) -> {
//do things
return newExchange;
})
I have no error, I am sure that the url is ok, because when I am using the same url in the from(), the file gets consumed.
I don't understand what is happening here, I am using camel 2.24.0 and camel-extra:camel-jcifs:2.23.1. I have tried to use smb2 using the library from github.jborza.camel-smbj, still the same outcome.
I tried to debug, I can see in the GenericFileComponent class in the createEndpoint method, that the endpoint is correctly created, then I tried (in debug mode) to get the exchanges from my endpoint, I can get them successfully, further this will be a SmbEndpoint, when I try to get the exchanges from my smbEndpoint it returns exactly the needed file from the server, further a EventDrivenPollingConsumer is created for this endpoint, I had a look at it, is started (seems ok). When it hits the consumer.receive() from the PollEnricher it blocks, no file is consumed. I tried using a timeout, than returns null, so somehow cannot find the file, or the consumer is wrong, I honestly have no clue at this point.
I had a look here too: https://github.com/apache/camel/blob/b9a3117f19dd19abd2ea8b789c42c3e86fe4c488/core/camel-core/src/test/java/org/apache/camel/component/file/FileConsumePollEnrichFileTest.java
and I have played with delays
&consumer.initialDelay=100&consumer.delay=100&consumer.bridgeErrorHandler=true
Then I tried to implement with processor like here:
https://github.com/apache/camel/blob/b9a3117f19dd19abd2ea8b789c42c3e86fe4c488/core/camel-core/src/test/java/org/apache/camel/component/file/FileConsumePollEnrichFileUsingProcessorTest.java
The same result :(
At some point the file was consumed, suddenly, but this happened only once, I cannot understand this behavior.
Sounds that you have a readlock problem, can you find any files in .done with the same name as the file you try to consume?
Related
I'm trying to upload a file using multipart/form-data to a Camel route.
All is good, however, I can't get the original file name.
Camel version is: 3.14.1
Update
With the following modification to the route. I managed to process binary files (getting the file name and storing them). However, with text files, the file is appended with the boundary footer:
------WebKitFormBoundary7BH9nQ2RqDXvTRAJ--
The route definition:
rest("/v1/file-upload-form")
.post()
.consumes(MediaType.MULTIPART_FORM_DATA_VALUE)
.route()
.process((exchange) -> {
InputStream is = exchange.getIn().getBody(InputStream.class);
MimeBodyPart mimeMessage = new MimeBodyPart(is);
DataHandler dh = mimeMessage.getDataHandler();
exchange.getIn().setBody(dh.getInputStream());
exchange.getIn().setHeader(Exchange.FILE_NAME, dh.getName());
})
.to("file://" + incomingFolder);
Thank you in advance
Edwardo
Edit: Since you have everything else already working, I'd recommend the Stream Caching option.
As Nicolas suggested, checkout Camel's MIME Multipart data format.
Also, the reason you're getting "Missing start boundary" is because your processor is consuming the InputStream. You can try to reset() it, but it might be better to just consume the InputStream once, or enable Stream Caching.
Instead of stream caching, you could also just convert the stream to a string. Before your processor, add:
.convertBodyTo(String.class)
The string can be read over and over. If you still get the missing start boundary error, try logging the body before the unmarshal operation. Make sure the message is intact and that it indeed contains the start boundary.
Scenario: I've been stuck on this for way to long and I think solution might be easy but I just can't see it, this is the scenario:
cURL POST to http://localhost:8080/my_imports (raw JSON data on body)
->
MyImportsCustomHandler (extends ThreadedHttpRequestHandler [Validations]
->
MyObjectProcessor (extends Processor) [JSON deserialize and data massage]
->
MyFirstDocumentProcessor (extends DocumentProcessor) [Set some fields and save]
Problem is that execution never reaches MyFirstDocumentProcessor, likely because request didn't started from the document_api endpoints (intentionaly).
There are no errors thrown, just processing route never reaches the document processor chain, I think it should because on MyObjectProcessor I'm doing:
DocumentType type =
localDocHandler.getDocumentTypeManager().getDocumentType("my_doc");
DocumentId id = new DocumentId("id:default:my_doc::2");
Document document = new Document(type, id);
DocumentPut docPut = new DocumentPut(document);
Processing proc = com.yahoo.docproc.Processing.of(docPut);
I got this idea from here: https://github.com/vespa-engine/vespa/blob/master/docproc/src/test/java/com/yahoo/docproc/util/SplitterJoinerTestCase.java
but on that test I see this line splitter.process(p);, which I'm not able to find a suitable replacement that works inside a Processor, in that context I only have the Request, Execution and DocumentProcessingHandler
I hope somebody versed on Vespa con shine some light on this, is just the last hop on the processing chain that I can't bridge :|
To write documents from Java code, you need to use the Document Access API:
http://docs.vespa.ai/documentation/document-api-guide.html#document-access
A working solution is in https://github.com/vespa-engine/sample-apps/pull/44
I am using talend-ESB and want to parse EDI message to XML using smooks & I am getting null in body. The code looks as below.
from(
"file://D:/cimt/InvoiceEDI_Mapping/" + "?noop=true"
+ "&autoCreate=true" + "&flatten=false"
+ "&fileName=InDev_EDI_Msg.txt" + "&bufferSize=128")
.routeId("TestSmooksConfig_cFile_1")
.log(org.apache.camel.LoggingLevel.WARN,
"TestSmooksConfig.cLog_1", "${body}")
.id("TestSmooksConfig_cLog_1")
.to("smooks://EDI_Config.xml")
.to("log:TestSmooksConfig.cLog_2" + "?level=WARN")
.id("TestSmooksConfig_cLog_2");
}
My Talend route looks as below.
I used following set of external dependencies.
milyn-commons-1.7.0.jar
milyn-smooks-camel-1.7.0.jar
milyn-smooks-edi-1.7.0.jar
milyn-smooks-core-1.7.0.jar
jaxen-1.1.6.jar
milyn-edisax-parser-1.4.jar
Also, I see a strange behavior that, upon execution, I still see "starting" prior to cJavaDSLProcessor, which initially made me wonder if at all it gets executed. But later, when I intentionally made a mistake in EDI-Mapping, then the route was throwing errors, which kind of convinced me that it does parse the EDI message.
I did also search before posting this question here, and found a similar problem in this link
And I tried to lower my revision of org.milyn.* jars to 1.4.0, and got an exception that the route could not register smooks component. So I continued using 1.7.0 version of org.milyn.* jars.
For the benefit of others who might bump into similar issue, I 'assume' that the output of the smooks gets written into an Object of type StringResult.class. However, in my initial implementation, there was no such option and hence the output body was null.
Later, I tried alternative approach from http://smooks.org/guide where they used processor endpoint.Actually they had even made a statement that the data could be retrieved through exports element. The below code snippet helped to fix issue.
Smooks smooks = new Smooks("edi-to-xml-smooks-config.xml");
ExecutionContext context = smooks.createExecutionContext();
smooks.setExports(new Exports(StringResult.class));
SmooksProcessor processor = new SmooksProcessor(smooks, context);
from("file://input?noop=true")
.process(processor)
.to("mock:result");
I'm building an SPA in AngularJS served by a Laravel (5.1) backend. Of late I've been encountering an annoying error, a server 500 or code 0 error which is abit hard to explain how it comes but let me try to may be someone will understand the dental formula of my problem.
When i start my AngularJS controller, I make several server calls (via independent $http calls from services) to retrieve information i might later need in the controller. For example,
Functions.getGrades()
.then(function(response)
{
$scope.grades = response.data;
});
Subjects.offered()
.then(function(response)
{
$scope.subjects = response.data;
});
Later on i pass these variables (grades or subjects) to a service where they are used for processing. However, these functions are randomly returning code 500 server errors after they run, and sometimes returning status code 0 after running. This happens in a random way and it is hard for me to point out the circumstances leading to their popping up. This leaves me with frequent empty Laravel-ised error screens like the ones shown below.
Anyone reading my mind?
Ok, after a suggestion given in a comment above that I check my Laravel log files (located in storage/logs/laravel.log- Laravel 5.1), i found out that the main error most of these times was this one: 'PDOException' with message 'SQLSTATE[HY000] [1044] Access denied for user ''#'localhost' to database 'forge'' in ..., plus another one that paraphrased something like No valid encrypter found. These were the key opener.
On reading another SO thread here, it said in part:
I solved, sometimes laravel not read APP_KEY in .ENV. And returns a value "SomeRandomString" (default is defined in config / app.php), and have the error "key length is invalid", so the solution is to copy the value of APP_KEY, to the value 'key 'in config / app.php, that's all! I solved!
That was exactly the issue! When loading the DB params from the .env to config/database.php, Laravel was sometimes unable to read the environment variables and went for the fallback default fallback options (forge for DB name and username and SomeRandomString for the APP_KEY). So, to solve this i just did as advised: copied the APP_KEY in .env to the config/app.php and edited the default DB parameters to the actual DB name and username/password I'm using. Just that and i was free from pollution. Hope someone finds this helpful.
I'm creating a file and writing a String on it with encoding set to LATIN1. However, the finished file is set with a different encoding (us-ascii or utf-8 returned by "file -bi" on Linux, depending on the method I use to get the String).
Here follows the creation method:
new File("/home/username/dart_test/file.xml").create(recursive: true).then((file) {
file.writeAsString(_methodReturnsAString(), mode: FileMode.WRITE, encoding: LATIN1);
});
Any ideas on what could be wrong?
EDIT (RELATED TO ANSWER):
There's no problem on the method described above. The problem was the data that was being provided to the method inside "writeAsString". That data comes from an HttpRequest that was not being processed properly (in fact, the setting of the encoding to ISO-8859-1 was causing the problem).
There's no problem with the method described on the question. Actually, the problem lays in an http request body handler that was not set properly.
So I'm answering my own question in order to help others with the same problem.
Here follows my request handler (from http_server package):
HttpBodyHandler.processRequest(request/*, defaultEncoding: Encoding.getByName("ISO-8859-1")*/).then((body) {
// Do something with body.
}, onError: _printError);
Take a look at the commented out "defaultEncoding". That was the cause. I don't think you can set it if you are not processing any files (blobs) on the request. I don't know if there's any situation where you should set it when just processing some String (I would appreciate if someone could complete this answer with this information).