Camel 2.21.0 - how to process on exception with streaming - apache-camel

I would like to log an error on exception and continue on next record/split, but it does not work.
I tired OnExcepiton(), doTry() DSL but it does not work and goes to ErrorHandler.
onException(IOException.class)
.handled(true).process(exchange -> log.error("error!!"));
from("file:" + rootDir + "/" + account + "/inbox/?move=.done")
.unmarshal(csvDataFormat)
.split(body()).shareUnitOfWork().parallelProcessing().streaming()
.process(fileService)
.end()
Logs:
2018-07-18 14:01:59.883 DEBUG 45137 --- [/test1/request/] o.a.camel.processor.MulticastProcessor : Parallel processing failed due IOException reading next record: java.io.IOException: (line 4) invalid char between encapsulated token and delimiter
2018-07-18 14:01:59.885 ERROR 45137 --- [/test1/request/] o.a.camel.processor.DeadLetterChannel : Failed delivery for (MessageId: ID-**********-local-1531936914834-0-3 on ExchangeId: ID-*********-local-1531936914834-0-4). On delivery attempt: 0 caught: java.lang.IllegalStateException: IOException reading next record: java.io.IOException: (line 4) invalid char between encapsulated token and delimiter

#Bedla, Thank you for your input, I found this working for my UseCase,
Using onException() was still sending exchange to
DeadLetterChannel, so had to use doTry()
CasvFormat with using
maps - I couldn't modify csvFormat in process, so had to
read header from file and append csv header in body on each split using setBody
Full Route Definition:
CsvDataFormat csvDataFormat = new CsvDataFormat().setUseMaps(true);
from("file:" + rootDir + "/test/")
.log(LoggingLevel.INFO,"Start processing ${file:name}")
.unmarshal().pgp(pgpFileName,pgpUserId,pgpPassword)
.process(exchange -> { /* just to get csv header */
InputStream inputStream = exchange.getIn().getBody(InputStream.class);
try(BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream))){
String header = bufferedReader.readLine();
exchange.getIn().setHeader("CSV_HEADER",header);
csvDataFormat.setHeader(header.split(",")); //<- this does not work, so had to add in body below!
System.out.println("csvHeader is : " + header);// + " ? " + Arrays.asList(csvDataFormat.getHeader()));
}
})
.split(body().tokenize("\n")).shareUnitOfWork()
.parallelProcessing().streaming()
.setBody(exchange -> exchange.getIn().getHeader("CSV_HEADER") + "\n" + exchange.getIn().getBody())
.doTry()
.unmarshal(csvDataFormat)
.process(requestFileService)
.doCatch(IOException.class)
//TODO: custom processing here...
.process(exchange -> log.error("caught in dotry: " + exchange.getIn().getBody())).stop()
.end()//end try/catch
.choice()
.when(simple("${property." + Exchange.SPLIT_COMPLETE + "} == true"))
.log(LoggingLevel.INFO, "Finished processing ${file:name}")
.end();

Related

Rest API throws JSON parsing error: Exception during JSON parsing at row: 1 column: 1 buffer

I am using Rest API and when I use Rest.put(), it throws an error. It does not even reach the server at all. I don't know where I am going wrong.
Here is my code:
Response<String> res = Rest.put(URLLinks.getMainBackend() + "items")
.body(reqJson.toString())
.jsonContent()
.bearer(initForm.data.getString("jwt"))
.header("token", initForm.data.getString("token"))
.pathParam("id", item.getId())
.onErrorCodeJSON((errorData) -> {
if (errorData.getResponseCode() == 404) {
Dialog dlg = rich.Dialog("Not found!");
Button yes = new Button("Close");
yes.addActionListener(ev -> {
dlg.dispose();
});
dlg.add(new SpanLabel("//////!"));
dlg.add(FlowLayout.encloseRight(yes));
dlg.showPacked(BorderLayout.CENTER, true);
}else if (errorData.getResponseCode() == 402){
}
})
.getAsString();
It throws an error:
java.io.IOException: Stream closed
[Network Thread] 0:0:12,855 - Codename One revisions: 4afb54f6a5cecd2b6fbee170262d5c3c8d9431f9
[Network Thread] 0:0:12,855 - Exception: java.io.IOException - Stream closed
[Network Thread] 0:0:12,858 - Exception during JSON parsing at row: 1 column: 1 buffer:
[EDT] 0:0:12,874 - Exception: java.lang.NullPointerException - null
at com.codename1.io.BufferedInputStream.getInIfOpen(BufferedInputStream.java:140)
at com.codename1.io.BufferedInputStream.read1(BufferedInputStream.java:338)
at com.codename1.io.BufferedInputStream.read(BufferedInputStream.java:445)
at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
at com.codename1.io.JSONParser$ReaderClass.read(JSONParser.java:124)
at com.codename1.io.JSONParser.parse(JSONParser.java:188)
at com.codename1.io.JSONParser.parseJSON(JSONParser.java:475)
at com.codename1.io.rest.RequestBuilder$Connection.readUnzipedResponse(RequestBuilder.java:683)
at com.codename1.io.gzip.GZConnectionRequest.readResponse(GZConnectionRequest.java:67)
at com.codename1.io.ConnectionRequest.performOperation(ConnectionRequest.java:809)
at com.codename1.io.NetworkManager$NetworkThread.run(NetworkManager.java:282)
at com.codename1.impl.CodenameOneThread.run(CodenameOneThread.java:176)
java.lang.NullPointerException
at java.base/java.lang.String.<init>(String.java:537)
at com.codename1.io.rest.RequestBuilder.getAsString(RequestBuilder.java:361)
at com.falcontechnology.items.NewItem.backendItem(NewItem.java:425)
at com.falcontechnology.items.NewItem.lambda$itemFields$7(NewItem.java:296)
at com.codename1.ui.util.EventDispatcher.fireActionEvent(EventDispatcher.java:349)
at com.codename1.ui.Button.fireActionEvent(Button.java:570)
at com.codename1.ui.Button.released(Button.java:604)
at com.codename1.ui.Button.pointerReleased(Button.java:708)
at com.codename1.ui.Form.pointerReleased(Form.java:3356)
at com.codename1.ui.Component.pointerReleased(Component.java:4552)
at com.codename1.ui.Display.handleEvent(Display.java:2080)
at com.codename1.ui.Display.edtLoopImpl(Display.java:1052)
at com.codename1.ui.Display.mainEDTLoop(Display.java:970)
at com.codename1.ui.RunnableWrapper.run(RunnableWrapper.java:120)
at com.codename1.impl.CodenameOneThread.run(CodenameOneThread.java:176)
The null pointer on line NewItem.java:425 points at the line where there is .getAsString().
What could i be doing wrong?
EDIT
I have updated my project libraries which solved the null pointer exception on .getAsString() but it still leaves another error which is:
java.io.IOException: Stream closed
[Network Thread] 0:0:54,316 - Exception: java.io.IOException - Stream closed
[Network Thread] 0:0:54,344 - Exception during JSON parsing at row: 1 column: 1 buffer:
at com.codename1.io.BufferedInputStream.getInIfOpen(BufferedInputStream.java:140)
at com.codename1.io.BufferedInputStream.read1(BufferedInputStream.java:338)
at com.codename1.io.BufferedInputStream.read(BufferedInputStream.java:445)
at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.base/java.io.InputStreamReader.read(InputStreamReader.java:185)
at com.codename1.io.JSONParser$ReaderClass.read(JSONParser.java:191)
at com.codename1.io.JSONParser.parse(JSONParser.java:278)
at com.codename1.io.JSONParser.parseJSON(JSONParser.java:568)
at com.codename1.io.rest.RequestBuilder$Connection.readUnzipedResponse(RequestBuilder.java:785)
at com.codename1.io.gzip.GZConnectionRequest.readResponse(GZConnectionRequest.java:67)
at com.codename1.io.ConnectionRequest.performOperationComplete(ConnectionRequest.java:1002)
at com.codename1.io.NetworkManager$NetworkThread.run(NetworkManager.java:340)
at com.codename1.impl.CodenameOneThread.run(CodenameOneThread.java:176)
Apparently when you use .pathparam(), you also have to add the parameter in the url. Hence I adjusted my code to:
Response<String> res = Rest.put(URLLinks.getMainBackend() + "items"+item.getId())
.body(reqJson.toString())
.jsonContent()
.bearer(initForm.data.getString("jwt"))
.header("token", initForm.data.getString("token"))
.pathParam("id", item.getId())
.onErrorCodeJSON((errorData) -> {
if (errorData.getResponseCode() == 404) {
Dialog dlg = rich.Dialog("Not found!");
Button yes = new Button("Close");
yes.addActionListener(ev -> {
dlg.dispose();
});
dlg.add(new SpanLabel("//////!"));
dlg.add(FlowLayout.encloseRight(yes));
dlg.showPacked(BorderLayout.CENTER, true);
}else if (errorData.getResponseCode() == 402){
}
})
.getAsString();

Issue merging xml output using camel pollEnrich

Creating XML output from various sources which need to be combine for XSLT processor but getting "No consumers available on endpoint" when using pollEnrich. The pollEnrich aggregator it not being passed the pollEnrich Exchange.
Took the aggregator out and used the default aggregator. Get the same issue. Adding logs show that there is XML output coming from the previous routes to the pollEnrich end points.
package com.hitrust.route;
import com.hitrust.aggregator.AddToOutput;
import com.hitrust.processor.ConvertResultToRecords;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.processor.aggregate.AggregationStrategy;
public class AuthoritativeSourceDocument extends RouteBuilder {
public void configure() throws Exception {
AggregationStrategy addToOutput = new AddToOutput();
restConfiguration()
.component("restlet")
.host("localhost").port("18082");
rest("/authoritativesourcedocument/{authoritativesourcedocumentid}")
.consumes("application/json").produces("application/json")
.get()
.to("direct:Start");
from("direct:Start")
.multicast()
.to("direct:GetSections")
.to("direct:GetTransactions")
.to("direct:MergeSections");
from("direct:GetSections")
.setBody(simple("SELECT * " +
" FROM [dbo].[Section] AS S" +
" WHERE [Id] = ${header.id}"))
.to("jdbc:dataSource")
.setProperty("paramName", simple("Sections"))
.process(new ConvertResultToRecords())
.to("direct:GetSectionsOutput");
from("direct:GetTransactions")
.setBody(simple("SELECT * " +
" FROM [dbo].[SectionTransaction] AS ST" +
" WHERE [Id] = ${header.id}"))
.to("jdbc:dataSource")
.setProperty("paramName", simple("SectionTransactions"))
.process(new ConvertResultToRecords())
.to("direct:GetTransactionsOutput");
from("direct:MergeSections")
.setBody(simple("<param><id>${header.id}</id></param>"))
.convertBodyTo(org.w3c.dom.Document.class)
.pollEnrich("direct:GetSectionsOutput", 500, addToOutput)
.pollEnrich("direct:GetTransactionsOutput", 500, addToOutput)
.to("xslt:file:src/main/resources/xslts/MergeSections.xsl")
}
}
To execute and have combine xml output from the last route.
Please try to use "seda:" instead of "direct:" to message from your Get* routes to pollEnrich in the MergeSections route:
[...]
from("direct:Start")
.multicast()
.to("direct:GetSections")
.to("direct:GetTransactions")
.to("direct:MergeSections");
from("direct:GetSections")
.setBody(simple("SELECT * " +
" FROM [dbo].[Section] AS S" +
" WHERE [Id] = ${header.id}"))
.to("jdbc:dataSource")
.setProperty("paramName", simple("Sections"))
.process(new ConvertResultToRecords())
.to("seda:GetSectionsOutput");
from("direct:GetTransactions")
.setBody(simple("SELECT * " +
" FROM [dbo].[SectionTransaction] AS ST" +
" WHERE [Id] = ${header.id}"))
.to("jdbc:dataSource")
.setProperty("paramName", simple("SectionTransactions"))
.process(new ConvertResultToRecords())
.to("seda:GetTransactionsOutput");
from("direct:MergeSections")
.setBody(simple("<param><id>${header.id}</id></param>"))
.convertBodyTo(org.w3c.dom.Document.class)
.pollEnrich("seda:GetSectionsOutput", 500, addToOutput)
.pollEnrich("seda:GetTransactionsOutput", 500, addToOutput)
.to("xslt:file:src/main/resources/xslts/MergeSections.xsl")
Your code also looks like you assume that multicast is dispatching to the endpoints in parallel by default - it does not. You need to add the parallelProcessing() option for that.

Code giving me an ArgumentExeption?

This code is giving me an ArgumentExeption when the correct values are put in both ComboBoxes, executing the code. The code basically just deletes a file and replaces it with a modified version taken from another folder.
Here is the exact text of the error message:
An unhandled exception of type 'System.ArgumentException' occurred in Microsoft.VisualBasic.dll
Additional information: The given file path ends with a directory separator character.
Here's the code:
If ComboBox1.Text = "Nokia" And ComboBox2.Text = "HTC" And My.Computer.FileSystem.FileExists("C:\Users\" + user + "\Documents\Fiddler2\Scripts\CustomRules.js") Then
My.Computer.FileSystem.DeleteFile("C:\Users\" + user + "\Documents\Fiddler2\Scripts\CustomRules.js")
My.Computer.FileSystem.CopyFile("Config\OEM\NokiaHTC.js", destinationFileName:="C:\Users\" + user + "\Documents\Fiddler2\Scripts\")
Else
My.Computer.FileSystem.CopyFile("Config\OEM\NokiaHTC.js", destinationFileName:="C:\Users\" + user + "\Documents\Fiddler2\Scripts\")
End If
The problem is that the destination file path ends with a "\" value. This isn't legal for the CopyFile API. Switch it to include the file name and this should fix the problem
My.Computer.FileSystem.CopyFile( _
"Config\OEM\NokiaHTC.js", _
destinationFileName:="C:\Users\" + user + "\Documents\Fiddler2\Scripts\NokiaHTC.js")

Exception: conversion from UNKNOWN to UNKNOWN is unsupported

I'm converting some jdbc code from MySql to SQL Server. When trying to
query = "Update ReportSetup "
+ "set N_ID=?, "
+ "R_Default=?, "
+ "R_Name=?, "
+ "R_Module=? "
+ " where R_ID = ?";
}
PreparedStatement stmt =
(PreparedStatement) con.prepareStatement(query, Statement.RETURN_GENERATED_KEYS);
stmt.setInt(1, ri.getNodeID());
stmt.setInt(2, (ri.isDefault()) ? 1 : 0);
stmt.setString(3, ri.getName());
Object o = ri.getReportModule();
stmt.setObject(4, o);
The last statement stmt.setObject(4,o) throws the exception.
ri.getReportModule returns an instance of a class which implements Externalizable.
The method writeExternal() of that class is implemented as
public final void writeExternal(final ObjectOutput objectOutput) throws IOException {
for (int i=0; i<pdV.size(); i++) {
PropertyDescriptor pd = pdV.elementAt(i);
try {
Method m = pd.getReadMethod();
Object val = pd.getReadMethod().invoke(this);
System.out.print("writing property " + i + ": " + pd.getName() + " = " + val);
objectOutput.writeObject(val);
} catch (Exception e) {
e.printStackTrace();
}
}
}
The database column in question is defined as
varbinary(max), not null
The code works well using MySql but I can't figure out what to do to make it run with Sql Server.
Any suggestions would be very much appreciated
The problem was that sql server is not happy to save a serialization (as done when implementing externalizable). .setObject() fails. The solution is to use setBinaryStream().
// Sql Server can't do an stmt.setObject(4,o) "Conversion from UNKNOWN to UNKNOWN not supported"
// Serialize the object to an output stream and then read it in again into the stmt.
Object o = ri.getReportModule();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream objectOutput = new ObjectOutputStream(bos);
objectOutput.writeObject(o);
objectOutput.flush();
InputStream objectInput = new ByteArrayInputStream(bos.toByteArray());
stmt.setBinaryStream(4, objectInput);
Cheers
Christian

Not able to access MSSQL analysis services cubes

I am using the following code to access mssql analysis services cubes from java using OLAP4j 1.1.0
Class.forName("org.olap4j.driver.xmla.XmlaOlap4jDriver");
OlapConnection con = (OlapConnection)
DriverManager.getConnection("jdbc:xmla:Server=http://mssql.com/mssql/msmdpump.dll;"+
"Cache=org.olap4j.driver.xmla.cache.XmlaOlap4jNamedMemoryCache;"+
"Cache.Name=MyNiftyConnection;Cache.Mode=LFU;Cache.Timeout=600;Cache.Size=100",
"username", "password");
OlapWrapper wrapper = (OlapWrapper) con;
OlapConnection olapConnection = wrapper.unwrap(OlapConnection.class);
OlapStatement stmt = olapConnection.createStatement();
CellSet cellSet = stmt.executeOlapQuery("SELECT {" +
" [Measures].[LoginTime_Format]," +
"[Measures].[EngageTime_Format]," +
"[Measures].[ChatTime_Format]," +
"[Measures].[AverageHandleTime_Format]," +
"[Measures].[MultipleChatTime_Format]," +
"[Measures].[NonEngagedTime_Format]," +
"[Measures].[TimeAvailable_Format]," +
"[Measures].[TimeAvailableNotChatting_Format]," +
"[Measures].[TimeNotAvailable_Format]," +
"[Measures].[TimeNotAvailableChatting_Format]," +
"[Measures].[AcdTimeouts]," +
"[Measures].[AvgConcurrency]," +
"[Measures].[OperatorUtilization]} ON 0," +
" NON EMPTY ([DimTime].[CalenderDayHour].[CalenderDayHour], [DimAgent].[Agent]."+
"[Agent],[DimAgent].[Agent Name].[Agent Name]) ON 1" +
" FROM (SELECT [DimClient].[Client].&[4] ON 0 FROM" +
" (SELECT [DimTime].[CalenderDayHour].[CalenderDayHour].&[2013010100]:"+
"[DimTime].[CalenderDayHour].[CalenderDayHour].&[2013121216] ON 0 FROM [247OLAP]))");
When I run this code I get the following exception at executeOlapQuery line-
Exception in thread "main" java.lang.RuntimeException: [FATAL]:1:1: Content is not allowed in prolog.
at org.olap4j.driver.xmla.XmlaOlap4jUtil.checkForParseError(XmlaOlap4jUtil.java:134)
at org.olap4j.driver.xmla.XmlaOlap4jUtil.parse(XmlaOlap4jUtil.java:83)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.executeMetadataRequest(XmlaOlap4jConnection.java:884)
at org.olap4j.driver.xmla.XmlaOlap4jDatabaseMetaData.getMetadata(XmlaOlap4jDatabaseMetaData.java:137)
at org.olap4j.driver.xmla.XmlaOlap4jDatabaseMetaData.getMetadata(XmlaOlap4jDatabaseMetaData.java:67)
at org.olap4j.driver.xmla.XmlaOlap4jDatabaseMetaData.getDatabaseProperties(XmlaOlap4jDatabaseMetaData.java:1044)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.makeConnectionPropertyList(XmlaOlap4jConnection.java:324)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.generateRequest(XmlaOlap4jConnection.java:1037)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.populateList(XmlaOlap4jConnection.java:849)
at org.olap4j.driver.xmla.DeferredNamedListImpl.populateList(DeferredNamedListImpl.java:136)
at org.olap4j.driver.xmla.DeferredNamedListImpl.getList(DeferredNamedListImpl.java:90)
at org.olap4j.driver.xmla.DeferredNamedListImpl.size(DeferredNamedListImpl.java:116)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.getOlapDatabase(XmlaOlap4jConnection.java:451)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.getOlapCatalog(XmlaOlap4jConnection.java:501)
at org.olap4j.driver.xmla.XmlaOlap4jConnection.getCatalog(XmlaOlap4jConnection.java:496)
at org.olap4j.driver.xmla.XmlaOlap4jStatement.executeOlapQuery(XmlaOlap4jStatement.java:291)
at com.tfsc.ilabs.olap4j.POC.main(POC.java:28)
Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.olap4j.driver.xmla.XmlaOlap4jUtil.parse(XmlaOlap4jUtil.java:80)
Any help will be much appreciated.
You should check what's being sent back by the server, using WireShark or something similar. This kind of error happens when the XML parser tries to parse the response it got. The server is probably not sending XML content back.

Resources