I am using camel-bindy in order to create fixed-length files. I am creating the file correct, but I have a new request to separate each field-column with character ";". Thus at the end I need to have a fixed-length file semicolon separated. Do you know if this is possible with camel-bindy?
Next is my model class
#FixedLengthRecord(length = 313, paddingChar = ' ')
public class Fastbox {
#DataField(pos = 1, length = 7, align = "L")
private String field1;
#DataField(pos = 2, length = 10, align = "L")
private String field2;
#DataField(pos = 3, length = 3, align = "R")
private Integer field3;
#DataField(pos = 4, length = 10, align = "L")
private String field4;
...
If i try to use something like this
#DataField(pos = 1, length = 7, align = "L", delimiter = ";")
I am loosing the length. What I mean by that is that if the value is null, length is not taken into consideration and just puts a semicolon. So I am having
;field2 fi3field4
instead of
;field2 fi3field4
Thanks!
Related
I've a class that was working with Bindy 2.17, but having migrated to Camel 2.21.2, it is no longer trimming the incoming data when I unmarshall.
I've tried adding a paddingChar attribute to the record annotation, but that has had no effect, and as it is spaces I'm trying to trim, it should be the default paddingChar value anyway.
My class is of the form
#FixedLengthRecord( header = MyClass.MyHeader.class, footer = MyClass.MyFooter.class, skipHeader = true, skipFooter = true, ignoreTrailingChars = true, crlf="WINDOWS", paddingChar = ' ' )
public class MyClass{
#DataField( pos = 1, length = 2, trim = true )
private String field1;
#DataField( pos = 2, length = 15, trim = true )
private String field2;
#DataField( pos = 3, length = 15, trim = true )
private String field3;
#FixedLengthRecord( ignoreTrailingChars = true )
public static class MyHeader {
}
#FixedLengthRecord( ignoreTrailingChars = true)
public static class MyFooter {
}
}
Has something changed in the way trim is configured, or is there something else I'm missing?
Thanks!
I just had the same issue. By default, fixed length records are aligned to the right. Starting in version 2.18, only padding characters to the left of the record are trimmed in this case. If your padding characters are always on the right, you can align your record left with align="L". If you just want to trim everything independent of the alignment, you can use align="B" starting in version 2.20. Here is the relevant change: https://github.com/apache/camel/commit/26aa4e8f14cac9dcdaa8f369a8045b8e8df56f1e#diff-24aaa851bf960dc4d2e04c5fbbf8aada
When I use sql-compenent to retrieve records from database as Map<String,Object> is OK.
<to uri="sql-clsivtrk:{{clsiv_tracker_config_se}}?outputType=SelectOne" />
But, when I define a outputClass like this:
<to uri="sql-clsivtrk:{{clsiv_tracker_config_se}}?outputType=SelectOne&outputClass=br.com.rwit.clsi.m2m.rs.model.TrackerConfig" />
I got the error java.sql.SQLException: Fail to convert to internal representation
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[route9 ] [route9 ] [servlet:/events/config?httpMethodRestrict=PUT ] [ 93]
[route9 ] [to39 ] [direct:save-config ] [ 0]
[tracker-configurat] [convertBodyTo4 ] [convertBodyTo[java.lang.String] ] [ 0]
[tracker-configurat] [unmarshal4 ] [unmarshal[ref:trackerConfigJsonList] ] [ 2]
[tracker-configurat] [log4 ] [log ] [ 1]
[tracker-configurat] [split3 ] [split[simple{${body}}] ] [ 6]
[tracker-configurat] [to6 ] [sql-tracker:{{tracker_config_se}}?outputType=SelectOne&outputClass=br.com.acme] [ 83]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException; SQL state [99999]; error code [17059]; Fail to convert to internal representation; nested exception is java.sql.SQLException: Fail to convert to internal representation
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:89)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1402)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:620)
at org.apache.camel.component.sql.SqlProducer.process(SqlProducer.java:116)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
Caused by: java.sql.SQLException: Fail to convert to internal representation
at oracle.jdbc.driver.CharCommonAccessor.getBoolean(CharCommonAccessor.java:185)
at oracle.jdbc.driver.T4CVarcharAccessor.getBoolean(T4CVarcharAccessor.java:794)
at oracle.jdbc.driver.OracleResultSetImpl.getBoolean(OracleResultSetImpl.java:640)
at com.sun.gjc.spi.base.ResultSetWrapper.getBoolean(ResultSetWrapper.java:169)
at org.springframework.jdbc.support.JdbcUtils.getResultSetValue(JdbcUtils.java:148)
at org.springframework.jdbc.core.BeanPropertyRowMapper.getColumnValue(BeanPropertyRowMapper.java:377)
at org.springframework.jdbc.core.BeanPropertyRowMapper.mapRow(BeanPropertyRowMapper.java:298)
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:93)
at org.apache.camel.component.sql.DefaultSqlEndpoint.queryForObject(DefaultSqlEndpoint.java:488)
at org.apache.camel.component.sql]]
My outputType
import java.util.Calendar;
import java.util.Date;
import com.fasterxml.jackson.annotation.JsonFormat;
import java.util.Calendar;
import java.util.Date;
import com.fasterxml.jackson.annotation.JsonFormat;
public class TrackerConfig
{
private static final String HOST1 = "http://locahost/";
private static final String HOST2 = "http://localhost/";
private static final String CONTEXT = "my-ctx/";
private static final String API = "api/";
private static final int DEFAULT_INTERVAL_TRANSM = 60;
private static final int DEFAUL_INTERVAL_CAPT = 30;
private static final int DEFAUL_BATCHSIZE = 100;
private static final int DEFAUL_RENEW_CONF = 3600;
private String deviceImei1;
private String deviceImei2;
private String phoneNumber;
private String deviceSO;
private String deviceModel;
private String deviceSNumber;
private String myIp;
private String host1;
private String host2;
private String context;
private String api;
private Number intervalOftransmission;
private Number intervalOfCapture;
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ")
private Date captureBegin;
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ")
private Date captureFinal;
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ")
private Date transmissionBegin;
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ")
private Date transmissionFinal;
private Double minDistance;
private Double maxDistance;
private Number intervalOfRenewConfig;
private String levelConfig;
private String deprecated;
private String route;
private String version;
private Double accuracy;
private Number batchSize;
public TrackerConfig()
{
this.intervalOftransmission = DEFAULT_INTERVAL_TRANSM;
this.intervalOfCapture = DEFAUL_INTERVAL_CAPT;
this.batchSize = DEFAUL_BATCHSIZE;
this.intervalOfRenewConfig = DEFAUL_RENEW_CONF;
this.captureBegin = newTime(7, 0, 0);
this.captureFinal = newTime(18, 0, 0);
this.transmissionBegin = newTime(4, 6, 0);
this.transmissionFinal = newTime(23, 59, 59);
this.minDistance = 10D;
this.maxDistance = 10001D;
this.host1 = HOST1;
this.host2 = HOST2;
this.context = CONTEXT;
this.api = API;
this.levelConfig = "INFO";
}
// ommit getters and setters
}
using 2.22.2
Could you show your outputClass ?
See https://github.com/apache/camel/blob/camel-2.22.x/components/camel-sql/src/main/docs/sql-component.adoc
Under outputType it states:
Make the output of consumer or producer to SelectList as List of Map,
or SelectOne as single Java object in the following way: a) If the
query has only single column, then that JDBC Column object is
returned. (such as SELECT COUNT( ) FROM PROJECT will return a Long
object. b) If the query has more than one column, then it will return
a Map of that result. c) If the outputClass is set, then it will
convert the query result into an Java bean object by calling all the
setters that match the column names. It will assume your class has a
default constructor to create instance with. d) If the query resulted
in more than one rows, it throws an non-unique result exception.
One of the things that should be done is that the column names should match the setters and a default constructor should be present. Is this the case?
Consider a text file stored in an online location that looks like this:
;aiu;
[MyEditor45]
Name = MyEditor 4.5
URL = http://www.myeditor.com/download/myeditor.msi
Size = 3023788
Description = This is the latest version of MyEditor
Feature = Support for other file types
Feature1 = Support for different encodings
BugFix = Fix bug with file open
BugFix1 = Fix crash when opening large files
BugFix2 = Fix bug with search in file feature
FilePath = %ProgramFiles%\MyEditor\MyEditor.exe
Version = 4.5
Which details information about a possible update to an application which a user could download. I want to load this into a stream reader, parse it and then build up a list of Features, BugFixes etc to display to the end user in a wpf list box.
I have the following piece of code that essentially gets my text file (first extracting its location from a local ini file and loads it into a streamReader. This at least works although I know that there is no error checking at present, I just want to establish the most efficient way to parse this first. One of these files is unlikely to ever exceed more than about 250 - 400 lines of text.
Dim UpdateUrl As String = GetUrl()
Dim client As New WebClient()
Using myStreamReader As New StreamReader(client.OpenRead($"{UpdateUrl}"))
While Not myStreamReader.EndOfStream
Dim line As String = myStreamReader.ReadLine
If line.Contains("=") Then
Dim p As String() = line.Split(New Char() {"="c})
If p(0).Contains("BugFix") Then
MessageBox.Show($" {p(1)}")
End If
End If
End While
End Using
Specifically I'm looking To collate the information about Features, BugFixes and Enhancements. Whilst I could construct what would in effect be a rather messy if statement I feel sure that there must be a more efficient way to do this , possibly involving linq. I'd welcome any suggestions.
I have added the wpf tag on the off chance that someone reading this with more experience of displaying information in wpf listboxes than I have might just spot a way to effectively define the info I'm after in such a way that it could then be easily displayed in a wpf list box in three sections (Features, Enhancements and BugFixes).
Dom, Here is an answer in C#. I will try to convert it to VB.Net momentarily. First, since the file is small, read all of it into a list of strings. Then select the strings that contain an "=" and parse them into data items that can be used. This code will return a set of data items that you can then display as you like. If you have LinqPad, you can test thecode below, or I have the code here: dotnetfiddle
Here is the VB.Net version: VB.Net dotnetfiddle
Imports System
Imports System.Collections.Generic
Imports System.Linq
Public Class Program
Public Sub Main()
Dim fileContent As List(Of String) = GetFileContent()
Dim dataItems = fileContent.Where(Function(c) c.Contains("=")).[Select](Function(c) GetDataItem(c))
dataItems.Dump()
End Sub
Public Function GetFileContent() As List(Of String)
Dim contentList As New List(Of String)()
contentList.Add("sb.app; aiu;")
contentList.Add("")
contentList.Add("[MyEditor45]")
contentList.Add("Name = MyEditor 4.5")
contentList.Add("URL = http://www.myeditor.com/download/myeditor.msi")
contentList.Add("Size = 3023788")
contentList.Add("Description = This is the latest version of MyEditor")
contentList.Add("Feature = Support for other file types")
contentList.Add("Feature1 = Support for different encodings")
contentList.Add("BugFix = Fix bug with file open")
contentList.Add("BugFix1 = Fix crash when opening large files")
contentList.Add("BugFix2 = Fix bug with search in file feature")
contentList.Add("FilePath = % ProgramFiles %\MyEditor\MyEditor.exe")
contentList.Add("Version = 4.5")
Return contentList
End Function
Public Function GetDataItem(value As String) As DataItem
Dim parts = value.Split("=", 2, StringSplitOptions.None)
Dim dataItem = New DataItem()
dataItem.DataType = parts(0).Trim()
dataItem.Data = parts(1).Trim()
Return dataItem
End Function
End Class
Public Class DataItem
Public DataType As String
Public Data As String
End Class
Or, in C#:
void Main()
{
List<string> fileContent = GetFileContent();
var dataItems = fileContent.Where(c => c.Contains("="))
.Select(c => GetDataItem(c));
dataItems.Dump();
}
public List<string> GetFileContent()
{
List<string> contentList = new List<string>();
contentList.Add("sb.app; aiu;");
contentList.Add("");
contentList.Add("[MyEditor45]");
contentList.Add("Name = MyEditor 4.5");
contentList.Add("URL = http://www.myeditor.com/download/myeditor.msi");
contentList.Add("Size = 3023788");
contentList.Add("Description = This is the latest version of MyEditor");
contentList.Add("Feature = Support for other file types");
contentList.Add("Feature1 = Support for different encodings");
contentList.Add("BugFix = Fix bug with file open");
contentList.Add("BugFix1 = Fix crash when opening large files");
contentList.Add("BugFix2 = Fix bug with search in file feature");
contentList.Add("FilePath = % ProgramFiles %\\MyEditor\\MyEditor.exe");
contentList.Add("Version = 4.5");
return contentList;
}
public DataItem GetDataItem(string value)
{
var parts = value.Split('=');
var dataItem = new DataItem()
{
DataType = parts[0],
Data = parts[1]
};
return dataItem;
}
public class DataItem
{
public string DataType;
public string Data;
}
The given answer only focuses on the first part, converting the data to a structure that can be shaped for display. But I think you main question is how to do the actual shaping.
I used a somewhat different way to collect the file data, using Microsoft.VisualBasic.FileIO.TextFieldParser because I think that makes coding just al little bit easier:
Iterator Function GetTwoItemLines(fileName As String, delimiter As String) _
As IEnumerable(Of Tuple(Of String, String))
Using tfp = New TextFieldParser(fileName)
tfp.TextFieldType = FieldType.Delimited
tfp.Delimiters = {delimiter}
tfp.HasFieldsEnclosedInQuotes = False
tfp.TrimWhiteSpace = False
While Not tfp.EndOfData
Dim arr = tfp.ReadFields()
If arr.Length >= 2 Then
Yield Tuple.Create(arr(0).Trim(), String.Join(delimiter, arr.Skip(1)).Trim())
End If
End While
End Using
End Function
Effectively the same thing happens as in your code, but taking into account Andrew's keen caution about data loss: a line is split by = characters, but the second field of a line consists of all parts after the first part with the delimiter re-inserted: String.Join(delimiter, arr.Skip(1)).Trim().
You can use this function as follows:
Dim fileContent = GetTwoItemLines(file, "=")
For display, I think the best approach (most efficient in terms of lines of code) is to group the lines by their first items, removing the numeric part at the end:
Dim grouping = fileContent.GroupBy(Function(c) c.Item1.TrimEnd("0123456789".ToCharArray())) _
.Where(Function(k) k.Key = "Feature" OrElse k.Key = "BugFix" OrElse k.Key = "Enhancement")
Here's a Linqpad dump (in which I took the liberty to change one item a bit to demonstrate the correct dealing with multiple = characters:
You could do it with Regular Expressions:
Imports System.Text.RegularExpressions
Private Function InfoReader(ByVal sourceText As String) As List(Of Dictionary(Of String, String()))
'1) make array of fragments for each product info
Dim products = Regex.Split(sourceText, "(?=\[\s*\w+\s*])")
'2) declare variables needed ahead
Dim productProperties As Dictionary(Of String, String)
Dim propertyNames As String()
Dim productGroupedProperties As Dictionary(Of String, String())
Dim result As New List(Of Dictionary(Of String, String()))
'2) iterate along fragments
For Each product In products
'3) work only in significant fragments ([Product]...)
If Regex.IsMatch(product, "\A\[\s*\w+\s*]") Then
'4) make array of property lines and extract dictionary of property/description
productProperties = Regex.Split(product, "(?=^\w+\s*=)", RegexOptions.Multiline).Where(
Function(s) s.Contains("="c)
).ToDictionary(
Function(s) Regex.Match(s, "^\w+(?=\s*=)").Value,
Function(s) Regex.Match(s, "(?<==\s+).*(?=\s+)").Value)
'5) extract distinct property names, ignoring numbered repetitions
propertyNames = productProperties.Keys.Select(Function(s) s.TrimEnd("0123456789".ToCharArray)).Distinct.ToArray
'6) make dictionary of distinctProperty/Array(Of String){description, description1, ...}
productGroupedProperties = propertyNames.ToDictionary(
Function(s) s,
Function(s) productProperties.Where(
Function(kvp) kvp.Key.StartsWith(s)
).Select(
Function(kvp) kvp.Value).ToArray)
'7) enlist dictionary to result
result.Add(productGroupedProperties)
End If
Next
Return result
End Function
I am using camel-bindy to unmarshall a CSV to Java Object. Is it possible to ignore a particular column?
Consider following example, I don't want to map column 3 (Address). Please let me know if there is a way to do so. In reality I have more than 10 columns in my CSV that I want to ignore.
Example :-
CSV File:
Header : Name,Mobile,Address
Data Row : Rabbit,007,Rabbit Hole
Bindy mapping in Java class:
#CsvRecord(separator = "," , skipFirstLine = true)
public class Contacts {
#DataField(pos = 1, trim=true)
private String name;
#DataField(pos = 2, required = true, trim=true)
private Long Mobile;
Thanks for your time!
The latest version supports skipField
#CsvRecord(separator = ",",skipField =true)
You cannot skip a column. Bindy iterates through every token and checks if there is an associated data field, see BindyCsvFactory:
// Get DataField from model
DataField dataField = dataFields.get(pos);
ObjectHelper.notNull(dataField, "No position " + pos + " defined for the field: " + data + ", line: " + line);
The only solution is to define a class attribute that is just ignored:
#DataField(pos = 1)
public String ingoreMe;
Based upon a prior question I had Here. I wanted to join two collections together so that the merged data could be then displayed within a DataGrid of a Silverlight UI.
Using the Linq Zip function I was able to merge my two collections as follows
public void CombineAllCollections()
{
var combineAll = Persons.Zip(PhoneNumbers, (x, y) => new
{
FirstName = x.FirstName,
LastName = x.LastName,
Address = x.Address,
State = x.State,
Zip = x.Zip,
AreaCode = y.AreaCode,
PhoneNumber = y.Number
});
}
This seems to do just what I wanted. However the output is an Anonymous Type. How can I cast the combinedAll type to a collection (List, ObservableColl. IENum) that I can then pass to a view in my UI ( Bind to a datagrid) . The output should then display within the grid a column and value for each item ( as listed above seven columns).
Thanks in advance
-Cheers
Instead of making an anonymous type, you could create a concrete type specifically for the merged results with all of the properties you specified on the anonymous type.
For example:
public void CombineAllCollections()
{
var combineAll = Persons.Zip(PhoneNumbers, (x, y) => new PersonWithPhoneNumber
{
FirstName = x.FirstName,
LastName = x.LastName,
Address = x.Address,
State = x.State,
Zip = x.Zip,
AreaCode = y.AreaCode,
PhoneNumber = y.Number
});
}
Where PersonWithPhoneNumber is the type with all of the specified properties.
Then you can call ToList() on the result to convert it to an IList<PersonWithPhoneNumber> or you can leave it in the form it is in as an IEnumerable<PersonWithPhoneNumber>
EDIT
Given the information that you provided, it would appear about the only effective way to store so many values while maintaining the ability to use LINQ and not have to define a separate type would be to create a dictionary for each item in the zipped collection, for example:
using System;
using System.Collections.Generic;
using System.Linq;
namespace Test
{
class Program
{
static void Main(string[] args)
{
var xs = new[] {1, 2, 3, 4, 5, 6};
var ys = new[] {7, 8, 9, 10, 11, 12};
var zipped = xs.Zip(ys, (x, y) =>
new Dictionary<string, object>
{
{ "X", x},
{ "Y", y}
});
foreach (var pair in zipped.SelectMany(tuple => tuple))
{
Console.WriteLine("{0} = {1}", pair.Key, pair.Value);
}
}
}
}
This way, you can have as many Key/Value pairs in the dictionary for each item as you would like. However, you essentially lose any type safety using this method and will end up boxing all value types for each property. This means you will have to know the type to cast back to. I would honestly not recommend doing this. I would simply just make a type, even if that type needs 100 properties. In this scenario auto-properties and a good text editor are your best friend.
You could also us the new dynamic type in C# 4 with the .NET 4.0 framework as in the following example:
using System;
using System.Dynamic;
using System.Linq;
namespace Test
{
class Program
{
static void Main(string[] args)
{
var xs = new[] {1, 2, 3, 4, 5, 6};
var ys = new[] {7, 8, 9, 10, 11, 12};
var zipped = xs.Zip(ys, (x, y) =>
{
dynamic value = new ExpandoObject();
value.X = x;
value.Y = y;
return value;
});
foreach (var pair in zipped)
{
Console.WriteLine("X = {0}, Y = {1}", pair.X, pair.Y);
}
}
}
}
This will introduce dynamic into your code, though. Which may not have the performance metrics you desire and can lead to a lot of run-time errors if you don't keep an eye out on what you're doing. This is because you could misspell a property while creating or accessing it (this is also an issue with the dictionary setup) which will lead to a run-time exception when you attempt to access it. Again, probably best to just use a concrete, named typed in the end.