Flat file to json conversion using beanIo - maps

Question

I am trying to parse a fixedlength flat file using beanIo to json
Code:
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.util.Map;
import org.beanio.BeanIOConfigurationException;
import org.beanio.BeanReader;
import org.beanio.StreamFactory;
import org.junit.Test;
import com.google.gson.Gson;
public class EmployeeBeanIOHandlerTest {
#Test
public void testHandleEmployee() {
// mapping pattern file
String mappingPatternFile = "pattern-mapping.xml";
// data file (csv)
String objectFile = "employee.csv";
// stream name defined in pattern mapping file
String streamName = "empData";
Gson gson = new Gson();
BeanReader beanReader = null;
Reader reader = null;
StreamFactory factory = null;
InputStream in = null;
try {
System.out.println("## RESULT FOR " + objectFile + " ##");
// create a StreamFactory
factory = StreamFactory.newInstance();
// load the setting file
in = this.getClass().getClassLoader()
.getResourceAsStream(mappingPatternFile);
// get input stream reader of object file (data file)
reader = new InputStreamReader(this.getClass().getClassLoader()
.getResourceAsStream(objectFile));
// load input stream to stream factory
factory.load(in);
beanReader = factory.createReader(streamName, reader);
Map<?, ?> record = null;
while ((record = (Map<?, ?>) beanReader.read()) != null) {
System.out.println(beanReader.getRecordName() + ": "
+ gson.toJson(record));
}
} catch (BeanIOConfigurationException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
in.close();
if (beanReader != null) {
beanReader.close();
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
However the output i see:
header: {"id":"Header","date":"01012013"}
emp: {"lastName":"Lilik","title":"Senior Developer","hireDate":"Oct 1, 2009
12:00:00 AM","salary":7500000,"firstName":"Robertus"}
emp: {"lastName":"Doe","title":"Architect","hireDate":"Jan 15, 2008 12:00:00 AM","salary":8000000,"firstName":"Jane"}
emp: {"lastName":"Anderson","title":"Manager","hireDate":"Mar 18, 2006 12:00:00 AM","salary":9000000,"firstName":"Jon"}
trailer: {"id":"Trailer","count":"3"}
Which generates separate json object for each record found.
Reference site:
http://www.sourcefreak.com/2013/06/painless-flat-file-parsing-with-beanio/
Below is my requirement:
I want to have a consolidated Json file.
In case of duplicate record it should form an json array.
I would appreciate a help on this.

Answer 1

This answer is based on the data and pattern-mapping.xml file found in the link provided by the OP.
Data:
Header,01012013
Robertus,Lilik,Senior Developer,"75,000,00",10012009
Jane,Doe,Architect,"80,000,00",01152008
Jon,Anderson,Manager,"90,000,00",03182006
Footer,3
Mapping File:
This is the modified pattern-mapping.xml file. Note the use of a <group> element (myGroup) to encapsulate everything into a single group, which will force the BeanReader to read everything in one go. I also changed the maxOccurs to be 1 (one) for both the Header and the Footer records. Also, added the collection="list" attribute to theemp` record
<?xml version='1.0' encoding='UTF-8' ?>
<beanio xmlns="http://www.beanio.org/2012/03"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.beanio.org/2012/03 http://www.beanio.org/2012/03/mapping.xsd">
<stream name="empData" format="csv">
<group name="myGroup" class="map">
<record name="header" class="map" ridLength="0-2" maxOccurs="1">
<field name="id" rid="true" maxOccurs="1" literal="Header" />
<field name="date" />
</record>
<record name="emp" class="map" ridLength="4-5" collection="list">
<field name="firstName" />
<field name="lastName" />
<field name="title" />
<field name="salary" type="java.math.BigDecimal" format="#,###,###,00" />
<field name="hireDate" type="java.util.Date" format="MMddyyyy" minOccurs="0" />
</record>
<record name="trailer" class="map" ridLength="2" maxOccurs="1">
<field name="id" />
<field name="count" />
</record>
</group>
</stream>
</beanio>
Using the test case supplied and the modified mapping file, we get this result (reformatted by me):
myGroup: {
"trailer": {
"count": "3",
"id": "Footer"
},
"header": {
"date": "01012013",
"id": "Header"
},
"emp": [
{
"firstName": "Robertus",
"lastName": "Lilik",
"hireDate": "Oct 1, 2009 12:00:00 AM",
"title": "Senior Developer",
"salary": 7500000
},
{
"firstName": "Jane",
"lastName": "Doe",
"hireDate": "Jan 15, 2008 12:00:00 AM",
"title": "Architect",
"salary": 8000000
},
{
"firstName": "Jon",
"lastName": "Anderson",
"hireDate": "Mar 18, 2006 12:00:00 AM",
"title": "Manager",
"salary": 9000000
}
]
}
Hope this helps