Solr BlockJoin Indexing for Solr 4.10.1 - solr

Question

I am trying to index a nested structure as below and having difficulty indexing both with SOlrJ and the DIH. I have battled with this for a while and would really appreciate some help on this.
How do i fix this with either SolrJ or DIH.
Thanks
What i want my data to look like my index:
"docs": [
{
"name": "MR INCREDIBLE ",
"id": 101,
"job": "super hero",
"_version_": "1483934897344086016"
"children": [
{
"c_name":"Violet"
"c_age":10
"c_gender":"female"
},
{
"c_name":"Dash"
"c_age":8
"c_gender":"male"
}
]
}
]
My schema.xml
<schema name="datasearch" version="1.5">
<uniqueKey>id</uniqueKey>
<fields>
<field name="_version_" type="long" indexed="true" stored="true" />
<field name="_root_" type="string" indexed="true" stored="false"/>
<field name="id" type="string" indexed="true" stored="true" />
<field name="name" type="text" indexed="true" stored="true" />
<field name="job" type="string" indexed="true" stored="true"/>
<!-- I want to add children here -->
<!-- <field name="children" indexed="true" stored="true"/> -->
<field name="c_name" type="string" indexed="true" stored="true"/>
<field name="c_age" type="int" indexed="true" stored="true"/>
<field name="c_sex" type="string" indexed="true" stored="true"/>
</fields>
<types>
<fieldType name="string" class="solr.TrieLongField" />
<fieldType name="int" class="solr.TrieIntField" />
<fieldType name="date" class="solr.TrieDateField" omitNorms="true" />
<fieldType name="long" class="solr.StrField" sortMissingLast="true"/>
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
</fieldType>
</types>
<defaultSearchField>name</defaultSearchField>
</schema>
SolrJ Attempt
val serverUrl = current.configuration.getString("solr.server.url").get
val solr = new HttpSolrServer(serverUrl)
def testAddChildDoc={
val doc = {
new SolrInputDocument(){
addField("id", "101")
addField("name", "Mr Incredible")
}
}
val c1 = new SolrInputDocument(){
addField("c_name", "violet")
addField("c_age", 10)
}
val c2 = new SolrInputDocument(){
addField("c_name", "dash")
addField("c_age", 8)
}
doc.addChildDocument(c1)
doc.addChildDocument(c2)
solr.deleteByQuery("*:*")
solr.add(doc)
solr.commit(true, true)
}
Response
=>ERROR org.apache.solr.core.SolrCore – org.apache.solr.common.SolrException: [doc=null] missing required field: id
[RemoteSolrException: [doc=null] missing required field: id]
So i go ahead and add id to childDocs making the above
...
val c1 = new SolrInputDocument(){
addField("id", "101")
addField("c_name", "violet")
addField("c_age", 10)
}
val c2 = new SolrInputDocument(){
addField("id", "101")
addField("c_name", "dash")
addField("c_age", 8)
}
.....
Then rerun the get-all query, now i get the results below
SolrJ Attempt 2 plus get-all query
{
"responseHeader": {
"status": 0,
"QTime": 0,
"params": {
"indent": "true",
"q": "*:*",
"_": "1415194092582",
"wt": "json"
}
},
"response": {
"numFound": 3,
"start": 0,
"docs": [
{
"id": 101,
"c_name": violet,
"c_age": "10",
},
{
"id": 101,
"c_name": dash,
"c_age": "8"
},
{
"id": 101,
"name": "Mr Incredible",
"_version_": "1483938552238571520"
}
]
}
}
So i give up here and try the DIH as below
db-dataconfig.xml
<dataConfig>
<dataSource type="JdbcDataSource"
driver="org.postgresql.Driver"
url="jdbc:postgresql://xxx:5432/xxxx"
user="xx" password="xx"
readOnly="true" autoCommit="false" transactionIsolation="TRANSACTION_READ_COMMITTED" holdability="CLOSE_CURSORS_AT_COMMIT" />
<document>
<entity name="parent" query="select id,name, job from PARENTS LIMIT 1" >
<field column="name"/>
<field column="id"/>
<field column="job"/>
<entity child="true" name="children" query="select c_name, c_gender, c_age from CHILDREN" where="pid = ${parent.id}" processor="CachedSqlEntityProcessor">
<field column="c_age" />
<field column="c_gender" />
<field column="c_name"/>
</entity>
</entity>
</document>
</dataConfig>
query get-all after full import with DIH as above and no children indexed
{
"responseHeader": {
"status": 0,
"QTime": 0,
"params": {
"indent": "true",
"q": "*:*",
"_": "1415195060664",
"wt": "json"
}
},
"response": {
"numFound": 1,
"start": 0,
"docs": [
{
"name": "Mr Incredible",
"id": 101,
"_version_": "1483939357483073536"
}
]
}
}

Answer 1

To be able to use child="true" in DIH apply the patch from https://issues.apache.org/jira/browse/SOLR-5147 (I think it's the same DIH patch at solr-3076).
The patch itself seems to be incompatible in neglectable details with the current trunk.

Answer 2

In order to get the following response from Solr 4.10.1
{
"name": "MR INCREDIBLE ",
"id": 101,
"job": "super hero",
"type": "parent",
"_root_":"101"
"_version_": "1483934897344086016"
"childDocuments": [
{
"c_name":"Violet",
"c_age":10,
"c_gender":"female",
"id":"101_Violet",
"_root_":"101"
},
{
"c_name":"Dash",
"c_age":8,
"c_gender":"male",
"id":"101Dash",
"_root_":"101"
}
]
}
"type" field needs to be defined in the schema to differentiate between parent and child documents:
<fields>
<field name="_version_" type="long" indexed="true" stored="true" />
<field name="_root_" type="string" indexed="true" stored="false"/>
<field name="id" type="string" indexed="true" stored="true" />
<field name="name" type="text" indexed="true" stored="true" />
<field name="job" type="string" indexed="true" stored="true"/>
<field name="c_name" type="string" indexed="true" stored="true"/>
<field name="c_age" type="int" indexed="true" stored="true"/>
<field name="c_gender" type="string" indexed="true" stored="true"/>
<field name="type" type="string" indexed="true" stored="true" />
</fields>
Child documents also need to have an unique "id", just like any other document.
All the documents in the index should be in parent/child relation, otherwise the queries may return unexpected results. In case you need documents which are neither parents or children, assign them a fake parent.
SolrJ
To work with child/parent docs, solrj.jar version 4.5 or higher is required.
SolrServer solr = new HttpSolrServer(serverUrl);
SolrInputDocument doc = new SolrInputDocument();
String id = "101";
doc.addField("id", id);
doc.addField("name", "Mr Incredible");
doc.addField("job", "super hero");
doc.addField("type", "parent");
SolrInputDocument childDoc1 = new SolrInputDocument();
String name1 = "Violet";
childDoc1.addField("id", id + "_" + name1);
childDoc1.addField("c_name", name1);
childDoc1.addField("c_age", 10);
childDoc1.addField("c_gender", "female");
doc.addChildDocument(childDoc1);
SolrInputDocument childDoc2 = new SolrInputDocument();
String name2 = "Dash";
childDoc2.addField("id", id + "_" + name2);
childDoc2.addField("c_name", name2);
childDoc2.addField("c_age", 8);
childDoc2.addField("c_gender", "male");
doc.addChildDocument(childDoc2);
solr.add(doc);
solr.commit();
Finally, the query looks like this:
http://localhost/solr/core/select?q={!parent which='type:parent'}&fl=*,[child parentFilter=type:parent]&wt=json&indent=true
To get only results of female gender:
http://localhost/solr/core/select?q={!parent which='type:parent'}c_gender:female&fl=*,[child parentFilter=type:parent childFilter=c_gender:female]&wt=json&indent=true