Solr is not showing result after indexing table records - solr

Question

The dataimporthandler status is showing it has indexed and 10 documents are added, but not showing any result when I search for word that is part of added document. If I give : in search it displays all records
example of clob record:
<?xml version="1.0" encoding="UTF-8" ?>
<message xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="someurl" xmlns:csp="someurl.xsd" xsi:schemaLocation="somelocation jar: id="002" message-type="create">
<content>
<dsp:row>
<dsp:channel>100</dsp:channel>
<dsp:role>115</dsp:role>
</dsp:row>
<![CDATA[ <ol><li>java</li></ol><li>ASP</li>]]>
</body></content></message>
data-config.xml
<document name="doc">
<entity name="MYCONTENT" transformer="ClobTransformer"
query="SELECT CID,XML FROM MYCONTENT">
<field column="CID" name="CID"/>
<field column="XML" clob="true" name="XML"/>
</entity>
</document>
schema.xml
<field name="CID" type="string" indexed="true" stored="true" required="true"/>
<field name="XML" type="string" indexed="true" stored="true" required="true"/>
<dynamicField name="*" type="ignored" />
<uniqueKey>CID</uniqueKey>
<defaultSearchField>XML</defaultSearchField>
solrconfig.xml
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">/data-config.xml</str>
<str name="rows">10</str>
</lst>
</requestHandler>
I do not know why it is not showing result when I search for "Java" "ASP". Any help is greatly appreciated.
thanks in advance
srini

Answer 1

You have two things to fix.
First, the "string" field type treats the entire document as a single token. You need a text field type.
Second, Solr does not parse the XML in your CLOB, it indexes it as raw text, splitting tokens as specified by your choice of tokenizer for that field. For example, if you used a whitespace tokenizer, it would treat "115" as a single token, and a search for "115" would not match.
For testing, I would try using the HTMLStripCharFilterFactory in that field definition before the tokenizer. See: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.HTMLStripCharFilterFactory
That should strip a fair amount of the XML. If you want to process it a specific way, you will probably want to learn about XPathEntityProcessor, which can extract parts of the XML for indexing. See: http://wiki.apache.org/solr/DataImportHandler#XPathEntityProcessor