Re-do solr query with spellchecking suggestions - solr

Using the following configuration to run the spellchecking on Solr queries:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textTitle</str>
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">text</str>
<str name="classname">solr.DirectSolrSpellChecker</str>
<str name="distanceMeasure">internal</str>
<float name="accuracy">0.7</float>
<int name="maxEdits">2</int>
<int name="minPrefix">1</int>
<int name="maxInspections">10</int>
<int name="minQueryLength">3</int>
<float name="maxQueryFrequency">0.10</float>
<float name="thresholdTokenFrequency">.00001</float>
</lst>
</searchComponent>
<requestHandler name="standard" class="solr.StandardRequestHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
I get the correct suggestions for bad spelled queries, however, is it possible to re-do the query again to return the results for the spellchecked terms instead of just returning the suggestions?

Related

SOLR: Spell check not working with select query

In the SOLR admin, we can see there is a spellcheck option but it is not showing the result.
How this is works with the select query.
If I searched with the spell URL direct, It gives me result as expected
http://localhost:8983/solr/prashant1/spell?q=blakc&spellcheck=on&wt=json
Result
{
"responseHeader":{
"status":0,
"QTime":8},
"response":{"numFound":0,"start":0,"docs":[]
},
"spellcheck":{
"suggestions":[
"blakc",{
"numFound":10,
"startOffset":0,
"endOffset":5,
"origFreq":0,
"suggestion":[{
"word":"black",
"freq":65146},
{
"word":"blanc",
"freq":151},
{
"word":"blake",
"freq":10},
{
"word":"blac",
"freq":2},
{
"word":"block",
"freq":1863},
{
"word":"blanca",
"freq":32},
{
"word":"blank",
"freq":31},
{
"word":"blade",
"freq":23},
{
"word":"blacks",
"freq":12},
{
"word":"blanco",
"freq":11}]}],
"correctlySpelled":false,
"collations":[]}}
But I need the same result with the select query which is not working from the SOLR admin.
Solrconfig.xml
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">text_general</str>
<!-- Multiple "Spell Checkers" can be declared and used by this
component
-->
<!-- a spellchecker built from a field of the main index -->
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">Name</str>
<str name="classname">solr.DirectSolrSpellChecker</str>
<!-- the spellcheck distance measure used, the default is the internal levenshtein -->
<str name="distanceMeasure">internal</str>
<!-- minimum accuracy needed to be considered a valid spellcheck suggestion -->
<float name="accuracy">0.5</float>
<!-- the maximum #edits we consider when enumerating terms: can be 1 or 2 -->
<int name="maxEdits">2</int>
<!-- the minimum shared prefix when enumerating terms -->
<int name="minPrefix">1</int>
<!-- maximum number of inspections per result. -->
<int name="maxInspections">5</int>
<!-- minimum length of a query term to be considered for correction -->
<int name="minQueryLength">4</int>
<!-- maximum threshold of documents a query term can appear to be considered for correction -->
<float name="maxQueryFrequency">0.01</float>
<!-- uncomment this to require suggestions to occur in 1% of the documents
<float name="thresholdTokenFrequency">.01</float>
-->
</lst>
</searchComponent>
<requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="spellcheck.dictionary">default</str>
<str name="spellcheck">on</str>
<str name="spellcheck.extendedResults">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.alternativeTermCount">5</str>
<str name="spellcheck.maxResultsForSuggest">5</str>
<str name="spellcheck.collate">true</str>
<str name="spellcheck.collateExtendedResults">true</str>
<str name="spellcheck.maxCollationTries">10</str>
<str name="spellcheck.maxCollations">5</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
It should work with:
http://localhost:8983/solr/prashant1/select?q=Name%3Ablakc&spellcheck.q=blakc&spellcheck=on
Is there any setting and steps to be done?
Try by adding the spellcheck component to the standard query handler like
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
<str name="spellcheck">on</str>
<str name="spellcheck.extendedResults">true</str>
<str name="spellcheck.count">10</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
You can then call it like this:
http://localhost:8983/solr/select?q=yogik&spellcheck=true
Also don't forget to build the spellcheck dictionary before you use it:
http://localhost:8983/solr/select/?q=*:*&spellcheck=true&spellcheck.build=true

Solr SpellChecking suggestions

When I use the "spellcheck" in Solr I have always 1 result even if it should be more. For example if I have the word "conjecture" and I search "conjicture" I will find it, the same thing if I have the word "conjoncture" but if I have the two words (conjecture and conjoncture) I will have only one result.
Currently, I'm using this :
searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">text_spell</str>
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">copie_content</str>
<str name="classname">solr.DirectSolrSpellChecker</str>
<str name="distanceMeasure">internal</str>
<float name="accuracy">0.5</float>
<int name="maxEdits">2</int>
<int name="minPrefix">0</int>
<int name="maxInspections">10</int>
<int name="minQueryLength">4</int>
<float name="maxQueryFrequency">0.01</float>
<str name="spellcheck.maxResultsForSuggest">5</str>
</lst>
</searchComponent>

Any working solr suggester example?

All:
I am using solr 4.10 default configuration now, I wonder if any one can give me a working configuration for suggester part? I tried the example in Solr in Action, but it does not work(I tried this /suggest handler in admin page, keeping other options default only specifying a keyword in q ).
<requestHandler name="/suggest"
class="org.apache.solr.handler.component.SearchHandler">
<lst name="defaults">
<str name="echoParams">none</str>
<str name="wt">json</str>
<str name="indent">false</str>
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggestDictionary</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">false</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggestDictionary</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst
.FSTLookupFactory</str>
<str name="field">suggest</str>
<float name="threshold">0.</float>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
Thanks

Not able to read results of SuggestComponent using SolrJ API

I am using SuggestComponent to get auto suggest keywords. I configured search component and request handler correctly in solrconfig.xml. I am getting expected results also.
Now my problem is i am not able to read XML output using SolrJ api. i spent couple of hours on SolrJ API, but not able to find method to read suggested keywords.
My Java Code
SolrServer server = //solrserver object creation
ModifiableSolrParams params = new ModifiableSolrParams();
params.set("qt", "/suggest");
params.set("q", q);
QueryResponse rsp = server.query(params);
**//How to read from QueryResponse?**
Here is my solrconfig.xml configuration
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">name</str>
<str name="weightField">reviewsNum</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="suggest">true</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">mySuggester</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
Response:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">24</int>
</lst>
<lst name="suggest">
<lst name="mySuggester">
<lst name="a">
<int name="numFound">10</int>
<arr name="suggestions">
<lst>
<str name="term">JNTU <b>A</b>nantapur</str>
<long name="weight">2</long>
<str name="payload"/>
</lst>
<lst>
<str name="term"><b>A</b>merican School of Bombay</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term">
Society For <b>A</b>dvanced Study In Rehabilitation
</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term"><b>a</b>mrita vishwa vidyapeetham</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term">
shanmugha <b>a</b>rts science technology & research <b>a</b>cademy (sastra)
</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term">Montesorri Hs <b>A</b>lampur</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term"><b>A</b>ditya Ps Nehru Road Proddutur</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term">Zphs <b>A</b>rakatavemula Rajupalem</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term">Sri <b>A</b>ditya Ps,pamur Pamuru</str>
<long name="weight">1</long>
<str name="payload"/>
</lst>
<lst>
<str name="term">
ramakrishna mission vivekananda educational <b>a</b>nd research institute
</str>
<long name="weight">0</long>
<str name="payload"/>
</lst>
</arr>
</lst>
</lst>
</lst>
</response>
You'll have to access the SpellCheckResponse property of the response:
rsp.getSpellCheckResponse()
This returns a SpellCheckResponse object, which contains the getSuggestions() method, which in turn returns a List of SpellCheckResponse.Suggestion objects.
List<SpellCheckResponse.Suggestion> suggestions = rsp.getSpellCheckResponse().getSuggestions()
You also want to use setRequestHandler("/suggest"), and not the qt parameter.

How to configure solrconfig.xml for suggester in solr?

I have configured my solrconfig.xml to use Suggester component as suggested in http://wiki.apache.org/solr/Suggester.
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup</str>
<str name="field">name</str>
<!-- the indexed field to derive suggestions from -->
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<!--
<str name="sourceLocation">american-english</str>
-->
</lst>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
However, when i browse from the browser url i am not getting any suggestions.
http://localhost:8080/solr/collection1/suggest?q=ava&fq=type%3ACategory&wt=xml
<response>
<lst name="responseHeader">
<int name="status">
0
</int>
<int name="QTime">
0
</int>
</lst>
<lst name="spellcheck">
<lst name="suggestions"/>
</lst>
</response>
Even if i remove the fq filter, I am not getting the suggestions.
Please help.
Thanks
I am not sure, can you try adding spellcheck=on
http://localhost:8080/solr/collection1/suggest?q=ava&spellcheck=on&fq=type%3ACategory&wt=xml

Resources