I am trying to implement the auto suggest of solr this is the changes that I made in solrconfig.xml file
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookupFactory</str>
<str name="field">displayName</str> <!-- the indexed field to derive suggestions from -->
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
when I try to query with sample input as 'p'
http://localhost:8983/solr/food/suggest?q=p&wt=json&indent=true
it returns 5 words
"pizza", "potato", "pasta", "protein", "premium"
but in the displayName field I got words like paneer , palak etc which is not showing up why is it so?
Can you added the following to your configuration and run the below query. Don't forget to reload the solr core after putting these changes.
<str name="suggestAnalyzerFieldType">string</str>
<str name="storeDir">suggester_fuzzy_dir</str>
http://localhost:8983/solr/food/suggest?suggest=true&suggest.build=true&suggest.dictionary=suggest&wt=json&suggest.q=p&suggest.count=10
Related
I am having a lock issue when trying to initialize a Solr core:
SolrCore Initialization Failures:
MySolrCore: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: org.apache.lucene.store.LockObtainFailedException: Lock held by this virtual machine: C:\solr-6.0.1\server\solr\MySolrCore\data\analyzingInfixSuggesterIndexDir\write.lock
I am trying to implement multiple spell checkers on this core and I get this lock error.
Some interesting notes from trying different approaches:
I tried extending the writeLockTimeout to 30 seconds and that did nothing.
None of the spellcheckIndexDir folders are getting created. I added them in hopes of fixing the lock issue. I even tried creating them manually, and no files were added to them.
If I comment out one of the components and handlers, the other one works.
If I comment out the spellecheckers that use the AnalyzingInfixLookupFactory then both of the spellcheckers that use the WFSTLookupFactory and both of the handlers work. However, the spellcheckIndexDir is not created.
How do I fix the lock issue? I think that if I can get all of the spellcheckIndexDir specified to work that the indexes will be located in different folders and there won't be a lock. Thanks for any help!
Here's my Solr config search components and handlers for the two fields.
<searchComponent name="NameSuggest" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="name">NameFST</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory</str>
<str name="field">NameFST</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./NameFSTSuggestIndexDir</str>
</lst>
<lst name="spellchecker">
<str name="name">NameAI</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
<str name="field">NameAI</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./NameAISuggestIndexDir</str>
</lst>
<str name="queryAnalyzerFieldType">suggest_phrase</str>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/NameSuggest" startup="lazy">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">NameFST</str>
<str name="spellcheck.dictionary">NameAI</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.collate">false</str>
<str name="spellcheck.extendedResults">true</str>
</lst>
<arr name="components">
<str>NameSuggest</str>
</arr>
</requestHandler>
<searchComponent name="DescriptionSuggest" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="name">DescriptionFST</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory</str>
<str name="field">DescriptionFST</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./DescriptionFSTSuggestIndexDir</str>
</lst>
<lst name="spellchecker">
<str name="name">DescriptionAI</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
<str name="field">DescriptionAI</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./DescriptionAISuggestIndexDir</str>
</lst>
<str name="queryAnalyzerFieldType">suggest_phrase</str>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/DescriptionSuggest" startup="lazy">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">DescriptionFST</str>
<str name="spellcheck.dictionary">DescriptionAI</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.collate">false</str>
<str name="spellcheck.extendedResults">true</str>
</lst>
<arr name="components">
<str>DescriptionSuggest</str>
</arr>
</requestHandler>
Instead of spellcheckIndexDir I needed to use indexPath. It's working now.
<!-- <str name="spellcheckIndexDir">./NameAISuggestIndexDir</str> -->
<str name="indexPath">./NameAISuggestIndexDir</str>
I am using SOLR to generate suggestions. I have given a float field named length (tfloat) as the weightField in solrconfig.xml. But in my suggestions, this value does not come up in the suggestion response and shows as zero. I was expecting the suggestions to be sorted by weight and the weight to hold the value of length which represents the string length of the suggestions. I am using solr 6.2.1
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="indexPath">suggester_infix_dir</str>
<str name="payloadField">payload</str>
<str name="weightField">lengthval</str>-->
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">value</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<str name="buildOnStartup">true</str>
<str name="buildOnCommit">true</str>
<str name="highlight">false</str>
<requestHandler name="/suggesthandler" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">900</str>
<str name="suggest.dictionary">mySuggester</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
Hello it started working , not sure what was wrong
All:
I am using solr 4.10 default configuration now, I wonder if any one can give me a working configuration for suggester part? I tried the example in Solr in Action, but it does not work(I tried this /suggest handler in admin page, keeping other options default only specifying a keyword in q ).
<requestHandler name="/suggest"
class="org.apache.solr.handler.component.SearchHandler">
<lst name="defaults">
<str name="echoParams">none</str>
<str name="wt">json</str>
<str name="indent">false</str>
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggestDictionary</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">false</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggestDictionary</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst
.FSTLookupFactory</str>
<str name="field">suggest</str>
<float name="threshold">0.</float>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
Thanks
I'm trying to use the suggest component (solr 4.6) with multiple cores. I have added a search component and a request handler in my solrconfig. That works fine for 1 core but querying my solr instance with the shards parameter does not work.
But did you mean' (spell check ) is working fine with multiple cores using shard.
Here is the configuration part of solrconfig file :
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggestDictionary</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookupFactory</str>
<str name="field">suggest</str>
<float name="threshold">0.0005</float>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="org.apache.solr.handler.component.SearchHandler">
<lst name="defaults">
<str name="echoParams">none</str>
<str name="wt">xml</str>
<str name="indent">false</str>
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggestDictionary</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">false</str>
<str name="qt">/suggest</str>
<str name="shards.qt">/suggest</str>
<str name="shards">localhost:8080/cores/core1,localhost:8080/cores/core2</str>
<bool name="distrib">false</bool>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
<shardHandlerFactory class="HttpShardHandlerFactory">
<int name="socketTimeOut">1000</int>
<int name="connTimeOut">5000</int>
</shardHandlerFactory>
</requestHandler>
It works for me..
You can get the suggestions using this RestURL
http://localhost:8983/solr/demo/spell?q=howoo&wt=json&indent=true&qt=spell&shards.qt=/spell&shards=localhost:8983/solr/demo_shard2_replica1,localhost:8983/solr/demo_shard1_replica2
OR Simply use this :
http://localhost:8983/solr/demo/spell?q=hoo&wt=json&indent=true&shards.qt=/spell
shards.qt=/spell : Need to add that allows suggestion on shards
Here, you have make changes and apply for things which requires.
Collection = demo
Shards = demo_shard2_replica1, demo_shard1_replica2
Replace collection and shards names with your names of collection and shards.
I am trying to set up spellchecker, according to solr documentation. But when I am testing, I don't have any suggestion. My piece of code follows:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textSpell</str>
<lst name="spellchecker">
<str name="classname">solr.IndexBasedSpellChecker</str>
<str name="name">default</str>
<str name="field">name</str>
<str name="spellcheckIndexDir">./spellchecker</str>
</lst>
<str name="queryAnalyzerFieldType">textSpell</str>
</searchComponent>
<requestHandler name="/spellcheck" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<!-- Optional, must match spell checker's name as defined above, defaults to "default" -->
<str name="spellcheck.dictionary">default</str>
<!-- omp = Only More Popular -->
<str name="spellcheck.onlyMorePopular">false</str>
<!-- exr = Extended Results -->
<str name="spellcheck.extendedResults">false</str>
<!-- The number of suggestions to return -->
<str name="spellcheck.count">1</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
The query I send to Solr:
q=%2B%28text%3A%28gasal%29%29&suggestField=contentOriginal&ontologySeed=gasal&spellcheck.build=true&spellcheck.q=gasal&spellcheck=true&spellcheck.collate=true&hl=true&hl.snippets=5&hl.fl=text&hl.fl=text&rows=12&start=0&qt=%2Fsuggestprobabilistic
Does anybody know why?? Thanks in advance
First, don't repeat queryAnalyzerFieldType twice in the component configuration.
It is recommended not to use a /spellcheck handler but instead to bind the spellcheck component to the standard query handler (or dismax if it is what you use) like this:
<requestHandler name="standard" class="solr.SearchHandler" default="true">
<lst name="defaults">
...
</lst>
<arr name="last-components">
<str>spellcheck</str>
...
</arr>
</requestHandler>
You can then call it like this:
http://localhost:8983/solr/select?q=komputer&spellcheck=true
Also don't forget to build the spellcheck dictionary before you use it:
http://localhost:8983/solr/select/?q=*:*&spellcheck=true&spellcheck.build=true
You can force the dictionary to build at each commit by configuring it in the component:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textSpell</str>
<lst name="spellchecker">
<str name="classname">solr.IndexBasedSpellChecker</str>
<str name="name">default</str>
<str name="field">name</str>
<str name="spellcheckIndexDir">./spellchecker1</str>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
Finally, make sure that your name field is really an indexed field of type textSpell and that it contains enough content to build a good dictionary. In my case, I have a field named spellchecker that is populated from a couple of fields of my index (using copyField instructions in the schema).