I am trying to add autocomplete feature for the phrase queries. Have the following configuration -
in schema.xml file
<field name="textSpell" type="spell" indexed="true" stored="true"
multiValued="true" termVectors="true" termPositions="true"
termOffsets="true" />
<field name="suggest_phrase" type="suggest_phrase" indexed="true"
stored="false" multivalued="false"/>
in solrconfig.xml
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">default</str>
<str name="classname">solr.IndexBasedSpellChecker</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str> <!-- org.apache.solr.spelling.suggest.fst -->
<str name="dictionaryImpl">DocumentDictionaryFactory</str> <!-- org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory -->
<str name="field">textSpell</str>
<float name="thresholdTokenFrequency">.0001</float>
<!-- <str name="weightField">price</str>-->
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnCommit">true</str>
<!--<str name="buildOnOptimize">true</str>-->
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
<!-- Suggest Phrase -->
<searchComponent name="suggest_phrase" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="name">suggest_phrase</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup</str>
<str name="field">suggest_phrase</str>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
<requestHandler class="solr.SearchHandler"
name="/suggest_phrase" startup="lazy">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest_phrase</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.collate">false</str>
</lst>
<arr name="components">
<str>suggest_phrase</str>
</arr>
</requestHandler>
Also I am setting the following things - for the query to solr using solrj
SolrQuery suggestQuery = new SolrQuery();
suggestQuery.setParam(CommonParams.QT, "/terms");
suggestQuery.setParam(TermsParams.TERMS, true);
suggestQuery.setParam(TermsParams.TERMS_LIMIT, "5");
suggestQuery.setParam(TermsParams.TERMS_FIELD,"content");
suggestQuery.setParam(TermsParams.TERMS_LOWER, query);
suggestQuery.setParam(TermsParams.TERMS_PREFIX_STR, query);
suggestQuery.setParam("spellCheck", "true");
suggestQuery.setParam("spellcheck.q", query);
However, it doesn't yield results for phrase queries works only on single terms. Any suggestions. I am using Solr4.10.2
You are using two fieldTypes: "spell" and "suggest_phrase." How are they defined? The first thing I would check is whether or not you are using a WhitespaceTokenizerFactory on them - in which case, it wouldn't work over a phrase because the space in a phrase would terminate the token.
Related
I am trying to filter using a contextField in Solr 7.2. In solrconfig.xml I have the following:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">suggest_artist</str>
<str name="lookupImpl">BlendedInfixLookupFactory</str>
<str name="dictionaryimpl">DocumentDictionaryFactory</str>
<str name="field">artist</str>
<str name="weightField">monthly_dlds</str>
<str name="contextField">territory</str>
<str name="queryAnalyzerFieldType">phrase_suggest</str>
<str name="suggestAnalyzerFieldType">text_suggest</str>
<str name="buildOnStartup">true</str>
<str name="buildOnCommit">true</str>
<str name="storeDir">suggest_a</str>
<str name="indexPath">suggest_a</str>
<str name="highlight">false</str>
</lst>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="echoParams">all</str>
<str name="wt">json</str>
<str name="indent">true</str>
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
In my schema, the territory field is configured as follows:
<field name="territory" type="string" indexed="true" stored="true" multiValued="true"/>
The territory field is multivalued, containing territories (['US', 'CA', etc.]).
I run the suggest query as follows:
http://localhost:8983/solr/test_suggester/suggest?suggest.dictionary=suggest_artist&suggest.q=m&suggest.cfq=US
and I get a response with no suggestions found.
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":0,
"params":{
"echoParams":"all",
"indent":"true",
"suggest.q":"m",
"suggest.count":"10",
"suggest":"true",
"suggest.dictionary":"suggest_artist",
"wt":"json",
"suggest.cfq":"US"}},
"suggest":{"suggest_artist":{
"m":{
"numFound":0,
"suggestions":[]}}}}
Without the suggest.cfg=US I am getting a list of suggestions (I have checked that there are items that should be returned by searching using fq=territory:US). I have tried using a single-valued field, using both boolean (eg: us_terr:true) and string field-types (us_terr:"t"), and the results have been the same. The suggester is in its own separate collection on SolrCloud, with only one shard.
The issue was a that dictionaryImpl was misspelt.
<str name="dictionaryimpl">DocumentDictionaryFactory</str>
should be:
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
This meant that it was using the default dictionary implementation, HighFrequencyDictionaryFactory, which doesn't support context filtering.
This is a follow up question of this question. I have a list of cities onto which I want to implement spell-checker. I have the priorities/weights of these cities with me. I tried implementing a solrsuggester with a FileDictionaryFactory as a base with the following format:
<city-name> <TAB> <weight> <TAB> <other parameters like citycode,country>
I am passing other attributes like citycode, country etc as pipe separated payload string.
Here's my solrconfig
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">FileDictionaryFactory</str>
<str name="field">name</str>
<str name="weightField">searchscore</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">false</str>
<str name="sourceLocation">spellings.txt</str>
<str name="storeDir">autosuggest_dict</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">mySuggester</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
and my schema
<field name="name" type="string" indexed="true" stored="true" multiValued="false" />
<field name="countrycode" type="string" indexed="true" stored="true" multiValued="false" />
<field name="latlng" type="location" indexed="true" stored="true" multiValued="false" />
<field name="searchfield" type="text_ngram" indexed="true" stored="false" multiValued="true" omitNorms="true" omitTermFreqAndPositions="true" />
<uniqueKey>id</uniqueKey>
<defaultSearchField>searchfield</defaultSearchField>
<solrQueryParser defaultOperator="OR"/>
<copyField source="name" dest="searchfield"/>
Now the problem I am facing is I am getting 0 results for each and every search query. Even though I can see the storeDir getting created and it has a bin file with data looks like my payload data.
This is the url format I am using
/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=cologne
So, I have the following questions:
What does the creation of storeDir signify ? Is it indexed successfully
If yes, then what's wrong with my query ? If no, Am I missing something here(indexPath ???).
Is it the right way to supply search parameters on payload field ? If no, is there any other way ?
There is slight change in your solrconfig.xml. you need to remove buildOnStartup from suggester configuration or set it true.
[solrconfig.xml]
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">FileDictionaryFactory</str>
<str name="field">name</str>
<str name="weightField">searchscore</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">true</str>
<str name="sourceLocation">spellings.txt</str>
<str name="storeDir">autosuggest_dict</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">mySuggester</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
There is a problem in File Based Suggester that it will not build their suggestions through query by setting suggest=true. You need to build the File Based Suggestion on startup.
I was using searchfield as defaultSearchField in schema, but had configured name as suggest field. The moment I changed field to searchfield and suggestAnalyzerFieldType to text_ngram, it started working.
Here is the working solrconfig:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">suggestions</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">FileDictionaryFactory</str>
<str name="field">searchfield</str>
<str name="weightField">searchscore</str>
<str name="suggestAnalyzerFieldType">text_ngram</str>
<str name="buildOnStartup">false</str>
<str name="buildOnCommit">false</str>
<str name="sourceLocation">spellings.txt</str>
<str name="storeDir">autosuggest_dict</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">suggestions</str>
<str name="suggest.dictionary">results</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
We are trying to use solr on our website as a search engine , but we have a problem , we can not sort the suggestions by the number of sales.
I tried the components Facet, Terms, FreeTextLookupFactory and the spellcheck component, but in none of the above components are able to get the results that I want.
The most important thing that I would understand its if we can sort the suggestion for a weight chosen by us.
The version of solr we are using is the 5.0.
schema.xml:
<field name="name_complete" type="text_shingle" indexed="true" stored="true" required="false" multiValued="false" omitTermFreqAndPositions="true"/>
<fieldType name="text_shingle" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ShingleFilterFactory" maxShingleSize="4" outputUnigrams="true"/>
</analyzer>
</fieldType>
solrconfig.xml
FreeTextLookupFactory:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">suggest_product_free</str>
<str name="lookupImpl">FreeTextLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">name_complete</str>
<str name="indexPath">suggest_product_free</str>
<str name="weightField">n_sales</str>
<str name="buildOnCommit">true</str>
<str name="suggestFreeTextAnalyzerFieldType">text_shingle</str>
<int name="ngrams">3</int>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<str name="wt">json</str>
<str name="indent">true</str>
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
spellcheck:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="queryAnalyzerFieldType">text_shingle</str>
<str name="name">autocomplete</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory</str>
<str name="field">name_complete</str>
<str name="buildOnCommit">true</str>
<float name="threshold">0.005</float>
<str name="spellcheckIndexDir">./suggester_autocomplete</str>
</lst>
</searchComponent>
<requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<!-- Solr will use suggestions from both the 'default' spellchecker
and from the 'wordbreak' spellchecker and combine them.
collations (re-written queries) can include a combination of
corrections from both spellcheckers -->
<str name="spellcheck">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.dictionary">autocomplete</str>
<!--str name="spellcheck.dictionary">wordbreak</str>
<str name="spellcheck.extendedResults">true</str>
<str name="spellcheck.alternativeTermCount">5</str>
<str name="spellcheck.maxResultsForSuggest">5</str>
<str name="spellcheck.collate">true</str>
<str name="spellcheck.collateExtendedResults">true</str>
<str name="spellcheck.maxCollationTries">10</str>
<str name="spellcheck.maxCollations">5</str-->
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
any ideas how I can make to order the suggestion for the weight?
Thanks to all and have a nice day.
example:
assume that I have these saved values on solr:
1: Reflex canon eos 7d (weight: 1)
2: Reflex canon eos 6d (weight: 2)
3: Reflex canon eos 70d (weight: 3)
and requested the string 'can'.
the result that I want:
canon eos
canon eos 70d
canon eos 6d
canon eos 7d
in practice would that it were a autocomplete that I can order for the weight.
I actually use Solr 4.8.1 and I set up spellcheck. After indexing, the request doesn't return any suggestion.
After the advice of #n0tting, I modified a little my files.
Here are steps:
1- solrconfig.xml
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">phraseText</str>
<lst name="spellchecker">
<str name="classname">solr.IndexBasedSpellChecker</str>
<str name="spellcheckIndexDir">./spellchecker</str>
<str name="name">default</str>
<str name="field">title_spellcheck</str>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
add some configurations in standard requestHandler:
<requestHandler name="standard" class="solr.StandardRequestHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
<!-- Optional, must match spell checker's name as defined above, defaults to "default" -->
<str name="spellcheck.dictionary">default</str>
<!-- omp = Only More Popular -->
<str name="spellcheck.onlyMorePopular">false</str>
<!-- exr = Extended Results -->
<str name="spellcheck.extendedResults">false</str>
<!-- The number of suggestions to return -->
<str name="spellcheck.count">1</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
2 schema.xml
Define a field for spell check:
<field name="title_spellcheck" type="phraseText" indexed="true" stored="false" multiValued="true" />
<copyField source="title" dest="title_spellcheck"/>
3 Request:
.../select?q=recommend&defType=edismax&qf=title&spellcheck=true&spellcheck.build=true&spellcheck.q=recommend&spellcheck.collate=true
I don't get any suggestion at result, neither <lst name="spellcheck">. can anybody give me an advice? Thanks a lot.
References:
https://cwiki.apache.org/confluence/display/solr/Spell+Checking
http://solr.pl/en/2011/05/23/%E2%80%9Ccar-sale-application%E2%80%9D-%E2%80%93-spellcheckcomponent-%E2%80%93-did-you-really-mean-that-part-5/
I have configured my solrconfig.xml to use Suggester component as suggested in http://wiki.apache.org/solr/Suggester.
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup</str>
<str name="field">name</str>
<!-- the indexed field to derive suggestions from -->
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<!--
<str name="sourceLocation">american-english</str>
-->
</lst>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
However, when i browse from the browser url i am not getting any suggestions.
http://localhost:8080/solr/collection1/suggest?q=ava&fq=type%3ACategory&wt=xml
<response>
<lst name="responseHeader">
<int name="status">
0
</int>
<int name="QTime">
0
</int>
</lst>
<lst name="spellcheck">
<lst name="suggestions"/>
</lst>
</response>
Even if i remove the fq filter, I am not getting the suggestions.
Please help.
Thanks
I am not sure, can you try adding spellcheck=on
http://localhost:8080/solr/collection1/suggest?q=ava&spellcheck=on&fq=type%3ACategory&wt=xml