Which lookup should i use for implementing context Filtering in solr Suggestor component?
I am trying to use contextFiler in FuzzyLookupFactory and AnalyzingLookupFactory but it is throwing me the below error:
Caused by:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://localhost:8983/solr/categories: this suggester doesn't support contexts.
My suggestor:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">displaySuggester</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">autoComplete</str>
<str name="contextField">productCategory</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">false</str>
</lst>
</searchComponent>
The reference manual has the required settings:
Context filtering lets you filter suggestions by a separate context field, such as category, department or any other token. The AnalyzingInfixLookupFactory and BlendedInfixLookupFactory currently support this feature, when backed by DocumentDictionaryFactory.
It's important to note that this is the Analyzing**Infix**LookupFactory, and not the AnalyzingLookupFactory.
You're already using the DocumentDictionaryFactory, so switch to either the AnalyzingInfixLookupFactory or the BlendedInfixLookupFactory to make it work.
Related
In the docs, it says that context filtering only comes into effect "when using AnalyzingInfixLookupFactory or BlendedInfixLookupFactory, when backed by a DocumentDictionaryFactory".
However, I have found that the context filtering is applied when a FileDictionaryFactory is used. This doesn't work, as there are no documents for the context filter to be applied to.
http://localhost:8983/solr/mycore/suggest?qt=suggest&suggest.dictionary=location&q=russia
> Returns ["Russia"]
http://localhost:8983/solr/mycore/suggest?qt=suggest&suggest.dictionary=location&q=russia&cfq=a
> Returns []
This is my suggester config:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">location</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="dictionaryImpl">FileDictionaryFactory</str>
<str name="sourceLocation">tdwg.txt</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<str name="highlight">false</str>
</lst>
<lst name="suggester">
<str name="name">common-name</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">region.vernacular_names_t</str>
<str name="indexPath">common_name_suggest</str>
<str name="contextField">searchable.context_ss</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<str name="highlight">false</str>
</lst>
</searchComponent>
As you can see, for one of the suggesters I do want context filtering (and it is working correctly). So I can't simply remove the suggest.cfq parameter from my request.
Is there anything I can change about my configuration so that the context filter is not applied to my FileDictionaryFactory suggester?
I already faced this issue in the past, it appears that when suggest.cfq is present in the request, context filtering will be applied for every (enabled) lookup implementations that supports it (AnalyzingInfix and BlendedInfix).
It seems there is no other solution than switching to another lookup impl. than these 2 for the dictionary which you don't want to apply context filtering.
For example you can try to use the FuzzyLookupFactory for the "location" suggester and the context filter won't be applied.
NB: this is a workaround, as it's not possible to get infix matches with FuzzyLookup or AnalyzingLookup implementations (only the whole prefix from the input token(s) is taken into account).
If you really need infix matches for both suggesters, it's likely you will have to make 2 parallel requests before merging the suggestions :/.
Is it possible to have dynamic field or pass field for suggestions at runtime (in query for example) for SuggestComponent?
Depending on user's language I would like to suggest him different things. I have dynamic field name_* that has concrete fields: name_pl, name_de and name_en (can be more, I want to have flexibility here) and I would like to search for suggestions depending on language: for pl I want to get suggestions in name_pl, for en in name_en and so on.
So far I have standard Suggester with field specified:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">HighFrequencyDictionaryFactory</str>
<str name="">name_pl</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">false</str>
</lst>
</searchComponent>
<requestHandler name="/suggest" class="solr.SearchHandler"
startup="lazy" >
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
But actually I need either to use name_* or preferably at runtime to pass the field name for example: http://localhost:8983/solr/services/suggest?suggest=true&suggest.build=true&suggest.dictionary=mySuggester&suggest.q=name&suggest.field=name_pl
How would you implement such mechanism?
It is not the answer you may expect but I started a comment and I ended up with this.
By using a dynamic field here you would have to rebuild the suggester at each query, I suggest ;) you require a specific suggestComponent' dictionary on query.
The value for field should remain static because it is parsed once to build a dictionary index from that field. Or you would have to delete/rebuild that index each time a suggest query requires a dictionary other than the one previously built.
Instead you should replicate the suggester definition for each language you may have so that Solr can build one dictionary index per field/language (just name the suggesters according to the target field language) :
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">suggest_nl</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">HighFrequencyDictionaryFactory</str>
<str name="field">name_pl</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">false</str>
</lst>
<lst name="suggester">
<str name="name">suggest_en</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">HighFrequencyDictionaryFactory</str>
<str name="field">name_en</str>
<str name="suggestAnalyzerFieldType">string</str>
<str name="buildOnStartup">false</str>
</lst>
<!-- etc. -->
</searchComponent>
Now you can query the target dictionary dynamically :
.../suggest?suggest=true&suggest.q=name&suggest.dictionary=suggest_nl
There is an easy way to do this, not sure if you are aware of it:
you create one dictionary per language: suggester_pl, suggester_en...each using the right field. They are all defined inside a single SuggestComponent
when calling, you select which one to hit with &suggest.dictionary=suggester_en
check the docs here
I need to handle suggestion using solr query. Suggestion is working fine. But the problem is that suggestions are conditional i.e. suggestions are based on geo locations. I want the following equivalent query:
q=<SEARCH TERM>&fq=country:"<search country>"
I have tried some approaches but they are not working. What is the way to handle such scenarios?
Edit 1:
Suggester Config:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">autofill</str>
<str name="contextField">allowed_location</str>
<str name="suggestAnalyzerFieldType">text_autofill</str>
<str name="buildOnStartup">false</str>
<str name="storeDir">path on file system</str>
</lst>
</searchComponent>
You will have to configure the suggester component in your solrconfig.xml qnd your suggestions queries need to be of the below form(shortened)
suggest.q=<search_term>&suggest.cfq=<search_country>
If you need the filtering query to work on country field, it must be configured as a contextField in your solrconfig.xml
For better understanding please refer to the reference guide link
I created a plugin for Solr (version 6.3) that adds a permission layer to filter the retrieved documents using a database query (e.g: the user with ID 2 hasn't permissions to see the document with the ID 1); As the logic that defines if an user has permissions needs fields that aren't indexed in the Solr, i need to check in the database.
To achieve that i created a Query Parser (called DocumentsByUserParser that extends the class QParserPlugin) defined in the solrconfig.xml with the following:
<queryParser name="filterDocument" class="mypackage.solr.plugin.DocumentsByUserParser" />
To call this plugin, i only have to set the fq parameter with the {!filterDocument userId='<user_id>'} along with the other query parameters. Note that the code above works well with the Search Component with an edismax type.
My question is: can i create a new similar plugin, as described above, that works with the Suggest Component? Because when I index a document, the user may have (or not) permissions to see that document, so the suggester shouldn't show suggestions that the user hasn't permissions to see.
I defined my Suggest Component along with the Request Handler with the following:
<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="indexPath">suggester_infix_dir</str>
<str name="highlight">true</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">AUTO_COMPLETE_FIELD</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<str name="buildOnStartup">false</str>
<str name="buildOnCommit">false</str>
</lst>
</searchComponent>
P.S - The context filter query described in https://lucene.apache.org/solr/guide/6_6/suggester.html only works with a single indexed field, so this will not work with my use case.
Im having trouble finding a way how to have 2 differently structured fields in one suggest component. (https://cwiki.apache.org/confluence/display/solr/Suggester)
The goal is to have an autocomplete module with these fields.
A field where StandardTokenizer is used
example output: This is a title
A field where a Custom tokenizer is used (Basically a regex to get a base domain of a full URL)
example output: thisisatitle.com
Therefore the requesthandler containing the the suggestcomponent is able to show both strings in the results array: thisisatitle.com and This is a title
Things ive tried are:
Multiple suggestcomponents
Ive googled and the only solution ive currently found is using shards as they allow for different schemas to be combined. To my mind that is rather ineffective as running 2 servers would be a waste of resources and also maintainability would suffer.
Any suggestions/workarounds are welcome.
To use multiple suggestion dictionaries (that can have different analyzers applied), you can use the "multiple dictionaries" configuration as shown in the documentation:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">cat</str>
<str name="weightField">price</str>
<str name="suggestAnalyzerFieldType">string</str>
</lst>
<lst name="suggester">
<str name="name">altSuggester</str>
<str name="dictionaryImpl">DocumentExpressionDictionaryFactory</str>
<str name="lookupImpl">FuzzyLookupFactory</str>
<str name="field">product_name</str>
<str name="weightExpression">((price * 2) + ln(popularity))</str>
<str name="sortField">weight</str>
<str name="sortField">price</str>
<str name="storeDir">suggest_fuzzy_doc_expr_dict</str>
<str name="suggestAnalyzerFieldType">text_en</str>
</lst>
</searchComponent>