I am using SOLR Cloud 6.0 with below setup,
3 physical VM, 5 collections with 6 shrad each with replication factor 2
I am running zookeeper on each VM
I am receiving below exception very frequently in solr logs,
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: RunUpdateProcessor has received an AddUpdateCommand containing a document that appears to still contain Atomic document update operations, most likely because DistributedUpdateProcessorFactory was explicitly disabled from this updateRequestProcessorChain
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:63)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:335)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
Can you please help me with the root cause ? Below is the snapshot of solrconfig,
<updateRequestProcessorChain name="add-unknown-fields-to-the-schema">
<!-- UUIDUpdateProcessorFactory will generate an id if none is present in the incoming document -->
<processor class="solr.UUIDUpdateProcessorFactory" />
<processor class="solr.LogUpdateProcessorFactory"/>
<processor class="solr.DistributedUpdateProcessorFactory"/>
<processor class="solr.RemoveBlankFieldUpdateProcessorFactory"/>
<processor class="solr.FieldNameMutatingUpdateProcessorFactory">
<str name="pattern">[^\w-\.]</str>
<str name="replacement">_</str>
</processor>
<processor class="solr.ParseBooleanFieldUpdateProcessorFactory"/>
<processor class="solr.ParseLongFieldUpdateProcessorFactory"/>
<processor class="solr.ParseDoubleFieldUpdateProcessorFactory"/>
<processor class="solr.ParseDateFieldUpdateProcessorFactory">
<arr name="format">
<str>yyyy-MM-dd'T'HH:mm:ss.SSSZ</str>
<str>yyyy-MM-dd'T'HH:mm:ss,SSSZ</str>
<str>yyyy-MM-dd'T'HH:mm:ss.SSS</str>
<str>yyyy-MM-dd'T'HH:mm:ss,SSS</str>
<str>yyyy-MM-dd'T'HH:mm:ssZ</str>
<str>yyyy-MM-dd'T'HH:mm:ss</str>
<str>yyyy-MM-dd'T'HH:mmZ</str>
<str>yyyy-MM-dd'T'HH:mm</str>
<str>yyyy-MM-dd HH:mm:ss.SSSZ</str>
<str>yyyy-MM-dd HH:mm:ss,SSSZ</str>
<str>yyyy-MM-dd HH:mm:ss.SSS</str>
<str>yyyy-MM-dd HH:mm:ss,SSS</str>
<str>yyyy-MM-dd HH:mm:ssZ</str>
<str>yyyy-MM-dd HH:mm:ss</str>
<str>yyyy-MM-dd HH:mmZ</str>
<str>yyyy-MM-dd HH:mm</str>
<str>yyyy-MM-dd</str>
</arr>
</processor>
<processor class="solr.AddSchemaFieldsUpdateProcessorFactory">
<str name="defaultFieldType">strings</str>
<lst name="typeMapping">
<str name="valueClass">java.lang.Boolean</str>
<str name="fieldType">booleans</str>
</lst>
<lst name="typeMapping">
<str name="valueClass">java.util.Date</str>
<str name="fieldType">tdates</str>
</lst>
<lst name="typeMapping">
<str name="valueClass">java.lang.Long</str>
<str name="valueClass">java.lang.Integer</str>
<str name="fieldType">tlongs</str>
</lst>
<lst name="typeMapping">
<str name="valueClass">java.lang.Number</str>
<str name="fieldType">tdoubles</str>
</lst>
</processor>
<processor class="solr.RunUpdateProcessorFactory"/>
</updateRequestProcessorChain>
Regards,
Pratik Thaker
Related
I need to count how many fields has a multivalue field to sort results out.
solrconfig.xml
<updateRequestProcessorChain name="add-numbers-count">
<processor class="solr.CloneFieldUpdateProcessorFactory">
<str name="source">cat_ids</str>
<str name="dest">cat_ids_count</str>
</processor>
<processor class="solr.CountFieldValuesUpdateProcessorFactory">
<str name="fieldName">cat_ids_count</str>
</processor>
<processor class="solr.DefaultValueUpdateProcessorFactory">
<str name="fieldName">cat_ids_count</str>
<int name="value">0</int>
</processor>
<!-- <processor class="solr.LogUpdateProcessorFactory" />
<processor class="solr.RunUpdateProcessorFactory" /> -->
</updateRequestProcessorChain>
<initParams path="/update/**,/query,/select,/spell">
<lst name="defaults">
<str name="df">_text_</str>
<str name="update.chain">add-numbers-count</str>
</lst>
</initParams>
manageschema.xml
<field name="cat_ids" type="plongs"/>
<field name="cat_ids_count" type="pint"/>
Note that RunUpdateProcessorFactory and LogUpdateProcessorFactory are commented out.
If I use them update fails with a non sense error:
ERROR: [doc=44996] Error adding field 'data_readout'='2025-06-01' msg=Invalid Date String:'2025-06-01'int(44996)
Solr is not creating this field cat_ids_count I guess because there is no RunUpdateProcessorFactory.
Do I have to delete and recreate collection? Or is there any error I can't see?
I am having a lock issue when trying to initialize a Solr core:
SolrCore Initialization Failures:
MySolrCore: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: org.apache.lucene.store.LockObtainFailedException: Lock held by this virtual machine: C:\solr-6.0.1\server\solr\MySolrCore\data\analyzingInfixSuggesterIndexDir\write.lock
I am trying to implement multiple spell checkers on this core and I get this lock error.
Some interesting notes from trying different approaches:
I tried extending the writeLockTimeout to 30 seconds and that did nothing.
None of the spellcheckIndexDir folders are getting created. I added them in hopes of fixing the lock issue. I even tried creating them manually, and no files were added to them.
If I comment out one of the components and handlers, the other one works.
If I comment out the spellecheckers that use the AnalyzingInfixLookupFactory then both of the spellcheckers that use the WFSTLookupFactory and both of the handlers work. However, the spellcheckIndexDir is not created.
How do I fix the lock issue? I think that if I can get all of the spellcheckIndexDir specified to work that the indexes will be located in different folders and there won't be a lock. Thanks for any help!
Here's my Solr config search components and handlers for the two fields.
<searchComponent name="NameSuggest" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="name">NameFST</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory</str>
<str name="field">NameFST</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./NameFSTSuggestIndexDir</str>
</lst>
<lst name="spellchecker">
<str name="name">NameAI</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
<str name="field">NameAI</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./NameAISuggestIndexDir</str>
</lst>
<str name="queryAnalyzerFieldType">suggest_phrase</str>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/NameSuggest" startup="lazy">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">NameFST</str>
<str name="spellcheck.dictionary">NameAI</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.collate">false</str>
<str name="spellcheck.extendedResults">true</str>
</lst>
<arr name="components">
<str>NameSuggest</str>
</arr>
</requestHandler>
<searchComponent name="DescriptionSuggest" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="name">DescriptionFST</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.WFSTLookupFactory</str>
<str name="field">DescriptionFST</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./DescriptionFSTSuggestIndexDir</str>
</lst>
<lst name="spellchecker">
<str name="name">DescriptionAI</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingInfixLookupFactory</str>
<str name="field">DescriptionAI</str>
<int name="minQueryLength">3</int>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
<str name="suggestAnalyzerFieldType">suggest_phrase</str>
<str name="spellcheckIndexDir">./DescriptionAISuggestIndexDir</str>
</lst>
<str name="queryAnalyzerFieldType">suggest_phrase</str>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/DescriptionSuggest" startup="lazy">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">DescriptionFST</str>
<str name="spellcheck.dictionary">DescriptionAI</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.collate">false</str>
<str name="spellcheck.extendedResults">true</str>
</lst>
<arr name="components">
<str>DescriptionSuggest</str>
</arr>
</requestHandler>
Instead of spellcheckIndexDir I needed to use indexPath. It's working now.
<!-- <str name="spellcheckIndexDir">./NameAISuggestIndexDir</str> -->
<str name="indexPath">./NameAISuggestIndexDir</str>
I am trying to implement the auto suggest of solr this is the changes that I made in solrconfig.xml file
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookupFactory</str>
<str name="field">displayName</str> <!-- the indexed field to derive suggestions from -->
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
</lst>
</searchComponent>
when I try to query with sample input as 'p'
http://localhost:8983/solr/food/suggest?q=p&wt=json&indent=true
it returns 5 words
"pizza", "potato", "pasta", "protein", "premium"
but in the displayName field I got words like paneer , palak etc which is not showing up why is it so?
Can you added the following to your configuration and run the below query. Don't forget to reload the solr core after putting these changes.
<str name="suggestAnalyzerFieldType">string</str>
<str name="storeDir">suggester_fuzzy_dir</str>
http://localhost:8983/solr/food/suggest?suggest=true&suggest.build=true&suggest.dictionary=suggest&wt=json&suggest.q=p&suggest.count=10
Using the following configuration to run the spellchecking on Solr queries:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textTitle</str>
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">text</str>
<str name="classname">solr.DirectSolrSpellChecker</str>
<str name="distanceMeasure">internal</str>
<float name="accuracy">0.7</float>
<int name="maxEdits">2</int>
<int name="minPrefix">1</int>
<int name="maxInspections">10</int>
<int name="minQueryLength">3</int>
<float name="maxQueryFrequency">0.10</float>
<float name="thresholdTokenFrequency">.00001</float>
</lst>
</searchComponent>
<requestHandler name="standard" class="solr.StandardRequestHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
I get the correct suggestions for bad spelled queries, however, is it possible to re-do the query again to return the results for the spellchecked terms instead of just returning the suggestions?
I have configured my solrconfig.xml to use Suggester component as suggested in http://wiki.apache.org/solr/Suggester.
<searchComponent class="solr.SpellCheckComponent" name="suggest">
<lst name="spellchecker">
<str name="name">suggest</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup</str>
<str name="field">name</str>
<!-- the indexed field to derive suggestions from -->
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<!--
<str name="sourceLocation">american-english</str>
-->
</lst>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggest">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggest</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
However, when i browse from the browser url i am not getting any suggestions.
http://localhost:8080/solr/collection1/suggest?q=ava&fq=type%3ACategory&wt=xml
<response>
<lst name="responseHeader">
<int name="status">
0
</int>
<int name="QTime">
0
</int>
</lst>
<lst name="spellcheck">
<lst name="suggestions"/>
</lst>
</response>
Even if i remove the fq filter, I am not getting the suggestions.
Please help.
Thanks
I am not sure, can you try adding spellcheck=on
http://localhost:8080/solr/collection1/suggest?q=ava&spellcheck=on&fq=type%3ACategory&wt=xml