Search suggestions in django-oscar using solr - solr

I've setup a django-oscar project and enabled solr 4.7.2 on it as per documentation.
Solr seems to be working fine. Testing the suggestions for 'exxample' (localhost:8983/solr/collection1/spell?spellcheck.q=exxample&spellcheck=true>) I get:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">10</int>
</lst>
<result name="response" numFound="0" start="0"/>
<lst name="spellcheck">
<lst name="suggestions">
<lst name="exxampl">
<int name="numFound">1</int>
<int name="startOffset">0</int>
<int name="endOffset">8</int>
<int name="origFreq">0</int>
<arr name="suggestion">
<lst>
<str name="word">exampl</str>
<int name="freq">2</int>
</lst>
</arr>
</lst>
<bool name="correctlySpelled">false</bool>
<lst name="collation">
<str name="collationQuery">exampl</str>
<int name="hits">2</int>
<lst name="misspellingsAndCorrections">
<str name="exxampl">exampl</str>
</lst>
</lst>
</lst>
</lst>
</response>
I've also enabled OSCAR_SEARCH_FACETS to make sure that Solr has been correctly registered by Django-Oscar, and it seems to be working fine.
HOWEVER, when I do a test search for a simple misspelling in django-oscar, I get 0 returned search results and no suggestions. I'm not sure what to do next.
Help would be greatly appreciated!

I've managed to fix this problem. I'll write my complete solution to setting up Solr with spelling suggestions on Django-Oscar since setup procedures require adjustments from that described in the official documentation. This is also my first time working with Solr (or any search engine), so don't expect some expert guidance, just a guide on how to get Solr up and running on Oscar.
I am using Oscar 1.5 with Solr 4.7.2 (solutions also works for 4.10.4 ... not sure about other versions). Do everything as per documentations - note that there is a slight difference in instructions for versions of Oscar that are < 1.5.
Once you have Solr installed and running you can test out an inquiry on the Solr server # localhost:8983/solr/collection1/spell?spellcheck.q=[your search inquiry goes here; no brackets]&spellcheck=true>. Needs to be a word from your database - either in product description or product title.
You will get an error result saying that Analyzer needs to be of same type. Fix this by editing the solrconfig.xml file located at ./solr-4.7.2/example/solr/collection1/conf/solrconfig.xml. Search for <str name="field">, and change each non-commented instance to <str name="field">text</str> - you can also change each instance to <str name="field">title</str>, but this restricts to words found in titles only. Restart the Solr server. These changes will do away with the Analyzer error and your Solr server will now start showing results, however they won't yet be fed into your Oscar site.
To fix this you need to make another adjustment to the same solrconfig.xml file. Search for <requestHandler name="/select" class="solr.SearchHandler">, and at the bottom of this request handler include the following code:
<arr name="last-components">
<str>spellcheck</str>
</arr>
Restart the server. Now you have spelling suggestions in your Oscar site. Hope others have found this helpful. Like I said - this is the first time I'm using Solr. If someone has anything to add, or extend Solr functionality on Oscar it would be great.

Related

SOLR why does its state this 'leaderUrl' must be specified without the /replication suffix

Why when I setup a Solr follower/slave do I get this message in the Solr logs
'leaderUrl' must be specified without the /replication suffix
This used to work fine in ersion 7.6 but now in version 8.11.1 it complains. What should the value be if not /replication?
This is the XML snippet in the solrconfig.xml that does the replications
<requestHandler name="/replication" class="solr.ReplicationHandler">
<lst name="slave">
<str name="leaderUrl">http://solr1:8983/solr/bookings/replication</str>
<str name="pollInterval">00:00:20</str>
</lst>
</requestHandler>
It seems to mean the leadUrl, I seemed to be transfixed on the name of the request handler!
So going from this which worked in 7.6
<str name="masterUrl">http://solr1:8983/solr/bookings/replication</str>
to this which works in 8.11.1
<str name="leaderUrl">http://solr1:8983/solr/bookings</str>

How to tune apache SOLR spellcheck for desired suggestion?

Enviroment: SAP Hybris 6.7.0.0, Apache Solr 7.7.2
I am using solr to power a indie eCommerce platform. In that context we have product data in the Solr dB. For example: productName_text, BrandName_string, etc.
I've created a spellcheck component with this current configuration below:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<lst name="spellchecker">
<str name="name">en</str>
<str name="classname">solr.DirectSolrSpellChecker</str>
<str name="field">spellcheck_en</str>
<str name="distanceMeasure">internal</str>
<float name="accuracy">0.7</float>
<int name="maxEdits">2</int>
<int name="minPrefix">0</int>
<int name="maxInspections">5</int>
<int name="minQueryLength">2</int>
</lst>
</searchComponent>
And turned on spellcheck on /select request handler
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">default</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.count">5</str>
<str name="spellcheck.collate">true</str>
and spellcheck is configured dynamically for the a single field. Suppose:
productName_text
which consists of product names from a typical electronic gadgets or it's cases. For example:
"Apple Watch Series 2 38mm Stainless Steel Case with Midnight Blue Modern Buckle Medium"
"A.O. Smith X4 RO Water Purifier (White)"
If we misspell "wath" for "watch" we get suggestion "water". Or spelling "suop maker" for "soup maker" we get "shop maker". How to tune spellchecker according to my data? Is there any other solution to implement for misbehaving queries.
Tried playing with all the spellcheck configuration from [1]: https://cwiki.apache.org/confluence/display/SOLR/SpellCheckComponent but couldn't find any solid solution yet.
Tried implementing WordBreakSolrSpellChecker, which doesn't seem to change any outcome
Played around with "spellcheck.collate" and other attributes, but it returns suggestion which has no search result.
I've observed, spellcheck is deeply affected by multivalued fields(?)
In general, How to go about the terms which should give wrong suggestion, or suggestions that are that must not come based on user preferences? Is it possible to handle two different spellcheck components, if "DirectSolrSpellChecker" does'nt give desired suggestion , I can switch to "FilebasedSpellChecker"? Can I maintain a .txt file to track all the terms which needs tuning, or the same in SAP hybris?

Autocomplete term suggestion as per popularity

I have implemented autocomplete term suggestion in my MVC application. Let me explain you how I have done this. I have created one table in DB and table columns is like:
Id SearchTerm CatID ResultCount Clicks Latency TermSearchTime
Now, whenever user search a term we store it in this table. Next time it same word match we display term suggestion. Moreover, we display term suggestion as term popularity. Which word is more searched is displayed first in suggestion.
But now I also want to provide term suggestion for misspell term. For example Samsung is already there in my table. If someone search for samsng in that case Samsung should be there in term suggestion.
As I do not know how to spell check in SQL server, I decided to do it using Solr.
How can I do it using Solr with my default behaviour which I have done with SQL Db? Moreover, please note Search result I fetch from the Solr. I have already index all products. Do I need to index Search Term as well?
Any help is appreciation. Thanks.
check this in your solrconfig.xml file to use spellcheck handler.
<requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="df">text</str>
<!-- Solr will use suggestions from both the 'default' spellchecker
and from the 'wordbreak' spellchecker and combine them.
collations (re-written queries) can include a combination of
corrections from both spellcheckers -->
<str name="spellcheck.dictionary">default</str>
<str name="spellcheck.dictionary">wordbreak</str>
<str name="spellcheck">on</str>
<str name="spellcheck.extendedResults">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.alternativeTermCount">5</str>
<str name="spellcheck.maxResultsForSuggest">5</str>
<str name="spellcheck.collate">true</str>
<str name="spellcheck.collateExtendedResults">true</str>
<str name="spellcheck.maxCollationTries">10</str>
<str name="spellcheck.maxCollations">5</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr></requestHandler>
if not present then copy paste in your file. restart solr. try /spell?q=ipad

SOLR stats facet return non-facet count total

I'm using SOLR to store music playback information
when i do a query (broken out for readability):
http://localhost:8983/solr/select/
?q=song_1:*
&stats=true
&stats.field=song_2
&stats.field=song_3
&rows=0
I'm trying to see, of the people that played song_1, how many also played the other songs.
question: How can I also return the totals for each stats.field regardless of my query song_1?
In other words, I already get the coun tof people that played song_1 and song_2, but how can I add how many played song_2, regardless of song_1.
<response>
<result name="response" numFound="5454" start="0"></result>
<lst name="stats">
<lst name="stats_fields">
<lst name="song_2">
<str name="min">4</str>
<str name="max">1500</str>
<long name="count">2000</long>
<long name="missing">3454</long>
<lst name="facets"/>
</lst>
<lst name="song_3">
<str name="min">10</str>
<str name="max">1500</str>
<long name="count">200</long>
<long name="missing">3454</long>
<lst name="facets"/>
</lst>
</response>
I don't know why you are using the stats query.
If your requirement is to see how many people have listened to song_1 and also you want to see how many people has listened to other songs..
Then you should used faceted search.
Check the following thing for more explanation:
http://technical-fundas.blogspot.in/2014/08/solr-implementing-facet-with-multiple.html
Hope this helps you in resolving this issue!!!

Unable to retrieve computed distance in Spatial Solr queries

I'm using this plugin to allow spatial queries in Solr. I have followed the steps included in the documentation and I've got the spatial queries working fine.
Now I want to retrieve the computed distance. I added these lines in the solrconfig.xml file:
<searchComponent name="geodistance" class="nl.jteam.search.solrext.spatial.GeoDistanceComponent">
<defaults>
<str name="distanceField">geo_distance</str>
</defaults>
</searchComponent>
And I have added the "geodistance" component to the standard request handler:
<requestHandler name="standard" class="solr.SearchHandler" default="true">
<lst name="defaults">
<str name="echoParams">explicit</str>
</lst>
<arr name="components">
<str>query</str>
<str>geodistance</str>
</arr>
</requestHandler>
Then, when I run a query such as "q={!spatial lat=41.641184 long=-0.894032 radius=2 calc=arc unit=km} cafeteria" it works, but only the first time. When I run the same query again I get this error:
GRAVE: java.lang.NullPointerException
at nl.jteam.search.solrext.spatial.DistanceFieldValueSource.getValues(DistanceFieldValueSource.java:57)
at nl.jteam.search.solrext.spatial.GeoDistanceComponent.process(GeoDistanceComponent.java:60)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
I have no idea where is the error because the first time the query works and I get the computed distance in the "geo_distance" field. But when repeating the query, I get a NullPointerException.
This problem is fixed in version 1.0-RC5. This version has been released some days ago.

Resources