I'm using this plugin to allow spatial queries in Solr. I have followed the steps included in the documentation and I've got the spatial queries working fine.
Now I want to retrieve the computed distance. I added these lines in the solrconfig.xml file:
<searchComponent name="geodistance" class="nl.jteam.search.solrext.spatial.GeoDistanceComponent">
<defaults>
<str name="distanceField">geo_distance</str>
</defaults>
</searchComponent>
And I have added the "geodistance" component to the standard request handler:
<requestHandler name="standard" class="solr.SearchHandler" default="true">
<lst name="defaults">
<str name="echoParams">explicit</str>
</lst>
<arr name="components">
<str>query</str>
<str>geodistance</str>
</arr>
</requestHandler>
Then, when I run a query such as "q={!spatial lat=41.641184 long=-0.894032 radius=2 calc=arc unit=km} cafeteria" it works, but only the first time. When I run the same query again I get this error:
GRAVE: java.lang.NullPointerException
at nl.jteam.search.solrext.spatial.DistanceFieldValueSource.getValues(DistanceFieldValueSource.java:57)
at nl.jteam.search.solrext.spatial.GeoDistanceComponent.process(GeoDistanceComponent.java:60)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
I have no idea where is the error because the first time the query works and I get the computed distance in the "geo_distance" field. But when repeating the query, I get a NullPointerException.
This problem is fixed in version 1.0-RC5. This version has been released some days ago.
Related
I am using DSE 5.0.1 version. Earlier we used facet query to show search suggestions. For performance reasons , looking for other alternatives to get suggestions and found solr search suggester component. But I couldn't find examples where suggester component is used from a CQL query. Its possible right?Can anyone help me on this.
Thanks in advance.
Yes, it's possible and relatively easy - you just need to understand how to map XML that you want to put into generated solrconfig.xml into JSON that is used for configuration.
For example, we want to configure suggestor to suggest on the data from field title, and use additional weights from the rating field. As per Solr documentation the XML piece should look following way:
<searchComponent class="solr.SuggestComponent" name="suggest">
<lst name="suggester">
<str name="name">titleSuggester</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="suggestAnalyzerFieldType">TextField</str>
<str name="field">title</str>
<str name="weightField">rating</str>
<str name="buildOnCommit">false</str>
<str name="exactMatchFirst">true</str>
<str name="contextField">country</str>
</lst>
</searchComponent>
<requestHandler class="solr.SearchHandler" name="/suggest">
<arr name="components">
<str>suggest</str>
</arr>
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
</lst>
</requestHandler>
In CQL, it will be converted
ALTER SEARCH INDEX CONFIG ON table ADD
searchComponent[#name='suggest',#class='solr.SuggestComponent']
WITH $$ {"suggester":[{"name":"titleSuggester"},
{"lookupImpl":"AnalyzingInfixLookupFactory"},
{"dictionaryImpl":"DocumentDictionaryFactory"},
{"suggestAnalyzerFieldType":"TextField"},
{"field":"title"}, {"weightField":"rating"},
{"buildOnCommit":"false"}, {"exactMatchFirst":"true"},
{"contextField":"country"}]} $$;
ALTER SEARCH INDEX CONFIG ON table ADD
requestHandler[#name='/suggest',#class='solr.SearchHandler']
WITH $$ {"defaults":[{"suggest":"true"},
{"suggest.count":"10"}],"components":["suggest"]} $$;
After that you need not to forget to execute:
RELOAD SEARCH INDEX ON table;
And your suggestor will work. In my example, the index for suggestor should be build explicitly because inventory doesn't change very often. This is done via HTTP call like this:
curl 'http://localhost:8983/solr/keyspace.table/suggest?suggest=true&suggest.dictionary=titleSuggester&suggest.q=Wat&suggest.cfq=US&wt=json&suggest.build=true&suggest.reload=true'
But you can control this by setting buildOnCommit to true. Or you can configure it to build suggestor index on start, etc. - see Solr's documentation.
Full example is here - this is an example of the e-commerce application.
I've setup a django-oscar project and enabled solr 4.7.2 on it as per documentation.
Solr seems to be working fine. Testing the suggestions for 'exxample' (localhost:8983/solr/collection1/spell?spellcheck.q=exxample&spellcheck=true>) I get:
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">10</int>
</lst>
<result name="response" numFound="0" start="0"/>
<lst name="spellcheck">
<lst name="suggestions">
<lst name="exxampl">
<int name="numFound">1</int>
<int name="startOffset">0</int>
<int name="endOffset">8</int>
<int name="origFreq">0</int>
<arr name="suggestion">
<lst>
<str name="word">exampl</str>
<int name="freq">2</int>
</lst>
</arr>
</lst>
<bool name="correctlySpelled">false</bool>
<lst name="collation">
<str name="collationQuery">exampl</str>
<int name="hits">2</int>
<lst name="misspellingsAndCorrections">
<str name="exxampl">exampl</str>
</lst>
</lst>
</lst>
</lst>
</response>
I've also enabled OSCAR_SEARCH_FACETS to make sure that Solr has been correctly registered by Django-Oscar, and it seems to be working fine.
HOWEVER, when I do a test search for a simple misspelling in django-oscar, I get 0 returned search results and no suggestions. I'm not sure what to do next.
Help would be greatly appreciated!
I've managed to fix this problem. I'll write my complete solution to setting up Solr with spelling suggestions on Django-Oscar since setup procedures require adjustments from that described in the official documentation. This is also my first time working with Solr (or any search engine), so don't expect some expert guidance, just a guide on how to get Solr up and running on Oscar.
I am using Oscar 1.5 with Solr 4.7.2 (solutions also works for 4.10.4 ... not sure about other versions). Do everything as per documentations - note that there is a slight difference in instructions for versions of Oscar that are < 1.5.
Once you have Solr installed and running you can test out an inquiry on the Solr server # localhost:8983/solr/collection1/spell?spellcheck.q=[your search inquiry goes here; no brackets]&spellcheck=true>. Needs to be a word from your database - either in product description or product title.
You will get an error result saying that Analyzer needs to be of same type. Fix this by editing the solrconfig.xml file located at ./solr-4.7.2/example/solr/collection1/conf/solrconfig.xml. Search for <str name="field">, and change each non-commented instance to <str name="field">text</str> - you can also change each instance to <str name="field">title</str>, but this restricts to words found in titles only. Restart the Solr server. These changes will do away with the Analyzer error and your Solr server will now start showing results, however they won't yet be fed into your Oscar site.
To fix this you need to make another adjustment to the same solrconfig.xml file. Search for <requestHandler name="/select" class="solr.SearchHandler">, and at the bottom of this request handler include the following code:
<arr name="last-components">
<str>spellcheck</str>
</arr>
Restart the server. Now you have spelling suggestions in your Oscar site. Hope others have found this helpful. Like I said - this is the first time I'm using Solr. If someone has anything to add, or extend Solr functionality on Oscar it would be great.
I have problem with very slow filters in Solr (version 4.9.1), there is ~50k documents. For first query which use specific category_id filter value, query takes ~15 seconds, second time is much more faster (it takes miliseconds). But i want to have fast filters always :) So after googling it i read that I must have filterCache and cache Autowarming
Sooo what I've done:
filterCache:
<filterCache
class="solr.FastLRUCache"
size="16384"
initialSize="4096"
autowarmCount="4096" />
firstSearcher:
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst>
<str name="q">*</str>
<str name="fq">category_id:1043</str>
</lst>
</arr>
</listener>
<useColdSearcher>true</useColdSearcher>
<useFilterForSortedQuery>true</useFilterForSortedQuery>
<maxWarmingSearchers>2</maxWarmingSearchers>
It doesn't work ;/ no idea why... For first entry on this category it takes 15s, than its fast. But I always must have fast response, for categories and for other filters.
I make an experiment, everything works better if I use mainquery instead of filters, but filters should be as fast as mainquery (i read it somewhere).
Summary:
What i'm doing wrong that autowarming dont work?
How make autowarming for each filter/each filter value?
What I'm trying to do:
Ok so, I have shop with ~50 000 products and ~1000 categories and a lot of other filters (type, price etc), my catalog is based on SOLR (filtering), now if I use filters first entry to category takes 15seconds, it must be fast every single time....
My example query:
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
<lst name="params">
<str name="debugQuery">true</str>
<str name="website_id:1"/>
<str name="stats.field">PLN_0_price_decimal</str>
<str name="product_status:1"/>
<str name="q">**</str>
<str name="store_id:1"/>
<str name="fq">category_id:10561</str>
</lst>
</lst>
So, solution was simple, I have to use * instead of ** in my query.
Part of debug section from response with *:
<str name="parsedquery">MatchAllDocsQuery(*:*)</str>
<str name="parsedquery_toString">*:*</str>
Same part of debug section from response with **:
<str name="parsedquery">textSearch:**</str>
<str name="parsedquery_toString">textSearch:**</str>
The first time you use a filter, every document needs to be looked at, even if the main query will match only a couple. You could disable caching for such filter or switch to a post-filter (by assigning filter cost). The fuller explanation is here.
I tried using Solr's Suggester component but it gives exception Unknown Search Component: spellcheck. I am using solr version 3.6. I made changes in solrconfig.xml so it looks like this
<searchComponent class="solr.SpellCheckComponent" name="suggester">
<lst name="spellchecker">
<str name="name">suggester</str>
<str name="field">name</str>
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookup</str>
<float name="threshold">0.005</float>
<str name="buildOnCommit">true</str>
<str name="spellcheckIndexDir">spellchecker</str>
</lst>
</searchComponent>
<requestHandler class="org.apache.solr.handler.component.SearchHandler" name="/suggester">
<lst name="defaults">
<str name="spellcheck">true</str>
<str name="spellcheck.dictionary">suggester</str>
<str name="spellcheck.count">10</str>
</lst>
<arr name="components">
<str>suggester</str>
</arr>
</requestHandler>
Now i saved it and started solr but it gives a 500 error. There is a long stack trace which goes like this
HTTP Status 500 - Severe errors in solr configuration. Check your log files for more detailed information on what may be wrong. If you want solr to continue after configuration errors, change: <abortOnConfigurationError>false</abortOnConfigurationError> in solr.xml ------------------------------------------------------------- org.apache.solr.common.SolrException: No cores were created, please check the logs for errors at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:172) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96) at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:295) at
---lot of lines---
and some more
Caused by: org.apache.solr.common.SolrException: Unknown Search Component: spellcheck at org.apache.solr.core.SolrCore.getSearchComponent(SolrCore.java:893) at org.apache.solr.handler.component.SearchHandler.inform(SearchHandler.java:118) at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:527) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:594) ... 32 more
According to the stack trace Solr is not able to find the spellcheck component but according to the Solr wiki this component comes with Solr version > 1.4.
You seem to have named your search component suggester, and your requestHandler refers to it correctly, so that's probably not where the issue is. Can you check in the rest of your solrconfig.xml whether some other requestHandler refers to a component named spellcheck in its last-components section or elsewhere? That would throw an error since it can no longer find a component with that name (it's been renamed to suggester).
The thing with solrconfig is that it comes with a lot of preconfigured search handlers, and if you change the default name of one item it could cause issues with handlers that refer to it.
I have indexed my database tables into solr using DataImportHandler. Now when I query the server it shows me that the number of results found 665. But when i try to assign it to beans like List itemList = rsp.getBeans(Item.class), it is giving me only 10 results.
Can some one help me out on this.
Thanks in Advance.
When you don't define the amount of rows (documents) to fetch, Solr defaults to fetching 10 documents, as explained in the docs.
By default Solr returns only 10 Documents. If you want to fetch all documents, you will need to update solrConfig.xml file of Core (path : /solr/server/solr/core_name/conf/solrConfig.xml) :
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10000000</int> <!--you can update it to some large value that is higher than the possible number of rows that are expected.-->
</lst>
</requestHandler>
You might have to edit your solrconfig.xml.
There change the "/select" Request Handler like this.
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">1000</int> <!-- Change this as you want -->
<str name="df">text</str>
</lst>
</requestHandler>