We are using solr 4.3 for search funcationlity. We have configured 2 shard and 2 replicas.
We have total 132905 Solr documents in index.
Our search query is very long takes around 3 second (from Solr Admin Console) for below query.
id:(FOLD5002861 FOLD5002890 FOLD5219963 FOLD4105003 FOLD4105005 FOLD4105006 FOLD4105007 FOLD4105008 FOLD4105009 FOLD4105010 FOLD4105011 FOLD4105012 FOLD4105013 FOLD4105014 FOLD4105018 FOLD4105019 FOLD4105020 FOLD4105021 FOLD4105022 FOLD4105023 FOLD4105024 FOLD4105025 FOLD4105026 FOLD4105027 FOLD5220166 FOLD5220168 FOLD5220169 FOLD5220170 FOLD5220171 FOLD5220172 FOLD5220173 FOLD5220174 FOLD5220175 FOLD5220176 FOLD5220177 FOLD5220178 FOLD5220179 FOLD5220180 FOLD5220181 FOLD4100876 FOLD4100877 FOLD4100878 FOLD4100879 FOLD4100880 FOLD4100881 FOLD4655426 FOLD4655428 FOLD4655429 FOLD4655430 FOLD4655431 FOLD4655432 FOLD4655433 FOLD4655434 FOLD4655435 FOLD4655436 FOLD4655437 FOLD4655438 FOLD4655439 FOLD4655483 FOLD4655487 FOLD4655523 FOLD4655874 FOLD4655884 FOLD4655856 FOLD4655858 FOLD4655859 FOLD4655860 FOLD4655861 FOLD4655862 FOLD4655863 FOLD4655864 FOLD4655865 FOLD4655866 FOLD4655867 FOLD4655868 FOLD4655869 FOLD4655870 FOLD4655871 FOLD4655872 FOLD4655882 FOLD4655892 FOLD4649510 FOLD4649512 FOLD4649513 FOLD4649514 FOLD4649515...50000 times)
We want to trace where it is taking time. we tried debugQuery option in solr Admin console but not getting useful information.
Is there any way to improve the query? How can we track detail timing?
If you transform this query into a filter query it will give better performance most of the time because it will apply the filter on top of a query (that is a subset of your index and that can be cached) and not on the entire index.
The best would be if you had another piece of query that you could run with this and then apply this as a filter on top of that , but if you donøt have it also running a query for field:[* TO *] should still give better performances.
have a look at this question that explain the difference between filter query and normal query:
SOLR filter-query vs main-query
Take a look at some factors that affect Solr performance.
Try to optimize your index with:
curl http://<solr_host>:<port>/solr/<core_name>/update -F stream.body=' <optimize />'
Related
Trying to OR main query (q) and filter query (fq) using Solr 4.7.
We have a high frequency term in the index
'field1:value' - 5 million documents
'field2:value' - 500 documents which should be ranked higher
When searching by
q=field1:value OR field2:value
The query takes very long (more than 2 seconds)
When searching by
q=*:*fq=field1:value OR field2:value
The query runs rather fast, but I have no way to get the field2:value at the top of the list.
Currently post re-ranking is not an option.
I understand the speed of the filter query (fq) is that no scoring is involved. (This is not a frequent query, no caching is necessary).
Tried in the QueryParser plugin to wrap the TermQuery with a ConstantScoreQuery.
But it seems performs just the same as an ordinary TermQuery.
What I am looking for is a way to run a filter query as an OR, what means q=field2:value&or_fq=field2:value
Or instead create a 'real-non-scoring' TermQuery within the main query.
Could you please help me? Thanks.
Use *:* as the query, then apply a filter query you did - field1:value OR field2:value as fq. You can then use bq or boost with field2:value to score those that have hits in field2.
I am running Solr 4.10.3 and trying to resort top 10 documents come from Solr. How can i do that? I am thinking of sub-query but don't know how to do that, needed help.
Example:
Suppose on query of "car" Solr return 250 documents on the basis of high score of relevancy. Now from 250 documents take top 10 documents and resort them on the basis of custom field.
i can't do that:
select?q=car&sort=score desc, pr desc
Because it will do sorting on entire 250 documents. So is there any solution?
I think you mean query reranking ? This is what is used when you want to:
use a first, more lightweight query to get a result based on the score
get a fair top x number of matches, like 2000 for example
use a heavier query to resort only those again, to get the final score
That is what you need to use if you want to do it in two steps, like you state. Now, I am not sure you need to use query reranking for your use case, maybe just boosting by that field, or sorting on a function should be enough for you.
I want boost at query time all documents that have value user_id=2. Basically I want on the top of my results all the documents belonged to a specific user.
After looking at some Solr resources I ended up writing a query like, but it is not working properly.
/solr/public-main/select?q={!boost b=if(div(155623,user_id),2,1)}sometext&wt=json&indent=true&debugQuery=true
Any hints?
Thanks
You don't need to use the boost with a dynamic boost. Apply a boost query which will boost all the documents that match the query: bq=user_id:2^4. Adjust 4 to a suitable boost value depending on the rest of your boosts (if any in q or qf).
One option is to have a function query with fl=x,y,userexists:exists(query({!v='user_id:2'})) and then u can sort by userexists and then by score field.
I'm starting to think that what I want to do is not possible but thought I would give this a try.
I'm running Solr 3.5.
I currently have two types of search:
A basic spatial query which returns the calulated distance between two points in the score field.
Sample Query from my Solr logs:
?fl=*,score&sort=score+asc&start=0&q={!func}geodist()&sfield=coordinates&pt=59.2363514,18.092783&version=2
A dismax query which allows free text queries on a number of fields.
Sample Query from Solr log:
mm=1&d=100.0&sfield=coordinates&qf=field1^5.0+fields2^3.0&defType=edismax&version=2&fl=*,score&start=1&q=monkeyhopper&pt=59.2363514,18.0927830000&fq={!geofilt}}
I want to replace my first query with the dismax query but I really need to get the calculated distance in the response. Yes, I can calulate the distance programatically but I would prefer not having to do this as Solr has done it for me already.
I still want to be able to sort my dismax query "by relevance", distance or any other field so the score given by my boosts could be interesting for sorting but I don't need it to be returned.
If I understood correctly you want to have the result of a function in your Solr response. The SOLR-2444 issue is what you're looking for I guess: it allows to include in the fl parameter pseudo-fields, functions etc. The only problem is that it's been committed only on trunk, so it isn't available on the current Solr release, neither will be in the coming 3.6 release. You have to wait for the 4 release but I don't think it will take a lot of time. Maybe you can already start playing around with a snapshot of the last successful Jenkins build.
Pseudo-fields are now available in Solr 4+ which allow you to do just this.
http://localhost:8983/solr/collection1/browse?q=*:*&rows=1000&wt=xml&pt=37.763649,-122.24313&sfield=store&fl=dist:geodist()
For instance, this request allows me to return a field "dist" which contains the distance of each entry to the stated point.
I am trying to make a query in solr.net that generates a solr query with a filter query with more than one term in it e.g.: fq=Size:(4 large)
However, when I pass ?f_Size=(4 large) in the query string to the SolrNet sample app (found here: http://code.google.com/p/solrnet/downloads/list), no results are found.
Looking at the logs, I can see that the filter query generated is fq=Size:"\(4+large\)" so it makes sense that no results are found.
Is there a way in SolrNet to generate a filter query with more than one term?
Where the filter queries are built, try replacing Query.Field(...).Is(...) with Query.Simple(...) (you have to build the query yourself). See the wiki for reference.