I have a problem with Solr where I can't reconcile inexact search with multiple words.
Currently my Solr is configured thus: query=ctnt_val:*keyword* where ctnt_val is the field I'm searching and keyword the value I pass.
So if I type lon it will return all results with longer, London, ... which is what I want.
The problem is that if my query is several words long (e.g., Gotham City), it returns all results containing Gotham and all results containing City, instead of returning only all results containing Gotham City.
If I change the query as query=ctnt_val:"keyword" it works but then I lose the ability to do inexact searches (lon will no longer return London). If I do query=ctnt_val:*"keyword"* I get ALL results from my DB, which is clearly not what I want.
Any ideas?
The default search handler uses to have the OR as the default query operator.
You can check using q=ctnt_val:gotham\ city&q.op=AND which will set the query operator to AND and would make all the query terms as mandatory.
gotham\ city - \ is to escape the space from the query
Related
I'm following this tutorial on how to perform indexing on sample documents using Solr. The default collection is "gettingstarted" as shown. Now I'm trying to query it. There are 52 entries as shown:
However, when I replace the q argument with say electronics, it should return 14 results. However, I get nothing.
When I replace the query string q with cat:electronics, then I actually get the 14 results. But why is this the case? isn't q=word supposed to search for word wherever it appears?
No, it's not. Your assumption that:
isn't q=word supposed to search for word wherever it appears?
is wrong. If you're using word as your only query, and nothing more - you're searching for word in the default search field. It does not search all available fields in all available documents.
Also be aware that the default query parser assumes that your query is in the Lucene Query Syntax. To handle more "natural" querying, you can use the edismax query parser. This query parser supports the qf parameter that tells Solr which fields to search, instead of having to use the cat:electronics syntax. Your example would then be q=electronics&qf=cat.
In the example documents you've given, qf=series_t author name cat is probably a decent value to search all these fields for the given query. You can also append ^<weight> to a field name to give hits in the different fields different weights. qf=name^10 cat would give a hit in name ten times the weight of a hit in the cat field.
I have indexed following document in Solr with app_name is multi-word string eg."Fire inspection" ,
{
"app_name":"Fire inspection",
"appversion":1,
"id":"app_1397_version_2417",
"icon":"/images/media/default_icons/app.png",
"type":"app",
"app_id":1397,
"account_id":556,
"app_description":"fire inspection app",
"_version_":1599441252925833216}]
}
if i execute following Solr query, Solr returning wrong response,
Query:
http://localhost:8983/solr/AxoSolrCollectionLocal/select?fq=app_name:*fire P*&q=*:*
I'm searching for record's whose app_name contains "fire P" but getting -response whose app_name contains "fire inspection". Here, string 'Fire P' does not match with below record but still it is responded by Solr.
Response:
{
"app_name":"Fire inspection",
"appversion":1,
"id":"app_1397_version_2417",
"icon":"/images/media/default_icons/app.png",
"type":"app",
"app_id":1397,
"account_id":556,
"app_description":"fire inspection app",
"_version_":1599441252925833216}]
}
Can someone please help me with the Solr query (same as that of like query in SQL) which will check for substring and spaces will not be mattered.
Your help is greatly appreciated.
First - your query does not mean what you think it means. app_name:*fire P* means "search for anything ending in fire in the field app_name and/or anything starting with p in the default search field". Since you haven't prefixed the second value with a field name, the default search field will be used.
If you want to search for a substring match inside a field like that (i.e. something that contains "fire P" as a substring inside the value, the field type has to be made a string field - or a field with a keyword tokenizer - that way the field retains its actual value, and it's not processed / filtered / tokenized further. If it's being tokenized, those tokens (i.e. fire, inspection etc) will be stored separately. You'll have to escape any spaces properly and query a single field (i.e. app_name:fire\ P`), and depending on the use case, performance may take a hit unless you have the ReversedWildcardFilter enabled as well.
However, you can probably also use the ComplexPhraseQueryParser to get support for wilcards in phrase queries:
{!complexphrase inOrder=true}app_name:"*fire P*"
should work, as long as you actually have uppercase letters in your tokens (wildcards disables many filters, so usually you'll want to match the end syntax in your tokens.
I have indexed the following record in my collection
{
"app_name":"atm inspection",
"appversion":1,
"id":"app_1427_version_2449",
"icon":"/images/media/default_icons/app.png",
"type":"app",
"app_id":1427,
"account_id":556,
"app_description":"inspection",
"_version_":1599625614495580160}]
}
and It's working fine unless an until i search records case sensitively i.e if i write following Solr query to search records whose app_name contains atm then Solr is returning above response which is a correct behaviour.
http://localhost:8983/solr/NewAxoSolrCollectionLocal/select?fq=app_name:*atm\ *&q=*:*
However, If i execute following Solr query to search records whose app_name contains ATM
http://localhost:8983/solr/NewAxoSolrCollectionLocal/select?fq=app_name:*ATM\ *&q=*:*
Solr is not returning above response because ATM!=atm.
Can someone please help me with the Solr query to search records case insensitively.
Your help is greatly appreciated.
You can't. The field type string requires an exact match (it's a single, unprocessed token being stored for the field value).
The way to do it is to use a TextField with an associated Tokenizer and a LowercaseFilter. If you use a KeywordTokenizer, the whole token will be kept intact (so it won't get split as you'd usually assume with a tokenizer), and since it's a TextField it can have a analysis chain associated - allowing you to add a LowercaseFilter.
The LowerCaseFilter is multiterm aware as far as I remember, but remember that wildcard queries will usually not have any filters applied. You should therefor lowercase the value before creating your query yourself (even if it probably will work in this simple case).
Solr newbie here.
I have created a Solr index and write a whole bunch of docs into it. I can see
from the Solr admin page that the docs exist and the schema is fine as well.
But when I perform a search using a test keyword I do not get any results back.
On entering * : *
into the query (in Solr admin page) I get all the results.
However, when I enter any other query (e.g. a term or phrase) I get no results.
I have verified that the field being queried is Indexed and contains the values I am searching for.
So I am confused what I am doing wrong.
Probably you don't have a <defaultSearchField> correctly set up. See this question.
Another possibility: your field is of type string instead of text. String fields, in contrast to text fields, are not analyzed, but stored and indexed verbatim.
I had the same issue with a new setup of Solr 8. The accepted answer is not valid anymore, because the <defaultSearchField> configuration will be deprecated.
As I found no answer to why Solr does not return results from any fields despite being indexed, I consulted the query documentation. What I found is the DisMax query parser:
The DisMax query parser is designed to process simple phrases (without complex syntax) entered by users and to search for individual terms across several fields using different weighting (boosts) based on the significance of each field. Additional options enable users to influence the score based on rules specific to each use case (independent of user input).
In contrast, the default Lucene parser only speaks about searching one field. So I gave DisMax a try and it worked very well!
Query example:
http://localhost:8983/solr/techproducts/select?defType=dismax&q=video
You can also specify which fields to search exactly to prevent unwanted side effects. Multiple fields are separated by spaces which translate to + in URLs:
http://localhost:8983/solr/techproducts/select?defType=dismax&q=video&qf=features+text
Last but not least, give the fields a weight:
http://localhost:8983/solr/techproducts/select?defType=dismax&q=video&qf=features^20.0+text^0.3
If you are using pysolr like I do, you can add those parameters to your search request like this:
results = solr.search('search term', **{
'defType': 'dismax',
'qf': 'features text'
})
In my case the problem was the format of the query. It seems that my setup, by default, was looking and an exact match to the entire value of the field. So, in order to get results if I was searching for the sit I had to query *sit*, i.e. use wildcards to get the expected result.
With solr 4, I had to solve this as per Mauricio's answer by defining type="text_en" to the field.
With solr 6, use text_general.
I currently have a SOLR query which uses the query (q), query fields (qf) and phrase fields (pf) to retrieve the results I want. An example is:
/solr/select
?q=superbowl
&qf=title^3+headline^2+intro+fulltext
&pf=title^3+headline^2+intro+fulltext
&fl=id,title,ts_modified,score
&debugQuery=true
The idea is that the title and headline of the "main item" give the best indication of what the result is "about", but the intro and fulltext provides some input too. Ie, imagine a collection of links, where the collection itself has metadata (what it's a collection of), but each link has it's own data (title of the link, synopsis, etc). If we search for "superbowl", the most relevant results are the ones with "superbowl" in the collection metadata, the least relevant results are those with "superbowl" in just the synopsis of one of the links... but they're all valid results.
What I'm trying to do is add a boost to the relevancy score so that the most recent results float towards the top, but retaining title,headline,intro,fulltext as part of the formula. A recent result with the search string in the collection metadata would be more relevant than one with it only in the links metadata... but that "links only" recent result might be more relevant than a very old result with the search string in the collection metadata. (I hope that's somewhat clear).
The problem is that I can't figure out how to combine the boost function documented on the SOLR site with the use of the qf/pf fields. Specifically...
From the SOLR site, something like the following works to boost the results by date:
/solr/select
?q={!boost%20b=$dateboost%20v=$qq}
&dateboost=ord(ts_modified)
&qq=superbowl
&fl=ts_modified,score
&debugQuery=true
However, I can't figure out how to combine that query with the use of qf and pf. Any suggestions would be more than welcome.
Thanks to danben's response, I was able to come up with the following:
/solr/select
?q={!boost%20b=$dateboost%20v=$qq%20defType=dismax}
&dateboost=ord(ts_modified)
&qq=superbowl
&qf=title^3+headline^2+intro^2+fulltext
&pf=title^3+headline^2+intro^2+fulltext
&fl=ts_modifieds,score
&debugQuery=true
It looks like the actual problems I was having were:
I left spaces in the q param instead of escaping them (%20) when copy/pasting
I didn't include the defType=dismax in my q param, so that it would pay attention to the qf/pf parameters
Check out http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents
This is based on the ms function, which returns the difference in milliseconds between two timestamps / dates, and ReciprocalFloatFunction which increases as the value passed decreases.
Since you are using the DisMaxRequestHandler, you may need to specify your query using the bq/bf parameters. From http://lucene.apache.org/solr/api/org/apache/solr/handler/DisMaxRequestHandler.html:
bq - (Boost Query) a raw lucene query that will be included in the
users query to influence the score. If
this is a BooleanQuery with a default
boost (1.0f), then the individual
clauses will be added directly to the
main query. Otherwise, the query will
be included as is. This param can be
specified multiple times, and the
boosts are are additive. NOTE: the
behaviour listed above is only in
effect if a single bq paramter is
specified. Hence you can disable it by
specifying an additional, blank, bq
parameter.
bf - (Boost Functions) functions (with optional boosts) that will be
included in the users query to
influence the score. Format is:
"funcA(arg1,arg2)^1.2
funcB(arg3,arg4)^2.2". NOTE:
Whitespace is not allowed in the
function arguments. This param can be
specified multiple times, and the
functions are additive.
Here is a nice article about Date-boosting Solr search results:
http://www.metaltoad.com/blog/date-boosting-solr-drupal-search-results
In Drupal this can be simply achieved by the following code:
using Apachesolr module
/**
* Implements hook_apachesolr_query_alter().
*/
function hook_search_apachesolr_query_alter(DrupalSolrQueryInterface $query) {
$query->addParam('bf', array('freshness' =>
'recip(abs(ms(NOW/HOUR,dm_field_date)),3.16e-11,1,.1)'
));
}