Here's my problem: I have a single text field that is indexed by SOLR, which is the usernames from our database. I'd like the search to be fuzzy and not an exact match. Eg; if the username is "krishnarayaprolu" and I search with a spelling mistake "krishnIrayaprolu", it should still return the record.
This is working fine for me except when the usernames have a space in them. So a username: "krishna rayaprolu", and a search string "krishnI rayaprolu~0.5" is not returning the record. It is returning fine if the spelling mistake is at the end like "krishna rayaprolI~0.5". Any ideas?
For my config, I tried WhiteSpaceTokenizerFactory and StandardTokenizerFactory. On the search side, I tried quotes and escaping the space. None of them helped with my space+fuzziness problem. I'm using the admin interface for searching. Appreciate any pointers.
I have solution for your problem, only need to add some fields in your schema.
Create new ngram field and copy all you title name in ngram field.
When you fire any query for missspell word and you get blank result then split
the word and again fire the same query you will get results as expected.
Example : Suppose user searching for word "krishna rayaprolu" but type it as "krishnI rayaprolu~0.5", then
create query in below way you will get results as expected hopefully.
**(ngram:"krishnI rayaprolu~0.5" OR ngram:"kri" OR ngram:"kris" OR ngram:"krish" OR ngram:"krishn" OR ngram:"krishnI" OR ngram:"ray" OR ngram:"raya" OR ngram:"rayap" ..... )**
We have split the word sequence wise and fire query on field ngram.
Hope it will help you.
Related
How to perform a simple query on a text field with an OR condition? Something like name:ABC OR name:XYZ so the resulting set would contain only those docs where name is exactly "XYZ" or "ABC"
Dug tons of manuals, cannot figure this out.
I use Solr 5.5.0
Update: Upgraded to Solr 6.6.0, still cannot figure it out. Below are illustrations to demonstrate my issue:
This works:
This works too:
This still works:
But this does not! Omg why!?
There are many ways to perform OR query. Below I have listed some of them. You can select any of it.
[Simple Query]
q=name:(XYZ OR ABC)
[Lucene Query Parser]
q={!lucene q.op=OR df=name v="XYZ ABC"}
Your syntax is right, but what you're asking for isn't what text fields are made for. A text field is tokenized (split into multiple tokens), and each token is searched by itself. So if the text inserted is "ABC DEF GHI", it will be split into three separate tokens, namely "ABC", "DEF" and "GHI". So when you're searching field:ABC, you're really asking for any document that has the token "ABC" somewhere.
Since you want to perform an exact match, you want to query against a field that is defined as a string field, as this will keep the value verbatim (including casing, so the matching will be case sensitive). You can tell Solr to index the same content into multiple fields by adding a copyFile instruction, telling it to take the content submitted for field foo and also copying it into field bar, allowing you to perform both an exact match if needed and a more general search if necessary.
If you need to perform exact, but case insensitive, searches, you can use a KeywordTokenizer - the KeywordTokenizer does nothing, keeping the whole string as a single token, before allowing you to add filters to the analysis chain. By adding a LowercaseFilter you tell Solr to lowercase the string as well before storing it (or querying for it).
You can use the "Analysis" page under the Solr admin page to experiment and see how content for your field is being processed for each step.
After that querying as string_field:ABC OR string_field:XYZ should do what you want (or string_field:(ABC OR XYZ) or a few other ways to express the same.
A wacky workaround I've just come up with:
I have a requirement where user should able to get same result when searched with a String in reversed or striaght for
example: q="F44" or q="44F" should result same result.
I have created a new field "text_rev" which is assigned to below Field Type.
And I did Copy field with actual/original field "retailId"
<copyField source="retailId" dest="text_rev"/>
<fieldType name="text_rvsstr" class="solr.TextField"><analyzer><tokenizer class="solr.StandardTokenizerFactory"/><filter class="solr.ReverseStringFilterFactory"/></analyzer></fieldType>
when I search with q=text_rev:F44 i get the result but when i search with q=text_rev:44F i get 0 results.
Please advice.
Those searches are on the same field. Searching the reverse direction is only going to work on the reversed field, and searching the forward direction is only going to work on the original field.
By searching both fields for the same information, you can check both directions in one query.
q=retailId:F44 OR text_rev:F44
You need to search both fields. Also, if you actually expect to search in reverse, you need to have asymmetric index and query-type definition. Otherwise your term will get reversed both during indexing and querying and you effectively loose any reason to do so.
You can test that by using Analyse screen of the Admin UI and providing content in both boxes. It will then show how the terms get processed and matched during indexing/querying.
I have a Some String in SOLR "Starting User","Ending User","Friend"
etc... I want to see the results no having user
I tried -t1:*User* but it is not bringing appropriate result.
If t1 is defined as type string, and you indexed the data "You are good boy", and you search with good/boy - you will not get any results. You have to use complete phrase
Solution is, use "text_general" for t1. So then when you index above phrase, it will be tokenized as you, are, good,boy etc and will return results when you search with good/boy
Please try to understand how analyzers, tokenizer and filters work during indexing and searching. happy Learning
The problem is this: I've got a column (named name)which consist of names for Example "Иван Кирилов Петров", "Нина Семова Мариножа" and so on.
So I want to make a query which will get all the names that has first name 'Иван' and last name 'Петров'; The second name doesn't matter so i will put * wildcard character.
Also there is a bigger problem: I should be able in a case if the user writes "Иван Кирилов Петров" to find this exact person
what I have tried :
I made the field text_ws type
and tested the following queries:
q=name:Иван*Петров
perfect - it finds what I want - all the names with first Иван and last Петров;
But then i want to find Иван Кирилов Петров i get no response because I want to make an exact search and my type should be string
How can I solve this!
Try adding autoGeneratePhraseQueries="true" flag on your text_ws type definition. And use debugQuery=true flag to see how it does the matches against the field. If the basic thing work, you can then look at pf3 flag in eDismax configuration to boost the query matches.
Solr also comes with dedicated Token Filters for Russian, but you probably don't care about that for the people's names.
I don't think you need a wild-card query. If you are only splitting on white-space during index time (text_ws) and you get complete first, last and/or middle names for query, you can do an AND query like
q=name:(Иван AND Петров)
or
q=name:(ИВАН AND МИНЧЕВ AND ПЕТРОВ)
Update: After your comment, I see that this will do a bag-of-words search and won't preserve the order. I guess you need to keep a string copy field of name, say name_str, which will give you more search options. For example, if there are 2 spaces in the query, meaning you get the first, middle and last names, then you can do an exact match on name_str like
q=name_str:"ИВАН%20МИНЧЕВ%20ПЕТРОВ"
If you are using Solr 4.0 and above, then regex query on the string field can help you. You can do
q=name_str:/ИВАН.*ПЕТРОВ/
will match anything that begins with ИВАН and ends with ПЕТРОВ.
or even
q=name_str:/Иван.*?Кирилов.*?Петров/
Unfortunately, there is no Solr wiki page on regex search yet, but you can google around.
You need to distinguish between the different types of queries you want to do and do different searches. Maybe give a check-box to your users asking if they want an exact match or not.
For a phrase search, we want to bring up results only if there's an exact match (without ignoring stopwords). If it's a non-phrase search, we are fine displaying results even if the root form of the word matches etc.
We currently pass our data through standardTokenizer, StopFilter, PorterStemFilter and LowerCaseFilter. Due to this when user wants to search for "password management", search brings up results containing "password manager".
If I remove StemFilter, then I will not be able to match for the root form of the word for non-phrase queries. I was thinking if I should index the same data as part of two fields in document.
For the first field (to be used for phrase searches), following tokenizers/filters will be used:
StandardTokenizer, LowerCaseFilter
For the second field (Non-phrase searches)
StandardTokenizer, StopFilter, PorterStemFilter, LowerCaseFilter
Now, based on whether it's a phrase search or not, I need to rewrite user's query to search in the appropriate field.
Is this the right way to address this issue? Is there any other way to achieve this without doubling index size?
let's say user's query is
summary:"Furthermore, we should also fix this"
Internally this will be translated to
summary_field1:"Furthermore, we should also fix this"
If user's query is
summary:(Furthermore, we should also fix this)
Internally this will be translated to
+summary_field2:furthermor +summary_field2:we +summary_field2:should +summary_field2:also +summary_field2:fix
both summary_field1 and summary_field2 index the same data. summary_field1 passes through only StandardTokenizer and LowerCaseFilter, whereas summary_field2 passes through StandardTokenizer, StopFilter, PorterStemFilter and LowerCaseFilter.
Please let me know if I'm missing something here.
By defining two different fields you can search for exact matches.
By using boosts you can also bring results in one query. For example:
(firstField:"password management")^5 OR (secondField:"pasword management")^1