I've defined a saved search in PW explorer using the search builder but it seems there are limits to the number of items in the query string. If I try to define more than 8 search terms, I get the following error message:
I also cannot add additional criteria using an AND statement as a separate line in the search query for a particular parameter. You will note that I have multiple queries on multiple general property attributes in an attempt to circumvent this problem, but I have now have run out of allowed search terms (given the limits described above), that will give me meaningful filtering results.
So my question is, is it possible to change the buffer size, described in the error message, to accommodate additional search criteria?
Related
Say I have the following solr/lucene query:
https://some_website.com//api/myapi/search?profile=myprofile&fl=&fq=batchid:,bodytextsize_i*&q=word_i_want_to_search&partner=mypartnerid&rows=10
I would like to know what I can add to this in order to have the sentence or a certain range of words around the query word returned to me in the response. For instance say a document has the following sentence:
The computer's word_i_want_to_search is broken.
I would like to be able to query word_i_want_to_search and have the response show me the entire sentence. Is this possible?
Thank you!
This is known as highlighting:
Highlighting in Solr allows fragments of documents that match the
user’s query to be included with the query response.
The fragments are included in a special section of the query response
(the highlighting section), and the client uses the formatting clues
also included to determine how to present the snippets to users.
Fragments are a portion of a document field that contains matches from
the query and are sometimes also referred to as "snippets" or
"passages".
Use the parameters hl=true&hl.fl=field_you_are_searching and go from there. There's a lot of small things you can tweak to get different behaviors, such as how much data is being included in the response.
I am attempting to implement a search in solr 5.5 which requires faceting on child document fields. I realize that flattening the data structure is the ideal solution for solr search but unfortunately because of business requirements of the search, I am required to maintain a relationship between various fields (hence the child documents).
I am experimenting with using the BlockJoinFacetComponent to facet on child document fields, and I am able to get everything working and get the counts I expect using the basic example, no problems there. The issue I am facing is that the BlockJoinFacetComponent requires a ToParentQuery, and I can't figure out how to combine this with my original search query and still get facet results.
To explain further:
I am basically following this example: http://www.slideshare.net/lucidworks/faceting-with-lucene-block-join-query-oleg-savrasov
In the example, the user originally searches for "dress", and then is shown facets to filter down by size, color. Size and Color are child fields, and the BlockJoinFacetComponent is used in the example to facet by size and color and retrieve the expected counts.
In the example, the query used to retrieve said facets (slide 22) is:
q= {!parent which="scope:product"} COLOR: Blue
child.facet.field = SIZE
Which works fine. What I am not understanding is in this example we have now lost the original search for "dress". So my question is basically how can I combine my original search (dress) with the ToParentQuery? I have tried everything I can think of to combine the queries, but I always end up getting the same exception:
"Block join faceting is allowed with ToParentBlockJoinQuery only".
I have even downloaded the solr source code and hooked up a remote debugger where this error is being thrown to try and debug this, but I still can't figure it out. No matter what I do it seems like unless the ToParentBlockJoinQuery is the only thing in the query, the BlockJoinFacetComponent will reject it. Which seems odd considering to use the component you've now lost what the user originally searched for.
After further debugging, the issue stems from the fact that BlockJoinFacetComponent seems to not be able to separate the ToParentBlockJoinQuery portion of a query if you are using a query parser other than the standard parser (I was using edismax).
For example, with the standard query parser, this works:
"original query" + _query_:"{!parent which="scope:product"} COLOR: Blue"
child.facet.field = SIZE
If you run this same query with a dismax or edismax query parser, you receive the error:
"Block join faceting is allowed with ToParentBlockJoinQuery only".
Since I am dependent on the edismax query parser, this was a show stopper for me. However, I was able to achieve the results I desired instead by using the JSON Facet API: http://yonik.com/solr-nested-objects/#faceting
I'm using Solr 5.x, standard highlighter, and i'm getting snippets which matches even one of the search terms only, even if i indicate q.op=AND.
I need ONLY the fields and snippets that matches ALL the terms (unless i say q.op=OR or just omit it), i.e. the field/snippet must satisfy the query. Solr does return the field/snippet that has all the terms, but also return many others.
I'm using hl.fl=*, to get the only fields having the terms, and searching against the default field ('text' containing full doc). Need to use * since i have multiple dynamic fields. Most fields are 'text_general' type (for search and HL), and some are 'string' type for faceting.
If its not possible for snippets to have all the terms, i MUST get only the fields that satisfy the query fully (since the question is more talking about matching all the terms, but the search query can become arbitrarily complex, so the fields/snippets should match the query).
Also, next is to get snippets highlighted with proximity based search/terms. What should i do/use for this? The fields coming in highlighting in this scenario should also satisfy the proximity query (unlike i get a field that contain any term, without regard to proximity constrains and other query terms etc)
Thanks for your help.
I've also encountered the same problem with highlighting. In my case, the query like
(foo AND bar) OR eggs
highlighted eggs and foo despite bar was not present in the document. I didn't manage to come up with proper solution, however I devised a dirty workaround.
I use the following query:
id:highlighted_document_id AND text:(my_original_query)
with debugQuery set to true. Then I parse explain text for highlighted_document_id. The text contains the terms from the query, which have contributed to the score. The terms, which should not be highlighted, are not present in the explanation.
The Python regex expressions I use to extract the terms (valid for Solr 5.2.1):
term_regex = re.compile(r'weight\(text:(.+) in')
wildcard_term_regex = re.compile(r'text:(.+), product')
then I simply search the markings in the highlighted text and remove them if the term doesn't match against any of the term in term_regex and wildcard_term_regex.
The solution is probably pretty limited, but works for me.
The problem is this: I've got a column (named name)which consist of names for Example "Иван Кирилов Петров", "Нина Семова Мариножа" and so on.
So I want to make a query which will get all the names that has first name 'Иван' and last name 'Петров'; The second name doesn't matter so i will put * wildcard character.
Also there is a bigger problem: I should be able in a case if the user writes "Иван Кирилов Петров" to find this exact person
what I have tried :
I made the field text_ws type
and tested the following queries:
q=name:Иван*Петров
perfect - it finds what I want - all the names with first Иван and last Петров;
But then i want to find Иван Кирилов Петров i get no response because I want to make an exact search and my type should be string
How can I solve this!
Try adding autoGeneratePhraseQueries="true" flag on your text_ws type definition. And use debugQuery=true flag to see how it does the matches against the field. If the basic thing work, you can then look at pf3 flag in eDismax configuration to boost the query matches.
Solr also comes with dedicated Token Filters for Russian, but you probably don't care about that for the people's names.
I don't think you need a wild-card query. If you are only splitting on white-space during index time (text_ws) and you get complete first, last and/or middle names for query, you can do an AND query like
q=name:(Иван AND Петров)
or
q=name:(ИВАН AND МИНЧЕВ AND ПЕТРОВ)
Update: After your comment, I see that this will do a bag-of-words search and won't preserve the order. I guess you need to keep a string copy field of name, say name_str, which will give you more search options. For example, if there are 2 spaces in the query, meaning you get the first, middle and last names, then you can do an exact match on name_str like
q=name_str:"ИВАН%20МИНЧЕВ%20ПЕТРОВ"
If you are using Solr 4.0 and above, then regex query on the string field can help you. You can do
q=name_str:/ИВАН.*ПЕТРОВ/
will match anything that begins with ИВАН and ends with ПЕТРОВ.
or even
q=name_str:/Иван.*?Кирилов.*?Петров/
Unfortunately, there is no Solr wiki page on regex search yet, but you can google around.
You need to distinguish between the different types of queries you want to do and do different searches. Maybe give a check-box to your users asking if they want an exact match or not.
For a phrase search, we want to bring up results only if there's an exact match (without ignoring stopwords). If it's a non-phrase search, we are fine displaying results even if the root form of the word matches etc.
We currently pass our data through standardTokenizer, StopFilter, PorterStemFilter and LowerCaseFilter. Due to this when user wants to search for "password management", search brings up results containing "password manager".
If I remove StemFilter, then I will not be able to match for the root form of the word for non-phrase queries. I was thinking if I should index the same data as part of two fields in document.
For the first field (to be used for phrase searches), following tokenizers/filters will be used:
StandardTokenizer, LowerCaseFilter
For the second field (Non-phrase searches)
StandardTokenizer, StopFilter, PorterStemFilter, LowerCaseFilter
Now, based on whether it's a phrase search or not, I need to rewrite user's query to search in the appropriate field.
Is this the right way to address this issue? Is there any other way to achieve this without doubling index size?
let's say user's query is
summary:"Furthermore, we should also fix this"
Internally this will be translated to
summary_field1:"Furthermore, we should also fix this"
If user's query is
summary:(Furthermore, we should also fix this)
Internally this will be translated to
+summary_field2:furthermor +summary_field2:we +summary_field2:should +summary_field2:also +summary_field2:fix
both summary_field1 and summary_field2 index the same data. summary_field1 passes through only StandardTokenizer and LowerCaseFilter, whereas summary_field2 passes through StandardTokenizer, StopFilter, PorterStemFilter and LowerCaseFilter.
Please let me know if I'm missing something here.
By defining two different fields you can search for exact matches.
By using boosts you can also bring results in one query. For example:
(firstField:"password management")^5 OR (secondField:"pasword management")^1