Just tried GAE(1.7.7 Java) Full Text Search and found if the search string is work, surprisingly it will not match working, worked, or hardworking, homework, I'd like to know if i miss something in the API, i read the tutorial but did not found any document about this except plural match.
Thanks.
P.S. I tried unit test for search service, not in working environment.
Tucked away in the docs (but unfortunately not in the table of operators), there is a '~' operator
To search for plural variants of an exact query, use the ~ operator:
~"car" # searches for "car" and "cars"
Not sure how far that will get you. Unfortunately thats about it.
See https://developers.google.com/appengine/docs/java/search/overview#Queries_on_Fields
There is so little documentation on this,but just having tried it, it just works on plurals.
One approach would be to do your own stemming on the words in the document, (though you wouldn't return that as the text ;-) Then you could perform stemming on your search term and be able to match worked, working etc..
This is a late answer, but to follow up the previous answer, what you want to do is not possible with the basic API functions. The search API works on full-text searching principles. To get around this you can tokenise your searchable data pre-index and store this in a field with the relevant document.
See: Partial matching GAE search API
Related
I'm trying to implement an Azure suggester feature into our pilot Azure search app and running into issues. The content I'm indexing are PDF files, so my suggester definition is based on the content field itself which can be thousands of lines of text. Following examples online, when I implement the suggester, I'm returned the entire content of the body of text from the PDF file. What I'd really like to do is return just a phrase found in the text.
For instance, suppose I'm indexing a Harry Potter book and I type into my search field "Dum", I'd like to see suggested results back like "Dumbledore", "Dementor", etc VS the whole book. Is this possible?
Tks
If we want to search for words sharing the same prefix, Autocomplete is the right API for this job. https://learn.microsoft.com/en-us/rest/api/searchservice/autocomplete
In contrast, Suggester API helps users find the documents containing words with that prefix. It returns text snippets containing those worlds.
If you still believe suggester api does not behave as expected and autocomplete is not suitable, let me know your source document, query and expected results.
My use case:
I have a single-valued field called cqpath. This is a textfield and has a values that look something like the following:
"/content/domain/en/path/to/some/page"
"/content/domain/en/path/to/another/page"
"/content/domain/en-us/path/to/some/page"
"/content/domain/en-us/path/to/another/page"
I wanted to form a query that would return me 1. and 2. I'd been trying along the lines of writing:
cqpath: "/content/domain/en"
which has been discovered to be erroneous, since it retrieves items 3. and 4. as well. Could any of you think of a way to write a query that returns only 1. and 2. and not 3. and 4.?
This is a normal textfield field-type. Really do appreciate your help.
Starting from Solr 4.0 you can use a regex query. You can find some useful examples here.
In your case, you can get the results that you're looking for using something like:
cqpath:/.*content/domain/en.*/
It looks like you are trying to match partial paths here with boundaries on path elements (slashes). The usual generic solution is to tokenize during index to generate all alternative completions and not tokenize during query. So, the field type declaration is not symmetric. There are examples of that in Solr distribution. And you would look at using something like (index-time only) EdgeNGramFilterFactory instead of much more expensive regex matching.
For your specific case, you may want to look at testPathHierarchyTokenizer which does that for you automatically.
And if your content were more like full URLs than just path, you could also be interested by a custom update request processor chain that includes URLClassify URP. It is not very documented, but mentions generating url parts, which is what I think you would want.
I use Solr 5 for searching in large (text) documents. For each search result, I display a fragment containing the highlighted search match. This works nicley using Solr's Standard Highlighter. Yet I found that if several matches are found close to each other, they will be merged into one fragment, even with hl.mergeContiguous=false. Params are
SolrQuery query = new SolrQuery();
query.setQuery(rawQuery);
query.set("defType", "lucene");
query.setRows(1000);
query.setHighlight(true);
query.setHighlightFragsize(200);
query.setHighlightSnippets(20);
query.setParam("hl.fl", "content");
query.setParam("hl.maxAnalyzedChars", "-1");
query.setParam("hl.mergeContiguous", false);
Example: I use a bible translation for testing, just because of its length. Searching for beast yields (among many others)
...7:8 Of clean beasts, and of beasts that are not clean, and of birds, and of everything that creeps upon the ground, 7:9 there went in two and two to Noah into...
I would rather have this fragment twice, because it contains two occurrences of the search term. Manually duplicating the fragment in this case appears clumsy to me. Am I missing a query parameter, or do I need a custom BoundaryScanner to achieve this?
You can think of using hl.regex - regex based fragmenter, and prepare the regex based on your terms and attach to the request. look for related hl.regex.slop, hl.regex.maxAnalyzedChars params also if you want to try this.
Or can reduce the fragment size for standard highlighter: hl.fragsize to something you think two occurrences of your terms may not be existing within.
BoundaryScanner works with FastVectorHighlighter only, and can be the option if no OOTB param works.
In the new GAE API for Full Text Search, I can't find any option to activate stemming. I have tried to search for singular/plural words in my application, and indeed searching for "document" does not return the same result set as searching for "documents". Same goes for accentuated characters, searching for "vehicule" or "véhicule" does not return the same result set.
Is there an option somewhere, either in the API or in the query language syntax, that I can use to activate stemming ? Or do I have to build my own stemming by pre-processing the query and translate for example "document" into "(document OR documents)" ?
In this other SO question they discuss the same. You should use the now documented ~ operator
you should assume charset type.
My text is like this: The searched word WildCard shall be partially highlighted
I search using wildcard expression "wild*".
What I need is to have the highlight snippet to be [tag]Wild[/tag]Card. What I got was [tag]WildCard[/tag], and I spent lots of time researching it, but could not find an answer.
This behavior can be illustrated on linkedin.com, where you type other people's name at the top right corner.
Once this is figured out, I will have a follow-up questions.
Thanks,
I am not sure if you can achieve what you want directly in solr. The obvious solution is to parse the returned doc yourself searching for [tag]WildCard[/tag] and find out what part of the term you need to highlight.
This is not possible with Solr. What you need to do is change the characters Solr uses to highlight found words to something you can easily remove (maybe an html comment) and then build a highlight yourself. You are likely already storing your query in a variable, so just scan the search return docs and highlight all instances of your query.