DSE Search And Solr - Issues with whitespace in UDT search queries - solr

I'm trying to get my DSE search query working (with Solr). However, while constructing queries with User Defined types (UDTs), I'm running into issues with whitespace character.
For eg: I have a Student table and a Name type, where the Student table has a list<frozen<Name> names. Name type has say, firstname and lastname. If I do the below query, it throws an error:
Unable to execute CQL Script : no field name specified in query and no default specified via ‘df’ param.
SELECT * from Student where solr_query= '{!tuple}names.firstname:John
Smith';
So I tried escaping the whitespace as below and it works just fine.
SELECT * from Student where solr_query= '{!tuple}names.firstname:John\
Smith';
But, when I use the above UDT field with an AND operator, it FAILS again.
SELECT * from Student where solr_query= 'student_id:123456 AND {!tuple}names.firstname:John\
Smith';
Unable to execute CQL Script : org.apache.solr.search.SyntaxError: Cannot parse names.firstname … Lexical error at line 1, column … Encountered: after : “”
This is the field type for first name:
<fieldType class="org.apache.solr.schema.TextField" name="DelimitedTextField">
<analyzer>
<tokenizer class="solr.PatternTokenizerFactory" pattern="[,\s]"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
As a beginner with Solr, I've been banging my head trying to make these queries work. Any help would be deeply appreciated. Thanks!

I am no expert at all of the DSE system you seem to be using, but taking a look to this resource[1] it seems you may be building boolean queries in a wrong way.
This seems a correct approach :
+{!tuple v='father.name.firstname:Sam'} +{!tuple v='mother.name.firstname:Anne'}
Hope it helps
[1] http://docs.datastax.com/en/datastax_enterprise/4.8/datastax_enterprise/srch/srchTupleUDTqueries.html

I was able to get this working by doing a couple of things.
I created a new org.apache.solr.schema.TextField and added
PatternTokenizerFactory tokenizer, with comma (,) as the pattern.
Trimmed the white spaces at the beginning and at the end, and
replaced the whitespaces within the text with '?' which matches any
single character. This was ok to do in my case.
I had to add braces () to the entire query.
Hence, with the updated schema.xml file and the other changes mentioned above, I have the following query working now:
SELECT * from Student where solr_query= '(student_id:123456 AND
{!tuple}names.firstname:John?Smith)';
Eventhough this would match John Smith, John-Smith, or even John.Smith, this was ok in my case since we were supposed to give back these results anyway.

Related

Solr special character search on person name

I am a newbie to Solr. I have indexed people name in my collection as below.
When searching for a name like Àlvarez Rubén,I am unable to retrieve the results.I tried escaping using / however I didnt get the correct result.Please help
Use mapping-ISOLatin1Accent.txt before Tokenizing your incoming values.
<charFilter class="solr.MappingCharFilterFactory" mapping="mapping-ISOLatin1Accent.txt"/>

Query Solr accented and unaccented

I'm working on configuring my core solr that save brazilian portuguese data.
About accents, I need to query something like:
search | return
computação | computacao
computacao | computação
What I need basicly is, with or without accent in a query, return both type of words
I tried:
<charFilter class="solr.MappingCharFilterFactory" mapping="mapping-ISOLatin1Accent.txt"/>
<filter class="solr.ASCIIFoldingFilterFactory"/>
Without success
I'm using Solr 5.2.1
Try by adding the BrazilianStemFilterFactory as a filter for your field type which used for searching the field.
This is specifically written for the Brazilian Portuguese.
This could solve your issue.
When using a multilingual index what I have done is create a new field for each language that uses the language specific text field.
So let's say you have English and Portuguese and thus you would declare two fields:
descriptionPt and use text_pt
descriptionEn and use text
Now when you run your search you would specify which field you would like to use or both via qf and specify deftype=edismax.
Worked fine for me.

Solr - search word immediately followed by partial match (with wildcard)

I have a Solr index filled with documents, with a field named issuer.
There is a document with issuer=first issuer.
I'm trying to implement matching of two consequent words. The first word needs to match completely, the second needs to match partially.
What I am trying to achieve is:
I search for something like: issuer:first\ iss*
I expect it to match "first iss uer"
I tried the following solutions but none is working:
issuer:first\ iss* -> returns nothing
issuer:"first iss"* -> returns everything
issuer:(first iss*) -> also returns "issuer first"
Does anybody have a clue on how to achieve the desired result?
My suggestion is to add a shiringle filter based field type to your schema. Below is a simple definition:
<fieldtype name="shingle">
<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.ShingleFilterFactory" minShingleSize="2" maxShingleSize="5"/>
</analyzer>
</fieldtype>
You then add another field with this type as shown below:
<field name="issuer_sh" type="shingle" indexed="true" stored="false"/>
At query time, you can issue the following query:
issuer_sh:"first iss*"
The shingleFilter creates n-gram tokens from your text. For instance, if the issuer field contains "first issue", then Solr will create and index the following tokens:
first
issue
first issue
You can't search with wildcards in phrase queries. Without changing how you are indexing (see #ameertawfik's answer), the standard query parser doesn't provide a good way to do this. You can, however, use the surround query parser to search using spans. This query would then look like:
1N(first, iss*)
Keep in mind, surround query parser does not analyze, so 1N(first, iss*) and 1N(First, iss*) will not find the same results.
You could also construct this query using lucene's SpanQueries directly, of course, like:
SpanQuery[] queries = new SpanQuery[2];
queries[0] = new SpanTermQuery(new Term("issuer","first"));
queries[1] = new SpanMultiTermQueryWrapper(new PrefixQuery(new Term("issuer","iss")));
Query finalQuery = new SpanNearQuery(queries, 0, true);

Solr lucene search only first word in a sentence - need query

I am searching data which matches only that sentence which starts with the search key.
Ex:
search key "what"
result : DESIRED ONE
**what** is your name
**what** are you doing
**what** is that
etc.
How i am getting now is
Is that **what** you want
some text before **what**
etc.
i am using EdgeNGram as well..But it is giving me the second one.
Any help appreciated....
You can use a regex 'starts with query' q:(name:/what.*/) will yield results where the name field start with 'what'
I think you're using the WhitespaceTokenizer. You should try using the KeywordTokenizer with EdgeNGram from the left end.
If you're trying to implement auto suggest, have a look at the Suggester component.
I think you can use the following trick: add the following line to your field definition:
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="^(.*)$" replacement="AAAA $1" />
This will add the "AAAA" token to both your index and query. This way you can match the beginning of an indexed string.
More info here:
http://robotlibrarian.billdueber.com/2012/03/boosting-on-exactish-anchored-phrase-matching-in-solr-sst-4/
example: https://github.com/billdueber/solr_stupid_tricks/blob/SST4/solr/conf/schema.xml (search for "text_l")
http://blogs.perl.org/users/mark_leighton_fisher/2012/01/stupid-lucene-tricks-exact-match-starts-with-ends-with.html

How to force solr QParserPlugin not to use whitespace tokenizer for Keyword fields?

I have keyword field in Solr schema.
<fieldType name="text_keyword" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.SimpleKeywordTokenizerFactory"/>
</analyzer>
</fieldType>
When I try to search this field with default solr query or dismax query
category:(Mouse Pad) it creates query (category:Mouse) AND (category:Pad)
I want to know is there a way not to split terms by whitespaces if it is keyword field or so.
Added:
I need SimpleKeywordTokenizerFactory analyze (which is lowercase without white-space splitting) on query, so raw and term query parser doesn't work for me
You want to enter this query:
category:"Mouse Pad"
The query syntax already provides a way to do this. Quotes are for phrases. Parentheses mean something different. You can write your own query parser if you want, but I don't recommend it.
You could use TermQParserPlugin:
{!term f=category}Mouse Pad
Beware that no analysis is performed, so this will only work if the internal representation of your field is "Mouse Pad" (with title case).
Edit (2012-04-17):
If you still want analysis to be performed, all you need to do is to escape the space by prepending a backslash:
{!lucene}category:Mouse\ Pad

Resources