solr facet search truncate words - solr

Question

have a solr configured for french content. Search is fine, but when i activate facet search, words are truncated in a special way.
All e disappear, for eg automobil instead of automobile, montagn instead of montagne, styl instead of style , homm => homme etc....
<lst name="keywords">
<int name="automobil">1</int>
<int name="citroen">1</int>
<int name="minist">0</int>
<int name="polit">0</int>
<int name="pric">0</int>
<int name="shinawatr">0</int>
<int name="thailand">0</int>
</lst
here is the query q=fulltextfield:champpions&facet=true&facet.field=keywords
the keyword content :
<arr name="keywords">
<str>Ski</str>
<str>sport</str>
<str>Free style</str>
<str>automobile</str>
<str>Rallye</str>
<str>Citroen</str>
<str>montagne</str>
</arr>
here is the schema used :
<fieldtype name="text_fr" class="solr.TextField">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords_fr.txt"/>
<filter class="solr.WordDelimiterFilterFactory" preserveOriginal="1" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0" />
<filter class="solr.ISOLatin1AccentFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.SnowballPorterFilterFactory" language="French"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords_fr.txt"/>
<filter class="solr.WordDelimiterFilterFactory" preserveOriginal="1" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1" />
<filter class="solr.ISOLatin1AccentFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.SnowballPorterFilterFactory" language="French"/>
</analyzer>
</fieldtype>
the field def :
If somebody have an idea about that issue....
Thanks for your answer.
regards
Jerome longet

Answer 1

Generally, if you want to use a field as a facet, it should be stored as a string.
You're faceting on a tokenized and filtered field, so the individual values are the processed words in your keywords field.

Answer 2

All above said is correct, I just want to add one thing one facets. The facet values are the indexed terms, and not the stored ones. One recommendation for facets is to use a string-type. This is often a good choice. But sometimes you would like to to some things to your facet terms. In that case, you can use a text type, but treat the input only lightly. Avoid in any case your above choices of Stemming (SnowballPorter) or WordDelimiter.
A good choice to start with is KeywordTokenizerFactory, you could to PatternReplace to clean up your terms and input, and do a TrimFilter at the end. Don't do lowercasing, if your users are going to see the terms.
An example, my input are alphabetic language codes. The PatternReplace clean up non-alphabetic characters, the second correct an input-mistake:
`
<analyzer>
<tokenizer class="solr.KeywordTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.PatternReplaceFilterFactory"
pattern="([^a-z])"
replacement=""
replace="all" />
<filter class="solr.PatternReplaceFilterFactory"
pattern="fer|xxx"
replacement="und"
replace="all" />
<filter class="solr.LengthFilterFactory" min="3" max="3" />
</analyzer>
`
Have fun with solr
Oliver