Implement k means clustering in solr - solr

Question

How can i implement k means clustering in solr 6.5 ?
Requirements :-
1) I want to cluster the docs at the query time on the basis of their score
2) I have written my own handler and i want to add the clustering function in that handler only such that it does not the ordering of the docs
I had tried to write the clustering search component as below :-
<searchComponent name="clustering" enable="${solr.clustering.enabled:true}" class="solr.clustering.ClusteringComponent">
<lst name="engine">
<str name="name">kmeans</str>
<str name="carrot.algorithm">org.carrot2.clustering.kmeans.BisectingKMeansClusteringAlgorithm</str>
<str name="BisectingKMeansClusteringAlgorithm.clusterCount">4</str>
<str name="documents">100</str>
<str name="BisectingKMeansClusteringAlgorithm.maxIterations">4</str>
</lst>
</searchComponent>
My Request Handler is as :
<requestHandler name="abc" class="solr.SearchHandler">
<lst name="invariants">
<str name="defType">synonym_edismax</str>
<str name="synonyms">true</str>
<str name="indent">on</str>
</lst>
<lst name="appends">
<str name="fq">search_term</str>
</lst>
<lst name="defaults">
<str name="echoParams">none</str>
<str name="wt">json</str>
<str name="timeAllowed">15000</str>
<str name="qf">Field1</str>
<str name="qf">Field2^0.5</str>
<str name="pf">Field3</str>
<float name="tie">0.2</float>
<str name="fl">Field5,Field6</str>
<str name="facet">false</str>
<str name="mm">2<-1 4<70%</str>
<!-- spellcheck -->
<str name="spellcheck.dictionary">default</str>
<str name="spellcheck">on</str>
<str name="spellcheck.extendedResults">true</str>
<str name="spellcheck.count">1</str>
<str name="spellcheck.onlyMorePopular">true</str>
<str name="spellcheck.collate">true</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
How can i add the clustering in this request handler such that my number of clusters is 4 and iterations is also 4
Also whats the difference between
carrot.url
carrot.snippet
carrot.title
I read the docs definition but i m unable to understand it.

Answer 1

To add the clustering component to a request handler just :
<arr name="last-components">
<str>spellcheck</str>
<str>clustering</str>
</arr>
Then :
<str name="carrot.url">id</str> -> unique key of your document
This is the unique identifier for your document.
<str name="carrot.title">doctitle</str> -> the title(s)/label(s) for your document
This is the field or list of fields, which are short and tend to be more important to group your documents together
<str name="carrot.snippet">content</str> -> the content/text/body of your document
From the wiki :
carrot.title
The field (alternatively comma- or space-separated list of fields) that should be mapped to the logical document’s title. The clustering algorithms typically give more weight to the content of the title field compared to the content (snippet). For best results, the field should contain concise, noise-free content. If there is no clear title in your data, you can leave this parameter blank.
carrot.snippet
The field (alternatively comma- or space-separated list of fields) that should be mapped to the logical document’s main content. If this mapping points to very large content fields the performance of clustering may drop significantly. An alternative then is to use query-context snippets for clustering instead of full field content. See the description of the carrot.produceSummary parameter for details.
carrot.url
The field that should be mapped to the logical document’s content URL. Leave blank if not required.