I am using Solr 8.6.1, started in solrcloud mode.
The field type is
{
"add-field-type" : {
"name":"articleTitle",
"positionIncrementGap":100,
"multiValued":false,
"class":"solr.TextField",
"indexAnalyzer":{
"tokenizer":{ "class":"solr.StandardTokenizerFactory" },
"filters":[
{ "class":"solr.LowerCaseFilterFactory" },
{ "class":"solr.ManagedStopFilterFactory", "managed":"english" },
{ "class":"solr.ManagedSynonymGraphFilterFactory", "managed":"english" },
{ "class":"solr.FlattenGraphFilterFactory" },
{ "class":"solr.PorterStemFilterFactory" }
]
},
"queryAnalyzer":{
"tokenizer":{ "class":"solr.StandardTokenizerFactory" },
"filters":[
{ "class":"solr.LowerCaseFilterFactory" },
{ "class":"solr.ManagedStopFilterFactory", "managed":"english" },
{ "class":"solr.ManagedSynonymGraphFilterFactory", "managed":"english" },
{ "class":"solr.PorterStemFilterFactory" }
]
}
}
}
After I add a document
{
"id": 100,
"articleTitle": "Best smartphone"
}
I update the synonyms list by API
curl -X PUT -H 'Content-type:application/json' --data-binary '["iphone", "smartphone"]' "http://localhost:8983/solr/articles/schema/analysis/synonyms/english"
and reload the collection by API
http://localhost:8983/solr/admin/collections?action=RELOAD&name=articles
However when I try to search the documents don't pop-up.
http://localhost:8983/solr/articles/select?q=articleTitle:iphone
No result are returned. I expected that added document will be returned.
It works only if I first update the synonyms list and after that add the document into collection.
How to configure Solr to find the documents by synonyms if the synonyms are changed after documents are created?
Related
I am creating a custom catch-all field with a porter stemmer in Apache Solr. I want to index and query the data using this field.
First I created a field type using the following JSON
{
"add-field-type":{
"name":"text_general_extended",
"class":"solr.TextField",
"positionIncrementGap":"100",
"multiValued":true,
"indexAnalyzer":{
"tokenizer":{
"class":"solr.StandardTokenizerFactory"
},
"filters":[
{
"class":"solr.StopFilterFactory",
"words":"stopwords.txt",
"ignoreCase":"true"
},
{
"class":"solr.PorterStemFilterFactory"
},
{
"class":"solr.LowerCaseFilterFactory"
}
]
},
"queryAnalyzer":{
"tokenizer":{
"class":"solr.StandardTokenizerFactory"
},
"filters":[
{
"class":"solr.StopFilterFactory",
"words":"stopwords.txt",
"ignoreCase":"true"
},
{
"class":"solr.PorterStemFilterFactory"
},
{
"class":"solr.SynonymGraphFilterFactory",
"expand":"true",
"ignoreCase":"true",
"synonyms":"synonyms.txt"
},
{
"class":"solr.LowerCaseFilterFactory"
}
]
}
}
}
Then I created a field with the above field type
{
"add-field":{
"name":"_text_extended",
"type":"text_general_extended",
"multiValued":true,
"indexed":true,
"stored":false
}
}
Finally, I created a copy field and added my text field i.e. passage_text in destination '_text_extended'
{"add-copy-field" : {"source":"passage_text","dest":"_text_extended"}}
I created all the JSON using the _default configset in Solr. I am not sure what am I missing here.
Appreciate the help!
This is an example of what my data looks like for an Elastic Search index called video_service_inventory:
{
'video_service': 'netflix',
'movies' : [
{'title': 'Mission Impossible', 'genre: 'action'},
{'title': 'The Hangover', 'genre': 'comedy'},
{'title': 'Zoolander', 'genre': 'comedy'},
{'title': 'The Ring', 'genre': 'horror'}
]
}
I have established in my index that the "movies" field is of type "nested"
I want to write a query that says "get me all video_services that contain both of these movies":
{'title': 'Mission Impossible', 'genre: 'action'}
AND
{'title': 'The Ring', 'genre': 'horror'}
where, the title and genre must match. If one movie exists, but not the other, I don't want the query to return that video service.
Ideally, I would like to do this in 1 query. So far, I haven't been able to find a solution.
Anyone have suggestions for writing this search query?
the syntax may vary depending on elasticsearch version, but in general you should combine multiple nested queries within a bool - must query. For nested queries you need to specify path to "navigate" to the nested documents, and you need to qualify the properties with the part + the field name:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "movies",
"query": {
"bool": {
"must": [
{ "terms": { "movies.title": "Mission Impossible" } },
{ "terms": { "movies.genre": "action" } }
]
}
}
}
},
{
"nested": {
"path": "movies",
"query": {
"bool": {
"must": [
{ "terms": { "movies.title": "The Ring" } },
{ "terms": { "movies.genre": "horror" } }
]
}
}
}
}
]
}
}
}
This example assumes that the title and genre fields are not analyzed properties. In newer versions of elasticsearch you may find them as a .keyword field, and you would then use "movies.genre.keyword" to query on the not analyzed version of the data.¨
For details on bool queries you can have a look at the documentation on the ES website:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html
For nested queries:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-nested-query.html
I want to load my tweets from the Cloudant DB by ascending order. I thought using sort: "tweet.id" would work but no.
msg.payload = {
"query": "*:*",
limit: 6,
sort: "tweet.id",
};
return msg;
Node-RED flow:
I got this to work by creating a new Cloudant Query index in the Cloudant dashboard:
{
"index": {
"fields": [ "tweet.timestamp_ms" ]
},
"type": "json"
}
to index the tweet.timestamp_ms field. This can then be queried to return the data in timestamp order:
{
"selector": {
"_id": {
"$gt": 0
}
},
"sort": [
{
"tweet.timestamp_ms": "asc"
}
]
}
I solve this issue by adding the type of the variable.
What I was trying to do is to get all the documents with id=1 and then sort them by the attribute "nombre" which is a string.
My search index is:
function (doc) {
index("id", doc.id, {"store": true});
index("nombre", doc.nombre, {"store": true});
}
And the payload in Node-red:
I am working on search as you type functionality with angularjs and elastic search.I am passing the $viewValue to factory written in angular and it fetches data from angular.Please check code below.
services.factory('instantSearch',['$q', 'esFactory', '$location', function($q, elasticsearch, $location){
return{
instantResult : function(term){
var client = elasticsearch({
// host: $location.host() + ':9200'
host: 'localhost:9200'
});
var deferred = $q.defer();
client.search({
"index": 'stocks',
"type": 'stock',
"body": {
"from" : 0, "size" : 20,
"query": {
"bool":{
"should":[
{
"match_phrase":{
"name": term
}
},
{
"match_phrase":{
"symbol": term
}
},
{
"match":{
"industry": term
}
}
]
}
}
}
}).then(function(result) {
var hits = result.hits.hits;
deferred.resolve(hits);
},
function (err) {
console.trace(err.message);
}, deferred.reject);
return deferred.promise;
}
};
}]);
This code is working fine but the problem is that I get result when input matches complete term in elasticsearch index's field.So I want to implement token analyzer which will match token(ngram - 1,2,3) and provide result on typing of each character.
So to add analyzer code we have to add settings in te elasticserach index as below:
"settings": {
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "edge_ngram",
"min_gram": 1,
"max_gram": 20
}
},
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"autocomplete_filter"
]
}
}
}
}
But I am not getting the way to pass the argument here.Every example I checked shows output with curl command.How can we mix analyzer with the working code above.
Thanks for help.
Have you added the analyzer to the fields name, symbol and industry in your elastic search mapping?
curl -XPUT 'http://localhost:9200/index/type/_mapping?ignore_conflicts=true' -d'
{
"type": {
"properties": {
"name": {
"type": "string",
"analyzer": "autocomplete"
}
}
}
}'
Use ignore_conflicts=true without fail.
If you still face issues, then you might have to create a new index, add analyzer and filter to setting, create the desired mapping and then upload the data again.
I try get highlights from my search but the result does not have highlights.
I´m using this plugin: http://book.cakephp.org/3.0/en/elasticsearch.html#searching-indexed-documents
My code:
public function pesquisaIndice($searchText = null){
$this->loadModel('BooksI', 'Elastic');
$query = $this->BooksI->find()->limit(100)->highlight(['fields'=> ['number_of_fragments'=>20]]);
$query->where(function ($builder) {
return $builder->query(
[
'query_string' =>
[
'query'=> ' melhor',
'default_operator' =>'and'
],
]
);
});
var_dump($query);
}
in command line i can... but with this plugin i can´t
curl -XGET "http://localhost:9200/my_apps_index/_search?pretty=true" -d '{
"query": {
"query_string": {
"query": "melhor"
}
},
"highlight": {
"pre_tags": ["<b>"],
"post_tags": ["</b>"],
"fields": {
"content": {"number_of_fragments": 20}
}
}
}'