I try get highlights from my search but the result does not have highlights.
I´m using this plugin: http://book.cakephp.org/3.0/en/elasticsearch.html#searching-indexed-documents
My code:
public function pesquisaIndice($searchText = null){
$this->loadModel('BooksI', 'Elastic');
$query = $this->BooksI->find()->limit(100)->highlight(['fields'=> ['number_of_fragments'=>20]]);
$query->where(function ($builder) {
return $builder->query(
[
'query_string' =>
[
'query'=> ' melhor',
'default_operator' =>'and'
],
]
);
});
var_dump($query);
}
in command line i can... but with this plugin i can´t
curl -XGET "http://localhost:9200/my_apps_index/_search?pretty=true" -d '{
"query": {
"query_string": {
"query": "melhor"
}
},
"highlight": {
"pre_tags": ["<b>"],
"post_tags": ["</b>"],
"fields": {
"content": {"number_of_fragments": 20}
}
}
}'
Related
I have the following set of data in my MongoDB collection called orders:
{
"_id" : ObjectId("618e0e1b17687316dcdd6246"),
"groupUID": "abc",
"orderData" : {
"charges" : {
"total" : 18480.0,
"subtotal" : 13980.0
},
"items" : [
{
"name" : "Chocolate cookies",
"imageURL": "domainURL2.com/cookies"
},
{
"name" : "Chocolate muffins",
"imageURL": "domainURL2.com/muffins"
}
]
}
}
Now I want to update the imageURL substring part of "domainURL2" to "domainURL1" field in every document of this collection. I have the following query so far:
db.orders.update(
{
"groupUID" : "abc"
},
{ "$set": { "orderData.items.$.imageURL":"myURL.com" } } )
I also have the query in JavaScript form but I want this to be in pure Mongo query. So the query below will not do it for me unfortunately.
db.getCollection("orders").find({"groupUID" : "abc"}).forEach(function(aRow) {
if (aRow.orderDetails !== undefined) {
var updated = false;
aRow.orderData.items.forEach(function(item) {
item.imageURL = item.imageURL.replace("eddress/", "noknok-app/");
})
db.getCollection("orders").save(aRow);
}
});
I want to update all records' imageURL field's substring part. I am unable to figure out the rest of the query. Can anyone please help me?
My answer may look complex (Welcome for suggestion/improvement).
Work the update with Aggegration Pipeline.
$set - Update orderData.items field.
1.1. $map - Iterate orderData.items and returns new array.
1.1.1. $mergeObjects - Merge current object and imageURL field from 1.1.1.1.
1.1.1.1. $cond - With $regexMatch to find the imageURL starts with "domainURL2.com".
1.1.1.2. If true, then replace "domainURL2.com" with "domainURL1.com".
1.1.1.3. If false, remain existing value.
db.collection.update({
"groupUID": "abc"
},
[
{
"$set": {
"orderData.items": {
$map: {
input: "$orderData.items",
in: {
$mergeObjects: [
"$$this",
{
imageURL: {
$cond: {
if: {
$regexMatch: {
input: "$$this.imageURL",
regex: "^domainURL2.com"
}
},
then: {
$concat: [
"domainURL1.com",
{
$arrayElemAt: [
{
$split: [
"$$this.imageURL",
"domainURL2.com"
]
},
-1
]
}
]
},
else: "$$this.imageURL"
}
}
}
]
}
}
}
}
}
])
Sample Mongo Playground
Another approach is using $replaceOne (suggested by #rickhg12hs) which will be much easier.
$replaceOne to replace for 1.1.1.1.
db.collection.update({
"groupUID": "abc"
},
[
{
"$set": {
"orderData.items": {
$map: {
input: "$orderData.items",
in: {
$mergeObjects: [
"$$this",
{
imageURL: {
$replaceOne: {
input: "$$this.imageURL",
find: "domainURL2.com",
replacement: "domainURL1.com"
}
}
}
]
}
}
}
}
}
])
Sample Mongo Playground ($replaceOne)
How do I query and replace the value for SMT_PORT_3306_TCP_ADDR.
I tried
echo $task_definition | jq -r '.taskDefinition.containerDefinitions[0].environment[] | select(.name=="SMT_PORT_3306_TCP_ADDR")| .value = "myvalue" '
the output I get
{
"name": "SMT_PORT_3306_TCP_ADDR",
"value": "myvalue"
}
I do not get the full json
Input Json :
{
"taskDefinition": {
"taskDefinitionArn": "some value",
"containerDefinitions": [
{
"name": "common-api-img",
"environment": [
{
"name": "SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST",
"value": "false"
},
{
"name": "SMT_PORT_3306_TCP_ADDR",
"value": "valueToReplace"
}
],
"mountPoints": [],
"volumesFrom": []
}
],
"revision": 65,
"volumes": [],
"status": "ACTIVE"
}
}
Expected output without the top level taskDefinition value:
{
"taskDefinitionArn":"some value",
"containerDefinitions":[
{
"name":"common-api-img",
"environment":[
{
"name":"SERVER_API_TIMEOUT_SUBSCRIPTIONS_CANCEL_REQUEST",
"value":"false"
},
{
"name":"SMT_PORT_3306_TCP_ADDR",
"value":"myvalue"
}
],
"mountPoints":[
],
"volumesFrom":[
]
}
],
"revision":65,
"volumes":[
],
"status":"ACTIVE"
}
Use |= with if.
jq '.taskDefinition.containerDefinitions[0].environment[]
|= if .name == "SMT_PORT_3306_TCP_ADDR"
then .value = "myvalue"
else .
end'
I am new in groovy and I want to construct a json object with the builder
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{ "match": { "content": "scontent" } },
{ "match": { "title":"stitle" } }
]
}
},
{
"bool": {
"should": [
{ "match": { "a1": "v1" } },
{ "match": { "a2":"v2" } },
... and so on ...
{ "match": { "an":"vn" } }
]
}
}
]
}
},
"highlight": {
"fields": {
"content":{}
}
}
}
I search a lot of on other posts on stackoverflow and I write this code
So I did this but no way to get what I want :
JsonBuilder builder = new JsonBuilder()
def body = builder {
from Lib.or(qQuery.start, 0)
size Lib.or(qQuery.num, 10)
query {
bool {
must [
{
bool {
should [
{ match { content 'scontent' } },
{ match { title 'stitle' } }
]
}
},
{
bool {
should myVals.collect {[
'match' : { it.key it.value }
]}
}
}
]
}
}
highlight {
fields {
content {}
}
}
}
Thanks for any help !
I think you can make this work with the JsonBuilder as is, but it is usually easier to create the data structure using maps and lists (which is what the builder outputs) in groovy as you have more control there.
Example code:
import groovy.json.*
def data = [
query: [
bool: [
must: [
[bool:
[should: [
[match: [ content: 'scontent']],
[match: [ title: 'stitle']]
]]
],
[bool:
[should: [
[match: [ a1: 'v1']],
[match: [ a2: 'v2']],
[match: [ vn: 'vn']]
]]
]
]
]
]
]
println JsonOutput.prettyPrint(JsonOutput.toJson(data))
produces:
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"match": {
"content": "scontent"
}
},
{
"match": {
"title": "stitle"
}
}
]
}
},
{
"bool": {
"should": [
{
"match": {
"a1": "v1"
}
},
{
"match": {
"a2": "v2"
}
},
{
"match": {
"vn": "vn"
}
}
]
}
}
]
}
}
}
I did not include your full json as it takes up some space, but the structure is there. Note the use of lists ([valueA, valueB]) vs maps ([someKey: someValue]) in the data structure.
Granted this makes the JsonBuilder less than 100% useful but I haven't seen any concise ways of including lists of large json objects in a list within the structure. You can do:
def json = JsonBuilder()
json.query {
bool('list', 'of', 'values')
}
but for larger structures as list elements I would say go with the lists and maps approach.
I am working on search as you type functionality with angularjs and elastic search.I am passing the $viewValue to factory written in angular and it fetches data from angular.Please check code below.
services.factory('instantSearch',['$q', 'esFactory', '$location', function($q, elasticsearch, $location){
return{
instantResult : function(term){
var client = elasticsearch({
// host: $location.host() + ':9200'
host: 'localhost:9200'
});
var deferred = $q.defer();
client.search({
"index": 'stocks',
"type": 'stock',
"body": {
"from" : 0, "size" : 20,
"query": {
"bool":{
"should":[
{
"match_phrase":{
"name": term
}
},
{
"match_phrase":{
"symbol": term
}
},
{
"match":{
"industry": term
}
}
]
}
}
}
}).then(function(result) {
var hits = result.hits.hits;
deferred.resolve(hits);
},
function (err) {
console.trace(err.message);
}, deferred.reject);
return deferred.promise;
}
};
}]);
This code is working fine but the problem is that I get result when input matches complete term in elasticsearch index's field.So I want to implement token analyzer which will match token(ngram - 1,2,3) and provide result on typing of each character.
So to add analyzer code we have to add settings in te elasticserach index as below:
"settings": {
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "edge_ngram",
"min_gram": 1,
"max_gram": 20
}
},
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"autocomplete_filter"
]
}
}
}
}
But I am not getting the way to pass the argument here.Every example I checked shows output with curl command.How can we mix analyzer with the working code above.
Thanks for help.
Have you added the analyzer to the fields name, symbol and industry in your elastic search mapping?
curl -XPUT 'http://localhost:9200/index/type/_mapping?ignore_conflicts=true' -d'
{
"type": {
"properties": {
"name": {
"type": "string",
"analyzer": "autocomplete"
}
}
}
}'
Use ignore_conflicts=true without fail.
If you still face issues, then you might have to create a new index, add analyzer and filter to setting, create the desired mapping and then upload the data again.
I have a very particular issue concerning querying over a boolean field and a string field which are nested to an array field. The index mapping is as follow:
indexes :string_field_1, type: 'string'
indexes :string_field_2, type: 'string'
indexes :boolean_field_1, type: 'boolean'
indexes :array_field_1 do
indexes :boolean_field_2, type: 'boolean'
indexes :string_field_3, type: 'string'
end
indexes :array_field_2 do
indexes :integer_field_1, type: 'integer'
end
indexes :array_field_3 do
indexes :integer_field_2, type: 'integer'
end
The document index also has many other fields which are not nested to the array field, but have to be included among the query fields.
I have tried an approach using filter and bool queries that is as follow:
"query":
{"bool":
{"must":
[
{"query_string":
{"query":"text which is being searched",
"fields":[
"string_field_1",
"string_field_2",
"array_field_1.string_field_3"
],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"}
}
],
"filter":[
{"bool":
{"must":
[
{"bool":
{"should":
[
{"term":{"boolean_field_1":false}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}]}},
{"bool":
{"should":
[
{"term":{"array_field_1.boolean_field_2":true}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}]}},
]
}
}
]
}
}
]
}
}
The problem with this query is that it is returning a document which, in my opinion, doesn't have to be returned.
The document, in this case, is the bellow:
_source": {
"string_field_1": "text 1",
"string_field_2": "text 2",
"boolean_field_1": false,
"array_field_1": [
{
"boolean_field_2": true,
"string_field_3": "some text which is not being searched"
},
{
"boolean_field_2": true,
"string_field_3": "some text which is not being searched"
},
{
"boolean_field_2": false,
"string_field_3": "text which is being searched"
},
{
"boolean_field_2": true,
"string_field_3": "some text which is not being searched"
}
],
"array_field_2": [
{
"integer_field_1": A
}
],
"array_field_3": [
{
"integer_field_2": B
}
]
}
As you can notice, the third item of array_field_1 contains boolean_field_2: false and also the text which is being searched. But, according to my filter: clause, only the documents which array_field_1.boolean_field_2 is true have to be retrieved, unless array_field_2.integer_field_1: or array_field_3.integer_field_1 occurs, which is not true, according to my query part.
It seems elastic is not considering that the array_field_1[2] is the one that the boolean_field_2 is false.
How can I make my query so that this document isn't retrieved?
Thanks is advance,
Guilherme
That was my solution:
"query":{
"bool":{
"should":
[
{
"query_string":
{
"query":"text which is being searched",
"fields":
[
"string_field_1",
"string_field_2"
],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"
}
},
{
bool: {
should:[
{
query:{
nested: {
path: 'array_field_1',
query: {
bool: {
must: [
{ match: { "array_field_1.string_field_3": "text which is being searched"} },
{term: {"array_field_1.boolean_field_2": true}}
]
}
}
}
}
},
{
bool:
{
must: [
{
query:{
nested: {
path: 'movimentos',
query: {
bool: {
must: [
{ match: { "array_field_1.string_field_3": "text which is being searched"} },
{term: {"array_field_1.boolean_field_2": false
]
}
}
}
}
},
{
query: {
bool: {
should: [
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}
]
}
}
}
]
}
}
]
}
}
]
}
}
Another approach consists of putting the array_field_1.string_field_3 query together with the bool query related to the boolean field:
"query":{
"bool":{
"should":
[
{
"query_string":
{
"query":"text which is being searched",
"fields":
[
"string_field_1",
"string_field_2"
],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"
}
},
{
"bool":{
"must":
[
{
"query_string":
{
"query":"text which is being searched",
"fields":["array_field_1.string_field_3"],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"
}
},
{
"bool":{
"should":
[
{"term":{"array_field_1.boolean_field_2":true}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}
]
}
}
]
}
}
],
"filter":
[
{
"bool":{
"should":
[
{"term":{"boolean_field_1":false}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}
]
}
}
]
}
}
This query also retrieves the document, unfortunately. I really do not know how to build this query properly.
The query above is organized as:
(X) OR (A AND (B OR C OR D))