Using Elasticsearch I am trying to combine a nested array of objects into one array.
This is what my data looks like:
GET invoices/_search
{
"hits": [
{
"_index": "invoices",
"_id": "1234",
"_score": 1.0,
"_source": {
"id": 1234,
"status": "unpaid",
"total": 15.35,
"payments": [
{
"id": 1981,
"amount": 10,
"date": "2022-02-09T13:00:00+01:00"
},
{
"id": 1982,
"amount": 5.35,
"date": "2022-02-09T13:35:00+01:00"
}
]
}
},
# ... More hits
]
}
I want to only get the payments array of each hit combined into one array, so that it returns something like this:
{
"payments": [
{
"id": 1981,
"amount": 10,
"date": "2022-02-09T13:00:00+01:00"
},
{
"id": 1982,
"amount": 5.35,
"date": "2022-02-09T13:35:00+01:00"
},
{
"id": 5658,
"amount": 3,
"date": "2021-12-19T13:00:00+01:00"
}
]
}
I tried to get this result using nested queries but could not figure it out, the query I used is:
# Query I used:
GET invoices/_search
{
"_source": ["payments"],
"query": {
"nested": {
"path": "payments",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "payments.id"
}
}
]
}
}
}
}
}
# Result:
{
"hits": [
{
"_index": "invoices",
"_id": "545960",
"_score": 1.0,
"_source": {
"payments": [
{
"date": "2022-01-22T15:38:15+01:00",
"amount": 374.5,
"id": 320320
},
{
"date": "2022-01-22T15:30:03+01:00",
"amount": 160.5,
"id": 320316
}
]
}
},
{
"_index": "invoices",
"_id": "545961",
"_score": 1.0,
"_source": {
"payments": [
{
"date": "2022-01-22T15:38:15+01:00",
"amount": 12,
"id": 320350
},
{
"date": "2022-01-22T15:30:03+01:00",
"amount": 60.65,
"id": 320379
}
]
}
}
]
}
The result returns only the payments array but divided over multiple hits. How can I combine those arrays?
In this json object, I want to add and change the values of the list called "m" while traversing the array called "j". What should I do ??
{
"a": "string",
"h": false,
"i": "string",
"j": [
{
"k": 0,
"l": "NONE",
"m": [
{
"n": "string",
"o": []
}
],
"p": [],
"q": [],
}
],
"r": [
{
"s": "string",
"t": "strin",
"u": "string",
"v": null,
}
]
}
In JavaScript, you'd do something like this:
let obj = JSON.parse(json_data);
for(let key in obj.j){
if(key=='m'){
//if you want to skip 'm'
continue;
}
let value = obj.j[key];
//do stuff
}
This is the way to traverse and modify the array and its elements.
let response={
"a": "string",
"h": false,
"i": "string",
"j": [
{
"k": 0,
"l": "NONE",
"m": [
{
"n": "string",
"o": []
}
],
"p": [],
"q": [],
}
],
"r": [
{
"s": "string",
"t": "strin",
"u": "string",
"v": null,
}
]
}
let newObject=response.j.map(ele=>
{ele.m.map(el=>{el.n="IamChanged",el.myName="IamChanged"})}
)
console.log(response);
What do you want in output??
My document collection has an attribute cfname2. Using JSON API I can query *:* and facet on this attribute and exclude 00 term:
GET http://localhost:5555/solr/db/query HTTP/1.1
content-type: application/json
{
"query" : "*:*",
"limit": 0,
"facet": {
"t" : {
"type": "terms",
"field": "cfname2",
"limit": 10,
"sort": "index",
"domain": {"excludeTags": "00"}
}
}
}
independent whether I use excludeTags or not, 00 is in the answer:
"response": {
"numFound": 20560849,
"start": 0,
"numFoundExact": true,
"docs": []
},
"facets": {
"count": 20560849,
"t": {
"buckets": [
{
"val": "0",
"count": 831411
},
{
"val": "00",
"count": 861685
},
{
"val": "01",
"count": 198584
},
I wonder why this does not work. How can I exclude terms matching regex [0-9][0-9] using the above JSON?
I am using angularjs for frontend and loopback for backend and elastic search for the database.
I have a model with properties as:
"name": {
"type": "string",
"required": true
},
"mobileNumber": {
"type": "string",
"required": true
},
"email": {
"type": "string"
},
"message": {
"type": "string",
"required": true
},
"quantity": {
"type": "number",
"required": true
},
"price": {
"type": "number",
"required": true
},
"status": {
"type": "string",
"required": true,
"default": "open"
}
},
data as:
{
"_index": "XXXXXX",
"_type": "XXXXX",
"_id": "XXXXXXX",
"_version": 1,
"_score": 1,
"_source": {
"name": "aadil kirana",
"email": "aadil#gmail.com",
"message": "dfgfb dgfggf",
"quantity": 3434,
"price": 5454,
"status": "open",
"createdAt": "2017-12-19T14:53:41.727Z",
"updatedAt": "2017-12-19T14:53:41.727Z"
}
}
Status could be open, processing, close, reject and failure.
All I want is to get the data in the order where I can see all the open status data ordered by createdAt date,
then all the prcoessing status data ordered by createdAt dat
and so on....
I tried using loopback filters as:
filter = {
order: ['status ASC','createdAt DESC'],
};
but this gives me First all the close status data ordered by date, then all the open status data ordered by date and so on, that status ordered alphabetically.
Please help me to get the desired result.
You can add a new property to your data as statusOrder and define
1 -> open
2 -> close
...
and order by statusOrder instead of status when you are ordering status.
All I want is to get the data in the order where I can see all the
open status data ordered by createdAt date, then all the prcoessing
status data ordered by createdAt dat and so on....
A workaround for this could be to let Elasticsearch do the sort with custom order e.g. in this context the status could be ordered as open followed by processing followed by close followed by reject followed by failure. It can be done with Function Score Query. Some more insights could also be found here
Sample input data for bulk insert:
POST custom/sort/_bulk?pretty
{"index" : {"_index" : "custom"}}
{"status": "open", "createdAt": "2017-12-19T14:53:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "open", "createdAt": "2017-12-18T14:53:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "processing", "createdAt": "2017-12-19T14:53:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "processing", "createdAt": "2017-12-17T14:53:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "close", "createdAt": "2017-12-19T14:53:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "close", "createdAt": "2017-12-19T15:53:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "failure", "createdAt": "2017-12-19T10:53:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "failure", "createdAt": "2017-12-19T14:59:41.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "reject", "createdAt": "2017-12-19T14:53:40.727Z"}
{"index" : {"_index" : "custom"}}
{"status": "reject", "createdAt": "2017-12-19T14:53:41.727Z"}
Sample response from elastic search (without custom order):
Query:
GET custom/sort/_search?filter_path=took,hits.total,hits.hits._score,hits.hits._source
{
"took": 0,
"hits": {
"total": 10,
"hits": [
{
"_score": 1,
"_source": {
"status": "processing",
"createdAt": "2017-12-19T14:53:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "close",
"createdAt": "2017-12-19T14:53:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "reject",
"createdAt": "2017-12-19T14:53:40.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "open",
"createdAt": "2017-12-18T14:53:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "failure",
"createdAt": "2017-12-19T10:53:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "failure",
"createdAt": "2017-12-19T14:59:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "reject",
"createdAt": "2017-12-19T14:53:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "open",
"createdAt": "2017-12-19T14:53:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "processing",
"createdAt": "2017-12-17T14:53:41.727Z"
}
},
{
"_score": 1,
"_source": {
"status": "close",
"createdAt": "2017-12-19T15:53:41.727Z"
}
}
]
}
}
Query to mimic custom ordering :
GET custom/sort/_search?filter_path=took,hits.hits._id,hits.hits._score,hits.hits._source,hits.hits.sort
{
"query": {
"function_score": {
"boost_mode": "replace",
"query": {
"constant_score": {
"filter": {
"terms": {
"status.keyword": [
"open",
"processing",
"close",
"reject",
"failure"
]
}
}
}
},
"functions": [
{
"filter": {
"term": {
"status.keyword": "open"
}
},
"weight": 4
},
{
"filter": {
"term": {
"status.keyword": "processing"
}
},
"weight": 3
},
{
"filter": {
"term": {
"status.keyword": "close"
}
},
"weight": 2
},
{
"filter": {
"term": {
"status.keyword": "reject"
}
},
"weight": 1
},
{
"filter": {
"term": {
"status.keyword": "failure"
}
},
"weight": 0
}
]
}
},
"sort": [
{
"_score": {
"order": "desc"
},
"createdAt": {
"order": "asc"
}
}
]
}
Output (with custom order):
{
"took": 4,
"hits": {
"hits": [
{
"_id": "grOucmABwtSchlgLKlaV",
"_score": 4,
"_source": {
"status": "open",
"createdAt": "2017-12-18T14:53:41.727Z"
},
"sort": [
4,
1513608821727
]
},
{
"_id": "gbOucmABwtSchlgLKlaV",
"_score": 4,
"_source": {
"status": "open",
"createdAt": "2017-12-19T14:53:41.727Z"
},
"sort": [
4,
1513695221727
]
},
{
"_id": "hLOucmABwtSchlgLKlaV",
"_score": 3,
"_source": {
"status": "processing",
"createdAt": "2017-12-17T14:53:41.727Z"
},
"sort": [
3,
1513522421727
]
},
{
"_id": "g7OucmABwtSchlgLKlaV",
"_score": 3,
"_source": {
"status": "processing",
"createdAt": "2017-12-19T14:53:41.727Z"
},
"sort": [
3,
1513695221727
]
},
{
"_id": "hbOucmABwtSchlgLKlaV",
"_score": 2,
"_source": {
"status": "close",
"createdAt": "2017-12-19T14:53:41.727Z"
},
"sort": [
2,
1513695221727
]
},
{
"_id": "hrOucmABwtSchlgLKlaV",
"_score": 2,
"_source": {
"status": "close",
"createdAt": "2017-12-19T15:53:41.727Z"
},
"sort": [
2,
1513698821727
]
},
{
"_id": "ibOucmABwtSchlgLKlaV",
"_score": 1,
"_source": {
"status": "reject",
"createdAt": "2017-12-19T14:53:40.727Z"
},
"sort": [
1,
1513695220727
]
},
{
"_id": "irOucmABwtSchlgLKlaV",
"_score": 1,
"_source": {
"status": "reject",
"createdAt": "2017-12-19T14:53:41.727Z"
},
"sort": [
1,
1513695221727
]
},
{
"_id": "h7OucmABwtSchlgLKlaV",
"_score": 0,
"_source": {
"status": "failure",
"createdAt": "2017-12-19T10:53:41.727Z"
},
"sort": [
0,
1513680821727
]
},
{
"_id": "iLOucmABwtSchlgLKlaV",
"_score": 0,
"_source": {
"status": "failure",
"createdAt": "2017-12-19T14:59:41.727Z"
},
"sort": [
0,
1513695581727
]
}
]
}
}
I'm currently working on a search using elasticsearch. We have a very large amount of users.
Here is the elasticsearch mapping:
PUT /example_index/_mapping/users
{
"properties": {
"user_autocomplete": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
},
"completion": {
"type": "text",
"analyzer": "user_autocomplete_analyzer",
"search_analyzer": "standard"
}
}
},
"firstName": {
"type": "text"
},
"lastName": {
"type": "text"
}
}
}
Here is the search query.
For example, I get 3 records
GET example_index/users/_search
{
"from": 0,
"size": 3,
"query": {
"query_string": {
"query": "*ro*",
"fields": [
"firstName",
"lastName"
]
}
},
"aggs": {
"user_suggestions": {
"terms": {
"size": 3,
"field": "user_autocomplete.raw"
}
}
}
}
Here is the output of elasticsearch
{
"took": 53,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 13,
"max_score": 1,
"hits": [
{
"_index": "example_index",
"_type": "users",
"_id": "08",
"_score": 1,
"_source": {
"firstName": "Eero",
"lastName": "Saarinen",
"user_autocomplete": "Eero Saarinen"
}
},
{
"_index": "example_index",
"_type": "users",
"_id": "16",
"_score": 1,
"_source": {
"firstName": "Aaron",
"lastName": "Judge",
"user_autocomplete": "Aaron Judge"
}
},
{
"_index": "example_index",
"_type": "users",
"_id": "20",
"_score": 1,
"_source": {
"firstName": "Robert",
"lastName": "Langdon",
"user_autocomplete": "Robert Langdon"
}
}
]
},
"aggregations": {
"user_suggestions": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 10,
"buckets": [
{
"key": "Eero Saarinen",
"doc_count": 1
},
{
"key": "Aaron Judge",
"doc_count": 1
},
{
"key": "Robert Langdon",
"doc_count": 1
}
]
}
}
}
I need result like in the following order:
Robert Langdon
Aaron Judge
Eero Saarinen
I have tried order method. It won't work. Is there a way to do this?