Using ES and AngularJS to make a small search app. I'm trying to understand how to implement the scan and scroll feature in ES to use for pagination.
The docs say to make a search request and then to include a 'search_type: scan' and 'scroll' parameters.
Do I just add those parameters to my current search request or do I need to make another search request and specify request as the scan and scroll search request?
As documentation explain you need to make the first call using
GET /old_index/_search?search_type=scan&scroll=1m
{
"query": { "match_all": {}},
"size": 1000
}
The response to this request doesn’t include any hits (means content you search) but scroll id like following
{
"_scroll_id" : "c2Nhbjs1OzMwNTYzMTkxNjpTSkM2S0cxVFJIeUk1NnZWbGFUV1FnOzMwNTYzMTkxNTpTSkM2S0cxVFJIeUk1NnZWbGFUV1FnOzMwNTYyMjEwNzp4OEkwZE54eVR0cXI4cHAzU2I5UmlBOzMwNTYzNTE0NjpQZEhCSUZXeFJZU3daaDJKZXZCRmh3OzMwNTY0OTg4OTphdEE1OTN2NFFsYVY5ZjJ4SUxuVFpROzE7dG90YWxfaGl0czoyOTIwOw==",
"took" : 10,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 2920,
"max_score" : 0,
"hits" : []
}
}
After this you can use normal call with the scroll_id
GET /_search/scroll?scroll=1m&_scroll_idc2Nhbjs1OzMwNTYzMTkxNjpTSkM2S0cxVFJIeUk1NnZWbGFUV1FnOzMwNTYzMTkxNTpTSkM2S0cxVFJIeUk1NnZWbGFUV1FnOzMwNTYyMjEwNzp4OEkwZE54eVR0cXI4cHAzU2I5UmlBOzMwNTYzNTE0NjpQZEhCSUZXeFJZU3daaDJKZXZCRmh3OzMwNTY0OTg4OTphdEE1OTN2NFFsYVY5ZjJ4SUxuVFpROzE7dG90YWxfaGl0czoyOTIwOw==
_scroll_id can be in request or body
Related
I have the following document:
{
"_id" : ObjectId("5a202aa0728bac010a8d2467"),
"tickers" : {
"information_technology" : [
"ACN",
"ATVI",
"ADBE",
"AMD",
],
"misc" : [
"AA",
"GE",
"AAPL",
"PFE",
]
},
"name" : "S&P500"
}
I want to query the document by name ("S&P500") and return a list within the "tickers" field.
I tried db.collection.find_one("$and": [{'name': 'S&P500'}, {'tickers': 'misc'}]) but no documents were returned.
I am new to mongodb so I may have missed something in the documentation.
Thanks for any help.
The API for Collection.find_one is similar with Collection.find except that limit is ignored and a single document returned for a match or None if there is no match.
find(filter=None, projection=None, skip=0, limit=0, no_cursor_timeout=False, cursor_type=CursorType.NON_TAILABLE, sort=None, allow_partial_results=False, oplog_replay=False, modifiers=None, manipulate=True) Docs
An appropriate filter is {'name': 'S&P500'} when looking to match documents with name equals S&P500.
Also, an appropriate projection is {'tickers.misc': True} when projecting only tickers.misc.
db.collection.find_one({'name': 'S&P500'}, {'tickers.misc': True})
I am new to MongoDB and would like to use Meteor Templates to display the queried data in a table.
I have a collection named "infoData" with this structure:
First Document:
{
"_id" : "A-89273498720",
"myItems" : [
{
"itemId" : "item_1",
"username" : "Homer",
"purpose" : "justBecause",
},
{
"itemId" : "item_2",
"username" : "March",
"purpose" : "justBecause2",
},
{
"itemId" : "item_3",
"username" : "Maggie",
"purpose" : "justBecause3",
}
]
}
Second Document:
{
"_id" : "B-564548461117",
"myItems" : [
{
"itemId" : "item_4",
"username" : "Lisa",
"purpose" : "justBecause4",
},
{
"itemId" : "item_5",
"username" : "Lisa",
"purpose" : "justBecause5",
},
{
"itemId" : "item_6",
"username" : "Bart",
"purpose" : "justBecause5",
}
]
}
Now I need to retrieve "itemId" as well as "username" and "purpose" with the "itemId" as a query operator. The "itemId" is unique. My first problem is to get the data. For example I tried this to get the single "itemId" field "item_2":
infoData.findOne({"myItems.itemId": "item_2"}, {_id: 0, 'myItems.$': 1})
which gets the same result as
infoData.findOne(
{
'myItems.itemId': "item_2"
},
{
'_id': 0, 'myItems': {$elemMatch: {'itemId': "item_2"}}
})
I am unsure if this is the result I need, because when I put this query in JSON.stringify() to see the data in the console I see all fields and objects within the array of the doc which contains "item_2" and not only the data "item_2" of the field "itemId". A possibility to get all fields "itemId", "username" and "purpose" of the query (only the object containing item_2) so that I can iterate over it later in a table would also work for me.
The second issue is that I need to display the data in a table.
So I wrote this helper:
'itemInfoDisplay': function() {
if (Meteor.userId()) {
var itemInfos= infoData.findOne(
{"myItems.itemId": "item_2"}, {_id: 0, 'myItems.$': 1});
return itemInfos
}
}
and want to show the data in a table:
{{#each itemInfoDisplay}}
{{#each myItems}}
<tr>
<td><h4>{{ itemId }}</h4></td>
</tr>
{{/each}}
{{/each}}
I know there is something wrong with the code and also with the HTML template. I guess that the result of the mongoDB query is not an array? Is it possible to make it to one? Is the used mongo query the correct solution for my needs?
At the end I just need to get all data which is assigned to a specific "itemId" and display it in a table. I would appreciate anything that helps.
I can think of two ways to do this. The first is to use a projection as mentioned in this post: Retrieve only the queried element in an object array in MongoDB collection
However, it appears you're using that $elemMatch format. If it's not working, not sure if you want to try find() rather than findOne(). There may be a difference in the way Meteor is sending the query to mongodb.
If that doesn't work, the second way is a little bit of a hack but will work. We take advantage of the fact that findOne (unlike find) is synchronous on the client. So we can get the record, then manually do a forEach to only get the appropriate array elements and return that array. Something like this:
var itemInfos= infoData.findOne({"myItems.itemId": "item_2"}, {_id: 0, 'myItems.$': 1});
var items = [];
itemInfos.myItems.forEach(function(myItem) {
if (myItem.itemId == "item_2") {
items.push(myItem);
};
});
return items;
If you use lodash, you can skip the forEach loop and use filter:
return( _.filter(itemInfos.myItems, {itemId: "item_2"}) );
The ideal way would be to restrict the data at the query level, but if that doesn't work option 2 should.
I want to show only the time on the view, when use $firebaseArray I get the list of items and i want to get only the time.
Firebase tree
{
"eventmodel" : {
"-KX6kDkufxLg-fLocsN7" : {
"info" : "ok",
"limit" : 2,
"timeCurrentStart" : "29/11/2016 17:38",
},
"-KXBB-R7xAPl65xhlTpg" : {
"info" : "ok",
"limit" : 2,
"timeCurrentStart" : "29/11/2016 17:38",
}
}
}
The Firebase client always retrieves complete nodes. There is no way to get only the timeCurrentStart property of each node.
So either you'll have to store the timeCurrentStart properties in a separate node:
{
"eventmodeltimes" : {
"-KX6kDkufxLg-fLocsN7" : "29/11/2016 17:38",
"-KXBB-R7xAPl65xhlTpg" : "29/11/2016 17:38"
}
}
Or you'll have to retrieve the entire nodes and only show timeCurrentStart:
<span>{{event.timeCurrentStart}}</span>
The document structure is as follows:
{
"_id" : "V001-99999999",
"vendor_number" : "V001",
"created_time" : ISODate("2016-04-26T22:15:34Z"),
"updated_time" : ISODate("2016-06-07T21:45:46.413Z"),
"items" : [
{
"sku" : "99999999-1",
"status" : "ACTIVE",
"listing_status" : "LIVE",
"inventory" : 10,
"created_time" : ISODate("2016-05-14T22:15:34Z"),
"updated_time" : ISODate("2016-05-14T20:42:21.753Z"),
},
{
"sku" : "99999999-2",
"status" : "INACTIVE",
"listing_status" : "LIVE",
"inventory" : 10,
"created_time" : ISODate("2016-04-26T22:15:34Z"),
"updated_time" : ISODate("2016-06-06T20:42:21.753Z"),
}
]
}
I want to obtain the sku from the item, the conditions are:
1) "vendor_number" = "XXX"
2) items.status = "ACTIVE" AND items.updated_time < [given_date]
Result example:
"sku" : "99999999-2"
or csv:
"sku","99999999-2"
Thank you for your support.
This should be what you want. Although I'm assuming you wanted "status": "active"?
db.getCollection('collection').aggregate([
{ $match: { "vendor_number": "XXXX" } },
{ $project: {
"items": {
$filter: {
input: "$items",
as: "item",
cond: { $eq: ["$$item.status", "ACTIVE"] } // or maybe ["$$item.listing_status", "LIVE"] ?
}
}
}
},
{ $project: { "items.sku": true } }
])
I love using aggregation to manipulate stuff. It's great all the things you can do with it. So here's what's going on:
The first part is simple. The $match step in the aggregation pipeline says just give me documents where vendor_number is "XXXX".
The next part is a bit hairy. The first projection step creates a new field, called "items", I could have called it "results" or "bob" if I wanted to. The $filter specifies which items should go into this new field. The new "items" field will be an array that will have all the results from the previous items field, hence the input: "$items", where you're using the keyword "item" to represent each input item that comes into the filter. Next, the condition says, for each item, only put it in my new "items" array if the item's status is "ACTIVE". You can change it to ["$$items.listing_status", "LIVE"] if that's what you needed. All of this will pretty much give you you're result.
The last project just get's rid of all other fields except for items.sku in each element in the new "items" array.
Hope this help. Play around with it and see what else you can do with the collection and aggregation. Let me know if you need any more clarification. If you haven't used aggregation before, take a look at the aggregation docs and the list of pipeline operators you can use with aggregation. Pretty handy tool.
I've got a filter so I can search for a list of products. The catch is, the list o products is nested (I've got an object which is called category, which contains an object with a series of objects called subcategories, which in turn contains an object called products that contains a series of products. These are the ones I want to filter on). My code is as follows (I'm using Jade btw):
accordion-group(ng-repeat="category in categories", is-open="category.active")
...
tab(ng-repeat="subcat in category.subcategories", active="subcat.active")
...
input(type="text", ng-model="searchText")
tr(ng-repeat="prod in subcat.products|filter:searchText")
...
I've used Angular's filter similarly before and it worked fine (not nested within ng-repeats though) and I can't see what I'm missing.
Any ideas?
EDIT: Adding subcat.products structure
"46ad-97d9-bd3cfe" : {
"name" : "Meat",
"id" : "46ad-97d9-bd3cfe",
"editing" : false,
"active" : true,
"products" : {
"45ed-a686-e45e52" : {
"name" : "Sirloin Steak",
"price" : 10,
"sku" : "MT001",
"id" : "45ed-a686-e45e52",
"editing" : false
},
...
}
},
Many thanks