I am having trouble understanding why an index is not able to cover a certain query, when my interpretation of documentation suggests it should... :)
The document I am referring to is: https://docs.mongodb.com/manual/core/index-multikey/
I am creating an index on a property which is part of an array of objects. The value indexed is present in other documents. The query looks up directly for the value of the property in the array. But when I look at the plan in the profiler, it is looking through the entire collection.
The structure of the document is as follows:
{
"userEmail": "string",
"basicInformation": {
"name" : "string"
},
"events": {
"live" : [
{"eventId": "id of event 1", // <--- field indexed : "events.live.eventId"
"date" : "date of event",
"duration": n},
{"eventId": "id of event 2",
"date" : "date of event",
"duration": n},
...
],
"onDemand" : [
{"eventId": "id of event 1", // <--- field indexed : "events.onDemand.eventId"
"date" : "date of event",
"duration": n},
{"eventId": "id of event 2",
"date" : "date of event",
"duration": n},
...
]
}
QUERY:
{
$facets: {
"liveUsers": [
{$match: {"events.live.eventId": "id of event 1"}},
{ $project: { .... }}
],
"onDemandUsers": [
{$match: {"events.live.eventId": "id of event 1"}},
{ $project: { .... }}
]
}
}
}
The plan does not seem to use the index and scans the collection. Currently the number of documents in the collection is over 63K, which leads to alerts. Can you help me understand how the indexes should be built or query restructured, so that we can avoid the full collection scan.
Related
I have a particular field in my document which has a multilevel nested array structure. The document looks like something this
{
"_id" : ObjectId("62171b4207476091a17f595f"),
"data" : [
{
"id" : "1",
"content" : [
{
"id" : "1.1",
"content" : []
},
{
"id" : "1.2",
"content" : [
{
"id" : "1.2.1",
"content" : [
{
"id" : "1.2.1.1",
"content" : []
}
]
},
{
"id" : "1.2.2",
"content" : []
}
]
}
]
}
]
}
(The ids in my actual data is a random string, I have added a more defined id here just for readability)
In my application code the nesting level is controlled so it won't go more than 5 levels deep.
I need to project a particular object inside one of the deeply nested arrays.
I have all the information needed to traverse the nested structure. For example if I need to fetch the object with id "1.2.2" my input will look something like this:
[{id: 1, index: 0}, {id: 1.2, index: 1}, {id: 1.2.2, index: 1}]
In the above array, each element represents one level of nesting. I have the Id and the index. So in the above example, I know I first need to travel to index 0 at the top level, then inside that to index 1 , then inside that to index 1 again to find my object.
Is there a way I can only get the inner object that I want directly using a query. Or will I need to get the whole "data" field and do the traversal in my application code. I have been unable to figure out any way to construct a query that would satisfy my need.
Query
if you know the path, you can do it using a series of nested
$getField
$arrayElemAt
you can do it in one stage with nested calls, or with many new fields like i did bellow, or with mongodb variables
*i am not sure what output you need, this goes inside to get the 2 using the indexes (if this is not what you need add if you can the expected output)
Test code here
Data
[
{
"_id": ObjectId( "62171b4207476091a17f595f"),
"data": [
{
"id": "1",
"content": [
{
"id": "1.1",
"content": []
},
{
"id": "1.2",
"content": [
{
"id": "1.2.1",
"content": [
{
"id": "1.2.1.1",
"content": []
}
]
},
{
"id": "1.2.2",
"content": [1,2]
}
]
}
]
}
]
}
]
Query
aggregate(
[{"$set":
{"c1":
{"$getField":
{"field":"content", "input":{"$arrayElemAt":["$data", 0]}}}}},
{"$set":
{"c2":
{"$getField":
{"field":"content", "input":{"$arrayElemAt":["$c1", 1]}}}}},
{"$set":
{"c3":
{"$getField":
{"field":"content", "input":{"$arrayElemAt":["$c2", 1]}}}}},
{"$project":{"_id":0, "c4":{"$arrayElemAt":["$c3", 1]}}}])
Results
[{
"c4": 2
}]
I have a collection named "devices" with roughly 50,000 documents. I'm trying to query the "routes" array within each document and have it return the document if multiple conditions are met for the individual array elements. The problem is it seems Mongo is giving back answers where the multiple conditions are satisfied for different array elements.
Sample Data:
{
"_id": 0,
"name": "example1",
"serial": "123456",
"routes": [
{
"description": "8989",
"zone": "front"
},
{
"description": "1221",
"zone": "back"
}
]
},
{
"_id": 1,
"name": "example2",
"serial": "987654",
"routes": [
{
"description": "1515",
"zone": "front"
},
{
"description": "8989",
"zone": "side"
}
]
}
I've tried simple .find() variations with no luck including
db.devices.find({"routes.description":"8989", "routes.zone":"front"})
db.devices.find({"$and": [{"routes.description":"8989"}, {"routes.zone":"front"}]})
I've also tried aggregations which seems to fail on me since my understanding of them is elementary. The desired results for the queries above would be a single document ("_id":0) and not both documents.
{ "_id" : 0, "name" : "example1", "serial" : "123456", "routes" : [ { "description" : "8989", "zone" : "front" }, { "description" : "1221", "zone" : "back" } ] }
Additionally, the ability to query the array using the $in operator would be desired. For example, the following query's desired output would be both documents since both of them have routes that match "zone":"front" and "descriptions" that are in the list.
db.devices.find({"$and": [{"routes.description": { $in: ["8989", "1515"] }}, {"routes.zone":"front"}]})
You simply need to use $elemMatch here
db.devices.find({routes: {$elemMatch: {description:"8989", zone:"front"}}})
Example
I have this kind of structure into a Mongo collection :
{
"_id": "12345678",
"Invoices": [
{
"_id": "123456789",
"Currency": "EUR",
"DueTotalAmountInvoice": 768.3699999999999,
"InvoiceDate": "2016-01-01 00:00:00.000",
"Items": [
{
"Item": 10,
"ProductCode": "ABC567",
"Quantity": 1
},
{
"Item": 20,
"ProductCode": "CDE987",
"Quantity": 1
}
]
},
{
"_id": "87654321",
"Currency": "EUR",
"DueTotalAmountInvoice": 768.3699999999999,
"InvoiceDate": "2016-01-01 00:00:00.000",
"Items": [
{
"Item": 30,
"ProductCode": "PLO987",
"Quantity": 1,
"Units": "KM3"
},
{
"Item": 40,
"ProductCode": "PLS567",
"Quantity": 1,
"DueTotalAmountInvoice": 768.3699999999999
}
]
}
]
}
So I have a first object storing several Invoices and each Invoice is storing several Items. An item is an embedded document.
So in relational modelisation :
A customer has 1 or several Invoice
An Invoice has 1 or several Item
I am facing an issue since I am trying to update a specific Item into a specific a specific Invoice. For example I want to change the quantity of the item 10 in Invoice 123456789.
How is it possible to do that in Mongodb ?
I tried :
Push statement but it doesn't seem to work for nested arrays
arrayFilters but it doesn't seem to work for embedded document in nested arrays (only simple value arrays).
Can you give me some advice about it ?
Thank you !
As per your problem description here:
For example I want to change the quantity of the item 10 in Invoice 123456789. I just changed the Quantity to 3. You can perform any operations here as you want. You just need to take note of how I used arrayFilters here.
Try this query:
db.collection.update(
{"_id" : "12345678"},
{$set:{"Invoices.$[element1].Items.$[element2].Quantity":3}},
{multi:true, arrayFilters:[ {"element1._id": "123456789"},{
"element2.Item": { $eq: 10 }} ]}
)
The above query successfully executed from mongo shell (Mongo 3.6.3). And I see this result:
/* 1 */
{
"_id" : "12345678",
"Invoices" : [
{
"_id" : "123456789",
"Currency" : "EUR",
"DueTotalAmountInvoice" : 768.37,
"InvoiceDate" : "2016-01-01 00:00:00.000",
"Items" : [
{
"Item" : 10,
"ProductCode" : "ABC567",
"Quantity" : 3.0
},
{
"Item" : 20,
"ProductCode" : "CDE987",
"Quantity" : 1
}
]
},
{
"_id" : "87654321",
"Currency" : "EUR",
"DueTotalAmountInvoice" : 768.37,
"InvoiceDate" : "2016-01-01 00:00:00.000",
"Items" : [
{
"Item" : 30,
"ProductCode" : "PLO987",
"Quantity" : 1,
"Units" : "KM3"
},
{
"Item" : 40,
"ProductCode" : "PLS567",
"Quantity" : 1,
"DueTotalAmountInvoice" : 768.37
}
]
}
]
}
Is that what you wanted?
Mongo Db has a way to get the specific array element by using its index. For example, you have an array and you need to get [your] index, then in mongo we use dot . but not braces [ ] !! And one thing is important either! - If you are getting the embedded value (in object or array) you must use " " for your way so if you are changing your value inside this must be like that:
yourModel.findOneAndUpdate(
{ _id: "12345678" },
{
$set: {
"Invoices.0.Items.0.Quantity": 10,
},
}
);
0 - is your element indexes in the array!
$set is the operator to set new value
10 - new value
Else you can go further, you can construct your way to the value with the variable indexes. Use string template
yourModel.findOneAndUpdate(
{ _id: "12345678" },
{
$set: {
[`Invoices.${invoiceIndex}.Items.${itemIndex}.Quantity`]:newValue ,
},
}
);
it is the same but you can paste variable indexes
Let's say I have a collection of documents in the following format:
{
// some fields
"name" : "some name",
"specs" : [
{
"key" : {
"en" : "English key name",
"xx" : "Other key name",
},
"value" : {
"en" : "English value",
"xx" : "Other value",
}
},
{
"key2" : {
"en" : "English key name2",
"xx" : "Other key name2",
},
"value2" : {
"en" : "English value2",
"xx" : "Other value2",
}
},
//and some more sub-documents
],
}
I'm trying to query it from the database to get it in the following format:
{
"name" : "some name",
"specs" : [
{
"key" : "English key name",
"value" : "English value",
},
{
"key2" : "English key name2",
"value2" : "English value2",
},
//and some more sub-documents
],
}
How can it be done, if it is possible at all?
Background
I'm making a software which must be available in multiple languages, and I think current document schema is most suitable for this (if you've got better ideas for the schema I'd like to see them).
To minimize amount of data queried from the database, I'm trying to select the data only in one language. And moreover I want to minimize nesting of structures in the code, so I'm searching a way to somehow select a value out from a sub-document and replace the sub-document.
I've tried a lot of ways writing such query. Here's the one, but it doesn't work as I expect it to:
db.software.aggregate({
$project : {
"name" : true,
"specs" : {
"key" : "$specs.key.en",
"value" : "$specs.value.en"
}
}
});
It transforms a key into an array of all "key.en" fields within specs field. May there be a way to reference a current array element inside "specs" instead of the whole specs array?
Consider the following document:
{
"entity_id" : 10,
"features" :
[
{
"10" : "Test System 2"
},
{
"20" : "System 2 Description"
},
{
"30" : ["Free", "Monthly", "Quaterly"]
},
{
"40" : ["Day", "Swing"]
}
],
}
I need to, in as few statements as possible, to achieve the following:
Given a document like so:
{"feature_id" : "30", "value" : ["Free"]}
get the corresponding element of the array "features" to contain ["Free"] instead of ["Free", "Monthly", "Quaterly"]
Given a document like so:
{"feature_id" : "50", "value" : ["Bonds", "Commodities"]}
create a new element of the array "features" looking like
{"50" : ["Bonds", "Commodities"]}
Given a document like so:
{"feature_id" : "40", "value" : ""}
remove the corresponding element from the array "features".
Data model
Your data model isn't easy to work with given your desired updates.
If you want to use an array, I would suggest changing the document structure to look like:
{
"entity_id" : 10,
"features" : [
{
feature_id: "10",
value : "Test System 2"
},
{
feature_id: "20",
value: "System 2 Description"
},
{
feature_id: "30",
value: ["Free", "Monthly", "Quaterly"]
},
{
feature_id: "40",
value: ["Day", "Swing"]
}
],
}
Alternatively, you could model as an embedded document:
{
"entity_id" : 10,
"features" : {
"10" : "Test System 2",
"20" : "System 2 Description",
"30" : ["Free", "Monthly", "Quaterly"],
"40" : ["Day", "Swing"]
}
}
The benefit of modeling as an array is that you can add a multikey index across all features/values.
If you model as an embedded document, you could reference fields directly (i.e. features.10). This assumes you know what the keys are going to be, and you would have to index each feature value separately.
I'll assume the first format for the examples below. Also note that your key values have to match in type (so string "10" will not match number 10).
Example 1
Given a document like so:
{"feature_id" : "30", "value" : ["Free"]}
get the corresponding element of the array "features" to contain ["Free"] instead of ["Free", "Monthly", "Quaterly"]
Sample update:
db.docs.update(
// Criteria (assumes entity_id is unique)
{
entity_id: 10,
features: {
// Using $elemMatch to find feature_id with string "30"
$elemMatch: { feature_id: "30" },
}
},
// Update
{ $set: {
"features.$.value" : ["Free"]
}}
)
Example 2
Given a document like so:
{"feature_id" : "50", "value" : ["Bonds", "Commodities"]}
create a new element of the array "features" looking like
{"50" : ["Bonds", "Commodities"]}
Sample update:
db.docs.update(
// Criteria (assumes entity_id is unique)
{
entity_id: 10,
},
// Update
{ $push: {
"features" : { "feature_id" : "50", value: ["Bonds", "Commodities"] }
}}
)
Example 3
Given a document like so:
{"feature_id" : "40", "value" : ""}
remove the corresponding element from the array "features".
Sample update:
db.docs.update(
// Criteria (assumes entity_id is unique)
{
entity_id: 10,
},
// Update
{ $pull: {
"features" : { "feature_id" : "40" }
}}
)