How to get subdocument element's count inside an array and how to update the subdocument's key in MongoDB
For eg, following is the whole doc stored in mongodb:
{
"CompanyCode" : "SNBN",
"EventCode" : "ET00008352",
"EventName" : "Sunburn Presents Avicii India Tour",
"TktDetail" : [
{
"Type" : "Category I",
"Qty" : {
"10-Dec" : {
"value" : 58
},
"11-Dec" : {
"value" : 83
},
"12-Dec" : {
"value" : 100
}
}
},
{
"Type" : "Category II",
"Qty" : {
"10-Dec" : {
"value" : 4
},
"11-Dec" : {
"value" : 7
},
"12-Dec" : {
"value" : 8
}
}
},
{
"Type" : "PRICE LEVEL 1",
"Qty" : {
"11-Dec" : {
"value" : 2
}
}
},
{
"Type" : "CatIV",
"Qty" : {
"18-Dec" : {
"value" : 20
}
}
}
],
"TransDate" : [
"10-Dec-2013",
"11-Dec-2013",
"12-Dec-2013",
],
"VenueCode" : "SNBN",
"VenueName" : "Sunburn",
"_id" : ObjectId("52452db273b92012c41ad612")
}
Here TktDetail is an array, inside which there is a Qty subdoc which contains multiple elements, I want to know how to get the elements count inside Qty per index?
For example, the 0th index of TktDetail array contains 1 Qty subdoc, which further has a element count of 3, whereas 3rd index has element count of 1 in Qty subdoc.
If I want to update the subdoc key, like, I want to update the date in Qty from "10-Dec" to "10-Dec-2013", how is it possible?
Thanks in advance, looking for a reply ASAP..
So the first thing here is that you actually asked two questions, being "how do I get a count of the items under Qty?" and "how can I change the names?". Now while normally unrelated I'm going to treat them as the same thing.
What you need to do is change your schema and in doing so I'm going to allow you to get the count of items and I'm going to encourage you to change those field names as well. Specifically you need a schema like this:
"TktDetail" : [
{
"Type" : "Category I",
"Qty" : [
{ "date": ISODate("2013-12-10T00:00:00.000Z") , "value" : 58 },
{ "date": ISODate("2013-12-11T00:00:00.000Z"), "value" : 83 },
{ "date": ISODate("2013-12-01T00:00:00.000Z"), "value" : 100 },
]
},
All the gory details are in my answer here to a similar question. But the problem basically is that when you use sub-documents in the way you have you are ruining your chances of doing any meaningful query operations on this, as to get at each element you must specify the full path to get there.
That answer has more detail, but the case is you really want an array. The trade-off, it's a little harder to update, especially considering you have nested arrays, but it's a lot easier to add and much easier to query.
Also, and related, change your dates to be dates and not strings. The strings, are no good for comparisons inside MongoDB. With them set as proper BSON dates (noting I clipped them to the start of day) you can compare, and query ranges and do useful things. Your application code will be happy to as the driver will return a real date object, rather than something you have to manipulate "both ways".
So once you have read through, understood and implemented this, on to counting:
db.collection.aggregate([
// Unwind the TktDetail array to de-normalize
{"$unwind": "$TktDetail"},
// Also Unwind the Qty array
{"$unwind": "$Qty" },
// Get some group information and count the entries
{"$group": {
"_id": {
"_id": "$_id,
"EventCode": "$EventCode",
"Type": "$TktDetail.Type"
},
"Qty": {"$sum": 1 }
}},
// Project nicely
{"$project": {
"_id": 0,
"EventCode": "$_id.EventCode",
"Type: "$_id.Type",
"Qty": 1,
}},
// Let's even sort it
{"$sort": { "EventCode": 1, "Qty" -1 }}
])
So that allowed us to get a count of the items in Qty for each EventCode by Type with the Qty ordered higest to lowest.
And that is not possible on your current schema without loading and traversing each document in code.
So there is the case. Now if you want to ignore this and just go about changing the sub-document key names, then you'll need to do remove the key and underlying document and replace with the new key name, using update:
db.collection.update(
{ EventCode: "ET00008352"},
{ $unset:{ "TktDetail.0.Qty.10-Dec": "" }}
)
db.collection.update(
{ EventCode: "ET00008352"},
{ $set:{ "TktDetail.0.Qty.10-Dec-2013": { value: 58 } }}
)
And you'll need to do that for every item that you have.
So you either work out that schema conversion or otherwise have a lot of work anyway in order to change the keys. For myself, I'd do it properly, and only do it once so I didn't run into the next problem later.
Related
I have a document in my mongodb that contains a very large array (about 10k items). I'm trying to only keep the latest 1k in the array (and so remove the first 9k elements). The document looks something like this:
{
"_id" : 'fakeid64',
"Dropper" : [
{
"md5" : "fakemd5-1"
},
{
"md5" : "fakemd5-2"
},
...,
{
"md5": "fakemd5-10000"
}
]
}
How do I accomplish that?
The correct operation to do here actually involves the $push operator using the $each and $slice modifiers. The usage may initially appear counter-intuitive that you would use $push to "remove" items from an array, but the actual use case is clear when you see the intended operation.
db.collection.update(
{ "_id": "fakeid64" },
{ "$push": { "Dropper": { "$each": [], "$slice": -1000 } }
)
You can in fact just run for your whole collection as:
db.collection.update(
{ },
{ "$push": { "Dropper": { "$each": [], "$slice": -1000 } },
{ "multi": true }
)
What happens here is that the modifier for $each takes an array of items to "add" in the $push operation, which in this case we leave empty since we do not actually want to add anything. The $slice modifier given a "negative" value is actually saying to keep the "last n" elements present in the array as the update is performed, which is exactly what you are asking.
The general "intended" case is to use $slice when adding new elements to "maintain" the array at a "maximum" given length, which in this case would be 1000. So you would generally use in tandem with actually "adding" new items like this:
db.collection.update(
{ "_id": "fakeid64" },
{ "$push": { "Dropper": { "$each": [{ "md5": "fakemd5-newEntry"}], "$slice": -1000 } }
)
This would append the new item(s) provided in $each whilst also removing any items from the "start" of the array where the total length given the addition was greater than 1000.
It is stated incorrectly elsewhere that you would use $pullAll with a supplied list of the array content already existing in the document, but the operation is actually two requests to the database.
The misconception being that the request is sent as "one", but it actually is not and is basically interpreted as the longer form ( with correct usage of .slice() ):
var md5s = db.collection.findOne({ "_id": "fakeid64" }).Dropper.slice(-1000);
db.collection.update(
{ "_id": "fakeid64" },
{ "$pullAll": { "Dropper": md5s } }
)
So you can see that this is not very efficient and is in fact quite dangerous when you consider that the state of the array within the document "could" possibly change in between the "read" of the array content and the actual "write" operation on update since they occur separately.
This is why MongoDB has atomic operators for $push with $slice as is demonstrated. Since it is not only more efficient, but also takes into consideration the actual "state" of the document being modified at the time the actual modification occurs.
you can use $pullAll operator
suppose you use python/pymongo driver:
yourcollection.update_one(
{'_id': fakeid64},
{'$pullAll': {'Dropper': yourcollection.find_one({'_id': 'fakeid64'})['Dropper'][:9000]}}
)
or in mongo shell:
db.yourcollection.update(
{ _id: 'fakeid64'},
{$pullAll: {'Dropper': db.yourcollection.findOne({'_id' : 'fakeid64'})['Dropper'].slice(0,9000)}}
)
(*) having saying that it would be much better if you didn't allow your document(s) to grow this much in first place
This is just a representation of query. Basically you can unwind with limit and skip, then use cursor foreach to remove the items like below :
db.your_collection.aggregate([
{ $match : { _id : 'fakeid64' } },
{ $unwind : "$Dropper"},
{ $skip : 1000},
{ $limit : 9000}
]).forEach(function(doc){
db.your_collection.update({ _id : doc._id}, { $pull : { Dropper : doc.Dropper} });
});
from mongo docs
db.students.update(
{ _id: 1 },
{
$push: {
scores: {
$each: [ { attempt: 3, score: 7 }, { attempt: 4, score: 4 } ],
$sort: { score: 1 },
$slice: -3
}
}
}
)
The following update uses the $push operator with:
the $each modifier to append to the array 2 new elements,
the $sort modifier to order the elements by ascending (1) score, and
the $slice modifier to keep the last 3 elements of the ordered array.
So heres my mongodb document:
{
"_id" : "",
"lists" : [
{
"name" : "list 1",
"items" : []
},
{
"name" : "list 2",
"items" : []
}
]
}
How would I go about adding an object inside "items"?
This is the code I have so far, but it doesn't work:
xxx.update(_id, {$push: { "lists.$.items": item}});
Note that I have access to the index (variable called 'index'), so its possible to insert an item at index, 0, 1, 2..., etc.
I tried this before, but it won't work:
xxx.update({_id, "lists": index}, {$push: { "lists.$.items": item}});
I also looked at other similar questions and couldn't find anything. Most of them have some sort of id field in their arrays, but I don't.
What about
xxx.update({_id}, {$push: { "lists.index.items": item}});
Of course this would fail, what I mean is replace index with real index values
xxx.update({_id}, {$push: { "lists.2.items": item}});
You can manipulate the update json based on the index maybe as below.
var update = '{$push: { "lists.'+index+'.items": '+item+'}}';
var updateObj = JSON.parse(update);
xxx.update({_id}, updateObj);
Not sure if it will work as it is or it would need further tweaking, but you get the idea.
I have a mongo document that contains an array called history:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another": 3
}
]
}
I want to add fields to each history object or update fields IF the name AND organisation match, however if they don't, I want to add a new object to the array with the queried name and organisation and add/update the other fields to the object when necessary.
So:
This query, finds one that matches:
db.users.find({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
})
However, I'm struggling to get the update/upsert to work IF that combination of history.name and history.organisation dont exist in the array.
What I think I need to do is a :
"If this history name does not equal 'Test123' AND the history organisation does not equal 'Rat' then add an object to the array with those fields and any other field provided in the update query."
I tried this:
db.users.update({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
}, {
history: { name: "Test123"},
history: { organisation: "Rat"}
}, {upsert:true})
But that gave me E11000 duplicate key error index: db.users.$email_1 dup key: { : null }
Any help greatly appreciated.
Thanks community!
Not possible with a single atomic update I'm afraid, you would have to do a couple of update operations that satisfy both conditions.
Break down the update logic into two distinct update operations, the first one would require using the positional $ operator to identify the element in the history array you want and the $set to update the existing fields. This operation follows the logic update fields IF the name AND organisation match
Now, you'd want to use the findAndModify() method for this operation since it can return the updated document. By default, the returned document does not include the modifications made on the update.
So, armed with this arsenal, you can then probe your second logic in the next operation i.e. update IF that combination of "history.name" and "history.organisation" don't exist in the array. With this second
update operation, you'd need to then use the $push operator to add the elements.
The following example demonstrates the above concept. It initially assumes you have the query part and the document to be updated as separate objects.
Take for instance when we have documents that match the existing history array, it will just do a single update operation, but if the documents do not match, then the findAndModify() method will return null, use this logic in your second update operation to push the document to the array:
var doc = {
"name": "Test123",
"organisation": "Rat"
}, // document to update. Note: the doc here matches the existing array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
After this operation for documents that match, querying the collection will yield the same
db.users.find({ "email": "email#address" });
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
}
]
}
Now consider documents that won't match:
var doc = {
"name": "foo",
"organisation": "bar"
}, // document to update. Note: the doc here does not matches the current array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
Querying this collection for this document
db.users.find({ "email": "email#address" });
would yield
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
},
{
"name" : "foo",
"organisation" : "bar"
}
]
}
I'm trying to get a specific field from a subdocument array
I'm not gonna include any of the fields in the parent doc
Here is the sample document
{
"_id" : ObjectId("5409dd36b71997726532012d"),
"hierarchies" : [
{
"rank" : 1,
"_id" : ObjectId("5409df85b719977265320137"),
"name" : "CTO",
"userId" : [
ObjectId("53a47a639c52c9d83a2d71db")
]
}
]
}
I would like to return the rank of the hierarchy if the a userId is in the userId array
here's what I have so far in my query
collectionName.find({{hierarchies:
{$elemMatch : {userId: ObjectId("53a47a639c52c9d83a2d71db")}}}
, "hierarchies.$.rank", function(err,data){}
so far it returns the entire object in the hierarchies array I want, but I would like to limit it to just the rank property of the object.
The projection available to .find() queries generally in MongoDB does not do this sort of projection for internal elements of an array. All you can generally do is return the "matched" element of the array entirely.
For what you want, you use the aggregation framework instead, which gives you more control over matching and projection:
Model.aggregate([
{ "$match": {
"hierarchies.userId": ObjectId("53a47a639c52c9d83a2d71db")
}},
{ "$unwind": "$hierarchies" },
{ "$match": {
"hierarchies.userId": ObjectId("53a47a639c52c9d83a2d71db")
}},
{ "$project": {
"rank": "$hierarchies.rank"
}}
],function(err,result) {
})
That basically matches the documents, filters the array content of the document to just the match and then projects only the required field.
I have a document in MongoDB as below.
{
"CorePrice" : 1,
"_id" : 166,
"partno" : 76,
"parttype" : "qpnm",
"shipping" :
[
{
"shippingMethod1" : "ground",
"cost1" : "10"
},
{
"shippingMethod2" : "air",
"cost2" : "11"
},
{
"shippingMethod3" : "USPS",
"cost3" : "3"
},
{
"shippingMethod4" : "USPS",
"cost4" : 45
}
]
}
My goal is to add CorePrice (1) to cost4 (45) and retrieve the computed value as a new column "dpv". I tried using the below query. However I receive an error exception: $add only supports numeric or date types, not Array. I'm not sure why. Any kind of help will be greatly appreciated.
db.Parts.aggregate([
{
$project: {
partno: 1,
parttype: 1,
dpv: {$add: ["$CorePrice","$shipping.cost1"]}
}
},
{
$match: {"_id":{$lt:5}}
}
]);
When you refer to the field shipping.cost1 and shipping is an array, MongoDB does not know which entry of the shipping-array you are referring to. In your case there is only one entry in the array with a field cost1, but this can't be guaranteed. That's why you get an error.
When you are able to change your database schema, I would recommend you to turn shipping into an object with a field for each shipping-type. This would allow you to address these better. When this is impossible or would break some other use-case, you could try to access the array entry by numeric index (shipping.0.cost1).
Another thing you could try is to use the $sum-operator to create the sum of all shipping.cost1 fields. When there is only one element in the array with a field cost1, the result will be its value.
I am able to achieve this by divorcing the query into two as below.
var pipeline1 = [
{
"$unwind": "$shipping"
},
{
$project:{
partno:1,
parttype:1,
dpv:{
$add:["$CorePrice","$shipping.cost4"]
}
}
},
{
$match:{"_id":5}
}
];
R = db.tb.aggregate( pipeline );