MongoDB: $pull / $unset with multiple conditions - arrays

Question

Example Document:
{
_id: 42,
foo: {
bar: [1, 2, 3, 3, 4, 5, 5]
}
}
The query:
I'd like to "remove all entries from foo.bar that are $lt: 4 and the first matching entry that matches $eq: 5". Important: The $eq part must only remove a single entry!
I have a working solution, that uses 3 update queries, but that's too much for that simple task. Nevertheless, here's what I did so far:
1. Find the first entry matching $eq: 5 and $unset it. (As you know: $unset doesn't remove it. It just sets it to null):
update(
{ 'foo.bar': 5 },
{ $unset: { 'foo.bar.$': 1 } }
)
2. $pull all entries $eq: null, so that former 5 is really gone:
update(
{},
{ $pull: { 'foo.bar': null } }
)
3. $pull all entries $lt: 4:
update(
{},
{ $pull: { 'foo.bar': { $lt: 4 } } }
)
Resulting Document:
{
_id: 42,
foo: {
bar: [4, 5]
}
}
Ideas and Thoughts:
Extend query 1., so that it will $unset the entries $lt: 4 and one entry $eq: 5. Afterwards we can execute query 2. and there's no need for query 3..
Extend query 2. to $pull everything that matches $or: [{$lt: 4}, {$eq: 5}]. Then there's no need for query 3..
Extend query 2. to $pull everything that is $not: { $gte: 4 }. This expression should match $lt: 4 and $eq: null.
I already tried to implement those queries, but sometimes it complained about the query syntax and sometimes the query did execute and just removed nothing.
Would be nice, if someone has a working solution for this.

Answer 1

Not sure if I get your full meaning of this, but to "bulk" update documents you can always take this approach in addition the oringal $pull and adding some "detection" of which documents you need to remove "duplicate" 5 from:
// Remove less than four first
db.collection.update({},{ "$pull": { "foo.bar": { "$lt": 4 } } },{ "multi": true });
// Initialize Bulk
var bulk = db.collection.initializeOrderdBulkOp(),
count = 0;
// Detect and cycle documents with duplicate five to be removed
db.collection.aggregate([
// Project a "reduced" array and calculate if the same size as orig
{ "$project": {
"foo.bar": { "$setUnion": [ "$foo.bar", [] ] },
"same": { "$eq": [
{ "$size": { "$setUnion": [ "$foo.bar", [] ] } },
{ "$size": "$foo.bar" }
] }
}},
// Filter the results that were unchanged
{ "$match": { "same": true } }
]).forEach(function(doc) {
bulk.find({ "_id": doc._id })
.updateOne({ "$set": { "foo.bar": doc.foo.bar.sort() } });
count++;
// Execute per 1000 processed and re-init
if ( count % 1000 == 0 ) {
bulk.execute();
bulk = db.collection.initializeOrderdBulkOp();
}
});
// Clean up any batched
if ( count % 1000 != 0 )
bulk.execute();
That trims out anything less than "4" and all duplicates where a "duplicate" is detected from the difference in "set" length.
If you just want values of 5 removed as duplicates you can take a similar logic approach to the detection and modification, just not with "set operators" that remove anything that is a "duplicate" making it a valid "set".
At any rate, some detection strategy is going to be better than iterating updates until "all but one" value is gone.
Of course you can simplify your statements a little and remove one update operation, it's not pretty because $pull does not allow an $or condition in a query, but I hope you get the idea if this applies:
db.collection.update(
{ "foo.bar": 5 },
{ "$unset": { "foo.bar.$": 1 } },
{ "multi": true }
); // same approach
// So include all the values "less than four"
db.collection.update(
{ "foo.bar": { "$in": [1,2,3,null] } },
{ "$pull": { "foo.bar": { "$in": [1,2,3,null] } }},
{ "multi": true }
);
It's a bit less processing but of course those need to be exact integer values. Otherwise stick with the three updates you are doing. Better than cycling in code.
For reference, the "nicer" syntax that will unfortunately not work would be something like this:
db.collection.update(
{
"$or": [
{ "foo.bar": { "$lt": 4 } },
{ "foo.bar": null }
]
},
{
"$pull": {
"$or": [
{ "foo.bar": { "$lt": 4 } },
{ "foo.bar": null }
]
}
},
{ "multi": true }
);
Probably worth a JIRA issue, but I suspect mostly because the array element is not the "first" argument directly following $pull.

Answer 2

You can use the Array.prototype.filter() and the Array.prototype.splice() methods
The filter() method creates a news array with foo.bar values $lt: 4 then you use the splice method to remove those values and the first value equal 5 from foo.bar
var idx = [];
db.collection.find().forEach(function(doc){
idx = doc.foo.bar.filter(function(el){
return el < 4;
});
for(var i in idx){
doc.foo.bar.splice(doc.foo.bar.indexOf(idx[i]), 1);
}
doc.foo.bar.splice(doc.foo.bar.indexOf(5), 1);
db.collection.save(doc);
} )