Suppose I have a collection in mongoDB like given below -
{
name : "Abhishek",
Roll_no : null,
hobby : stackoverflow
},
{
name : null,
Roll_no : 1,
hobby : null
}
Now I want to delete the fields in my Documents where the field values are null. I know that I can do it using the $unset in following way -
db.collection.updateMany({name: null}, { $unset : { name : 1 }});
And we could do it in the same way for hobby and name field.
But I was wondering if I can do the same deletion operation using just one query? I was wondering if maybe I could use $or or something else to achieve the same effect but in a single command.
Any ideas?
On MongoDB version >= 3.2 :
You can take advantage of .bulkWrite() :
let bulkArr = [
{
updateMany: {
filter: { name: null },
update: { $unset: { name: 1 } }
}
},
{
updateMany: {
filter: { Roll_no: null },
update: { $unset: { Roll_no: 1 } }
}
},
{
updateMany: {
filter: { hobby: null },
update: { $unset: { hobby: 1 } }
}
},
];
/** All filter conditions will be executed on all docs
* but respective update operation will only be executed if respective filter matches (kind of individual ops) */
db.collection.bulkWrite(bulkArr);
Ref : bulkwrite
On MongoDB version >= 4.2 :
Since you wanted to delete multiple fields(where field names can't be listed down or unknown) having null value, try below query :
db.collection.update(
{}, // Try to use a filter if possible
[
/**
* using project as first stage in aggregation-pipeline
* Iterate on keys/fields of document & remove fields where their value is 'null'
*/
{
$project: {
doc: {
$arrayToObject: { $filter: { input: { $objectToArray: "$$ROOT" }, cond: { $ne: ["$$this.v", null] } } }
}
}
},
/** Replace 'doc' object as root of document */
{
$replaceRoot: { newRoot: "$doc" }
}
],
{ multi: true }
);
Test : mongoplayground
Ref : update-with-an-aggregation-pipeline , aggregation-pipeline
Note :
I believe this would be one time operation & in future you can use Joi npm package or mongoose schema validators to restrict writing null's as field values. If you can list down your field names as if not too many plus dataset size is way too high then try to use aggregation with $$REMOVE as suggested by '#thammada'.
As of now, aggregation-pipeline in .updateMany() is not supported by many clients even few mongo shell versions - back then my ticket to them got resolved by using .update(), if it doesn't work then try to use update + { multi : true }.
With MongoDB v4.2, you can do Updates with Aggregation Pipeline, along with the $$REMOVE system variable
db.collection.updateMany({
$or: [{
name: null
}, {
Roll_no: null
}, {
hobby: null
}]
}, [{
$set: {
name: { $ifNull: ["$name", "$$REMOVE"] }
Roll_no: { $ifNull: ["$Roll_no", "$$REMOVE"] },
hobby: { $ifNull: ["$hobby", "$$REMOVE"] }
}
}]
Related
> db.Dashboards.find({user:{$regex:"adams"}}).pretty();
{
"_id" : ObjectId("123"),
"user" : "adam.adams",
"widgets" : [
{
"_id" : ObjectId("124"),
"column" : 1,
"configMap" : {
"coleg" : "bas.baser,cer.ceras,tom.tomsa"
},
}
]
}
I have a Mongo database that keeps records like the one above, unfortunately I need to find all users who have "cer.ceras" in the "coleg" field and then replace this with "per.peras"
I try with
db.Dashboards.find({widgets:{"$elemMatch":{configMap:{"$elemMatch":{coleg:/.*ceras.*/}}}}}}).pretty();
But I'm not finding anything for me
This may a bit complex.
Filter:
The criteria should work with $regex to find the occurrence of the text.
Update:
Require the update with aggregation pipeline.
$set - Set widgets field.
1.1. $map - Iterate the element in the widgets array and return a new array.
1.1.1. $mergeObjects - Merge current iterate document with the result of 1.1.1.1.
1.1.1.1. A document with configMap array. With $mergeObjects to merge current iterate configMap document with the result 1.1.1.1.1.
1.1.1.1.1. A document with coleg field. With $replaceAll to replace all matching text "cer.ceras" to "per.peras".
Update Options
{ multi: true } aims to update multiple documents.
db.collection.update({
widgets: {
$elemMatch: {
"configMap.coleg": {
$regex: "cer\\.ceras"
}
}
}
},
[
{
$set: {
widgets: {
$map: {
input: "$widgets",
in: {
$mergeObjects: [
"$$this",
{
configMap: {
$mergeObjects: [
"$$this.configMap",
{
coleg: {
$replaceAll: {
input: "$$this.configMap.coleg",
find: "cer.ceras",
replacement: "per.peras"
}
}
}
]
}
}
]
}
}
}
}
}
],
{
multi: true
})
Demo # Mongo Playground
I'm struggling to write a Mongo UpdateMany statement that can reference and update an object within an array.
Here I create 3 documents. Each document has an array called innerArray always containing a single object, with a single date field.
use test;
db.innerArrayExample.insertOne({ _id: 1, "innerArray": [ { "originalDateTime" : ISODate("2022-01-01T01:01:01Z") } ]});
db.innerArrayExample.insertOne({ _id: 2, "innerArray": [ { "originalDateTime" : ISODate("2022-01-02T01:01:01Z") } ]});
db.innerArrayExample.insertOne({ _id: 3, "innerArray": [ { "originalDateTime" : ISODate("2022-01-03T01:01:01Z") } ]});
I want to add a new date field, based on the original date field, to end up with this:
{ _id: 1, "innerArray": [ { "originalDateTime" : ISODate("2022-01-01T01:01:01Z"), "copiedDateTime" : ISODate("2022-01-01T12:01:01Z") } ]}
{ _id: 2, "innerArray": [ { "originalDateTime" : ISODate("2022-01-02T01:01:01Z"), "copiedDateTime" : ISODate("2022-01-02T12:01:01Z") } ]}
{ _id: 3, "innerArray": [ { "originalDateTime" : ISODate("2022-01-03T01:01:01Z"), "copiedDateTime" : ISODate("2022-01-03T12:01:01Z") } ]}
In pseudo code I am saying take the originalDateTime, run it through a function and add a related copiedDateTime value.
For my specific use-case, the function I want to run strips the timezone from originalDateTime, then overwrites it with a new one, equivalent to the Java ZonedDateTime function withZoneSameLocal. Aka 9pm UTC becomes 9pm Brussels (therefore effectively 7pm UTC). The technical justification and methodology were answered in another Stack Overflow question here.
The part of the query I'm struggling with, is the part that updates/selects data from an element inside an array. In my simplistic example, for example I have crafted this query, but unfortunately it doesn't work:
This function puts copiedDateTime in the correct place... but doesn't evaluate the commands to manipulate the date:
db.innerArrayExample.updateMany({ "innerArray.0.originalDateTime" : { $exists : true }}, { $set: { "innerArray.0.copiedDateTime" : { $dateFromString: { dateString: { $dateToString: { "date" : "$innerArray.0.originalDateTime", format: "%Y-%m-%dT%H:%M:%S.%L" }}, format: "%Y-%m-%dT%H:%M:%S.%L", timezone: "Europe/Paris" }}});
// output
{
_id: 1,
innerArray: [
{
originalDateTime: ISODate("2022-01-01T01:01:01.000Z"),
copiedDateTime: {
'$dateFromString': {
dateString: { '$dateToString': [Object] },
format: '%Y-%m-%dT%H:%M:%S.%L',
timezone: 'Europe/Paris'
}
}
}
]
}
This simplified query, also has the same issue:
b.innerArrayExample.updateMany({ "innerArray.0.originalDateTime" : { $exists : true }}, { $set: { "innerArray.0.copiedDateTime" : "$innerArray.0.originalDateTime" }});
//output
{
_id: 1,
innerArray: [
{
originalDateTime: ISODate("2022-01-01T01:01:01.000Z"),
copiedDateTime: '$innerArray.0.originalDateTime'
}
]
}
As you can see this issue looks to be separate from the other stack overflow question. Instead of being able changing timezones, it's about getting things inside arrays to update.
I plan to take this query, create 70,000 variations of it with different location/timezone combinations and run it against a database with millions of records, so I would prefer something that uses updateMany instead of using Javascript to iterate over each row in the database... unless that's the only viable solution.
I have tried putting $set in square brackets. This changes the way it interprets everything, evaluating the right side, but causing other problems:
test> db.innerArrayExample.updateMany({ "_id" : 1 }, [{ $set: { "innerArray.0.copiedDateTime" : "$innerArray.0.originalDateTime" }}]);
//output
{
_id: 1,
innerArray: [
{
'0': { copiedDateTime: [] },
originalDateTime: ISODate("2022-01-01T01:01:01.000Z")
}
]
}
Above it seems to interpret .0. as a literal rather than an array element. (For my needs I know the array only has 1 item at all times). I'm at a loss finding an example that meets my needs.
I have also tried experimenting with the arrayFilters, documented on my mongo updateMany documentation but I cannot fathom how it works with objects:
test> db.innerArrayExample.updateMany(
... { },
... { $set: { "innerArray.$[element].copiedDateTime" : "$innerArray.$[element].originalDateTime" } },
... { arrayFilters: [ { "originalDateTime": { $exists: true } } ] }
... );
MongoServerError: No array filter found for identifier 'element' in path 'innerArray.$[element].copiedDateTime'
test> db.innerArrayExample.updateMany(
... { },
... { $set: { "innerArray.$[0].copiedDateTime" : "$innerArray.$[element].originalDateTime" } },
... { arrayFilters: [ { "0.originalDateTime": { $exists: true } } ] }
... );
MongoServerError: Error parsing array filter :: caused by :: The top-level field name must be an alphanumeric string beginning with a lowercase letter, found '0'
If someone can help me understand the subtleties of the Mongo syntax and help me back on to the right path I'd be very grateful.
You want to be using pipelined updates, the issue you're having with the syntax you're using is that it does not allow the usage of aggregation operators and document field values.
Here is a quick example on how to do it:
db.collection.updateMany({},
[
{
"$set": {
"innerArray": {
$map: {
input: "$innerArray",
in: {
$mergeObjects: [
"$$this",
{
copiedDateTime: "$$this.originalDateTime"
}
]
}
}
}
}
}
])
Mongo Playground
This question already has answers here:
How to Update Multiple Array Elements in mongodb
(16 answers)
Closed 4 years ago.
I have a document like following in my stocks collection in mongodb.
{ _id: 'xRMuRBhqRLgASyQyW',
History:
[ { orderId: '12032017001877',
status: 'COMPLETED',
},
{ orderId: '22122017002450',
status: 'PROCESSED',
},
{ orderId: '22122018002450',
status: 'DEPOSIT',
}
]
}
I want to iterate through all document in stocks collection and add a field flag: true if status is not 'PROCESSED'.
You need to use the all $[] positional operator to update each element in the array
db.collection.update(
{ "History": { "$elemMatch": { "status": { "$ne": "PROCESSED" } } } },
{ "$set": { "History.$[].flag": false } },
{ "multi": true }
)
You can achieve it using script.
db.stocks.find().forEach(function (e) {
var modified = false;
e.History.forEach(function (o) {
if (o.status != "PROCESSED") {
o.flag = true;
modified = true;
}
});
if (modified) {
db.stocks.save(e);
}
});
You can simply do it by using '$' operator. Something like:
db.stocks.update( {"history.status" : { $not: "PROCESSED" } ,
{ $set: {"history.$.flag": true }} ,
false ,
true);
My answer is very similar #Anthony but
a two extra parameter( for upsert and multi) are added. For reference You can check official document.
db.stocks.update(
{ "History": { "$elemMatch": { "status": { "$ne": "PROCESSED" } } } },
{ "$set": { "History.$[].flag": false }},
false, // show that if there is not any document with specified criteria , it will not create a new document.
ture // according to document of mongodb if multi is true then more than one document will be updated, other wise only one document will be updated.
)
I have a collection of documents that look like this:
{
my_array : [
{
name : "...",
flag : 1 // optional
},
...
]
}
How can I query for documents that exactly one of their elements contain the flag field?
This will return documents where my_array length is exactly 1:
db.col.find({ "my_array" : { $size : 1 } })
This will return documents where at least one of my_array's object contain the flag field:
db.col.find({ "my_array.flag" : { $exists : true } })
And this will look for documents where flag is an array of size one:
db.col.find({ "my_array.flag" : { $size : 1 } })
I want to somehow combine the first two queries, checking for some inner field existence, and then querying for the size of the filtered array
You can try $redact with $filter for your query.
$filter with $ifNull to keep the matching element followed by $size and $redact and compare result with 1 to keep and else remove the document.
db.col.aggregate(
[{
$match: {
"my_array.flag": {
$exists: true
}
}
}, {
$redact: {
$cond: {
if: {
$eq: [{
$size: {
$filter: {
input: "$my_array",
as: "array",
cond: {
$eq: [{
$ifNull: ["$$array.flag", null]
}, "$$array.flag"]
}
}
}
}, 1]
},
then: "$$KEEP",
else: "$$PRUNE"
}
}
}]
)
Use Aggregation framework in mongodb,
pipeline stage 1: Match ,
checking for some inner field existence,
{ "my_array.flag" : { $exists : true } }
pipleline stage 2: Project,
querying for the size of the filtered array,
{$project:{ "my_array":{$size:"$my_array"}}}
mongoshell Command ,
db.col.aggregate([{
$match: {
"my_array.flag": {
$exists: true
}
}
}, {
$project: {
"my_array": {
$size: "$my_array"
}
}
}])
Following the current answers about using aggregate, here's a working solution using aggregation:
db.col.aggregate([
{ $match : { "my_array.flag" : { $exists : true } } },
{ $unwind : "$my_array" },
{ $match : { "my_array.flag" : { $exists : true } } },
{ $group : { _id : "$_id", count_flags : { $sum : 1 } } },
{ $match : { "count_flags" : 1 } }
])
Here are the steps explained:
Filter to only documents that contain at least one flag field (this will use my index to dramatically reduce the number of documents I need to process)
Unwind the my_array array, to be able to process each element as a separate doc
Filter the separated docs from elements not containing flag
Group again by _id, and count the elements (that contain the flag field)
Filter by the new count_flags field
I don't really like this solution, it seems like a lot of steps to get something that should have basic support in a standard mongo find query.
I am working on an express js application where I need to update a nested array.
1) Schema :
//Creating a mongoose schema
var userSchema = mongoose.Schema({
_id: {type: String, required:true},
name: String,
sensors: [{
sensor_name: {type: String, required:true},
measurements: [{time: String}]
}] });
2)
Here is the code snippet and explanation is below:
router.route('/sensors_update/:_id/:sensor_name/')
.post(function (req, res) {
User.findOneAndUpdate({_id:req.body._id}, {$push: {"sensors" :
{"sensor_name" : req.body.sensor_name , "measurements.0.time": req.body.time } } },
{new:true},function(err, newSensor) {
if (err)
res.send(err);
res.send(newSensor)
}); });
I am able to successfully update a value to the measurements array using the findOneAndUpdate with push technique but I'm failing when I try to add multiple measurements to the sensors array.
Here is current json I get if I get when I post a second measurement to the sensors array :
{
"_id": "Manasa",
"name": "Manasa Sub",
"__v": 0,
"sensors": [
{
"sensor_name": "ras",
"_id": "57da0a4bf3884d1fb2234c74",
"measurements": [
{
"time": "8:00"
}
]
},
{
"sensor_name": "ras",
"_id": "57da0a68f3884d1fb2234c75",
"measurements": [
{
"time": "9:00"
}
]
}]}
But the right format I want is posting multiple measurements with the sensors array like this :
Right JSON format would be :
{
"_id" : "Manasa",
"name" : "Manasa Sub",
"sensors" : [
{
"sensor_name" : "ras",
"_id" : ObjectId("57da0a4bf3884d1fb2234c74"),
"measurements" : [
{
"time" : "8:00"
}
],
"measurements" : [
{
"time" : "9:00"
}
]
}],
"__v" : 0 }
Please suggest some ideas regarding this. Thanks in advance.
You might want to rethink your data model. As it is currently, you cannot accomplish what you want. The sensors field refers to an array. In the ideal document format that you have provided, you have a single object inside that array. Then inside that object, you have two fields with the exact same key. In a JSON object, or mongo document in this context, you can't have duplicate keys within the same object.
It's not clear exactly what you're looking for here, but perhaps it would be best to go for something like this:
{
"_id" : "Manasa",
"name" : "Manasa Sub",
"sensors" : [
{
"sensor_name" : "ras",
"_id" : ObjectId("57da0a4bf3884d1fb2234c74"),
"measurements" : [
{
"time" : "8:00"
},
{
"time" : "9:00"
}
]
},
{
// next sensor in the sensors array with similar format
"_id": "",
"name": "",
"measurements": []
}],
}
If this is what you want, then you can try this:
User.findOneAndUpdate(
{ _id:req.body._id "sensors.sensor_name": req.body.sensor_name },
{ $push: { "sensors.0.measurements": { "time": req.body.time } } }
);
And as a side note, if you're only ever going to store a single string in each object in the measurements array, you might want to just store the actual values instead of the whole object { time: "value" }. You might find the data easier to handle this way.
Instead of hardcoding the index of the array it is possible to use identifier and positional operator $.
Example:
User.findOneAndUpdate(
{ _id: "Manasa" },
{ $push: { "sensors.$[outer].measurements": { "time": req.body.time } } }
{ "arrayFilters:" [{"outer._id": ObjectId("57da0a4bf3884d1fb2234c74")}]
);
You may notice than instead of getting a first element of the array I specified which element of the sensors array I would like to update by providing its ObjectId.
Note that arrayFilters are passed as the third argument to the update query as an option.
You could now make "outer._id" dynamic by passing the ObjectId of the sensor like so: {"outer._id": req.body.sensorId}
In general, with the use of identifier, you can get to even deeper nested array elements by following the same procedure and adding more filters.
If there was a third level nesting you could then do something like:
User.findOneAndUpdate(
{ _id: "Manasa" },
{ $push: { "sensors.$[outer].measurements.$[inner].example": { "time": req.body.time } } }
{ "arrayFilters:" [{"outer._id": ObjectId("57da0a4bf3884d1fb2234c74"), {"inner._id": ObjectId("57da0a4bf3884d1fb2234c74"}}]
);
You can find more details here in the answer written by Neil Lunn.
refer ::: positional-all
--- conditions :: { other_conditions, 'array1.array2.field_to_be_checked': 'value' }
--- updateData ::: { $push : { 'array1.$[].array2.$[].array3' : 'value_to_be_pushed' } }