I would like to upsert an element in an array, based on doc _id and element _id. Currently it works only if the element is allready in the array (update works, insert not).
So, these collection:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c"
}
]
}]
Receiving this request:
query = { _id: '5a65fcf363e2a32531ed9f9b', 'ressources._id': '5a65fd0363e2a32531ed9f9c' };
update = { '$set': { 'ressources.$': { '_id': '5a65fd0363e2a32531ed9f9c', qt: '153', unit: 'kg' } } };
options = {upsert:true};
collection.update(query,update,options);
Will give this ok result:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c",
"qt": 153,
"unit": "kg"
}
]
}]
How to make the same request work with these initial collections:
[{
"_id": "5a65fcf363e2a32531ed9f9b"
}]
OR
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": []
}]
How to make the upsert work?
Does upsert works with entire document only?
Currently, I face this error:
The positional operator did not find the match needed from the query.
Thanks
I also tried to figure out how to do it. I found only one way:
fetch model by id
update array manually (via javascript)
save the model
Sad to know that in 2018 you still have to do the stuff like it.
UPDATE:
This will update particular element in viewData array
db.collection.update({
"_id": args._id,
"viewData._id": widgetId
},
{
$set: {
"viewData.$.widgetData": widgetDoc.widgetData
}
})
$push command will add new items
Related
I have a Profile model, that contains this field:
interests: {
type: [String],
},
My app has been running for a while. So this means for several documents, this field has already been filled with an array of strings.
In order to achieve certain goals, I need to create a model Interest with a field name and then refer to it in the Profile like this:
interests: [{
type: Schema.Types.ObjectId,
ref: "interests",
}],
The field name should contain the already existing string interests in Profile.interests.
This is the approach that I think I will follow:
Create Interest model.
Fill name field with the existing Profile.interests strings.
a. When doing this replace Profile.interests with the _ids of the newly created Interest documents.
b. Make sure Interest.name is unique.
c. Remove spaces.
Wherever interests in the app are used in the backend, use populate to fill them.
This doesn't feel like a safe operation. So I would like to hear your thoughts on it. Is there a better approach? Should I avoid doing this?
Thank you.
Step 1:
Create a Model for interests,
specify your desired fields fir interests schema and set properties for particular fields
specify collection name in options as per your requirement
create a model and specify your desired name in model
const InterestsSchema = new mongoose.Schema(
{ name: { type: String } },
{ collection: "interests" }
);
const Interests = mongoose.model("Interests", InterestsSchema);
Instead of removing interests field add new field interest (you can choose desired field), for safe side whenever you feel the current update working properly you can remove it, Update profile schema,
update interest field as per your requirement, now newly added field is interest
interests: {
type: [String]
},
interest: [{
type: Schema.Types.ObjectId,
ref: "interests"
}],
Step 2:
Wherever interests in the app are used in the backend, use interest and populate to fill them.
Step 3: (just execute the query)
Make a collection for interests and store all unique interests string from profile collection, so write a aggregation query to select unique string and store in interests collection, you can execute this query in mongo shell or any editor that you are using after specifying your original profile collection name,
$project to show interests field only because we are going to deconstruct it
$unwind to deconstruct interests array
$group by interests and select unique field, and trim white space from interests string
$project to show name field and if you want to then add your desired fields
$out will create a new collection interests and write all interests with newly generated _id field
db.getCollection('profile').aggregate([
{ $project: { interests: 1 } },
{ $unwind: "$interests" },
{ $group: { _id: { $trim: { input: "$interests" } } } },
{ $project: { _id: 0, name: "$_id" } },
{ $out: "interests" }
])
Playground
You have example input:
[
{
"_id": 1,
"interests": ["sports","sing","read"]
},
{
"_id": 2,
"interests": ["tracking","travel"]
}
]
After executing above query the output/result in interests / new collection would be something like:
[
{
"_id": ObjectId("5a934e000102030405000000"),
"name": "travel"
},
{
"_id": ObjectId("5a934e000102030405000001"),
"name": "sports"
},
{
"_id": ObjectId("5a934e000102030405000002"),
"name": "read"
},
{
"_id": ObjectId("5a934e000102030405000003"),
"name": "tracking"
},
{
"_id": ObjectId("5a934e000102030405000004"),
"name": "sing"
}
]
Step 4: (just execute the query)
Add new field interest with reference _ids from interests collection in profile collection, there are sequence to execute queries,
find profile query and project only required fields _id and interests when interest (new field) field is not exists and iterate loop using forEach
trim interests string iterating loop through map
find the interests reference _id by its name field from created interests collection
update query for add interest field that have _ids in profile collection
db.getCollection('profile').find(
{ interest: { $exists: false } },
{ _id: 1, interests: 1 }).forEach(function(profileDoc) {
// TRIM INTEREST STRING
var interests = profileDoc.interests.map(function(i){
return i.trim();
});
// FIND INTERESTS IDs
var interest = [];
db.getCollection('interests').find(
{ name: { $in: interests } },
{ _id: 1 }).forEach(function(interestDoc){
interest.push(interestDoc._id);
});
// UPDATE IDS IN PROFILE DOC
db.getCollection('profile').updateOne(
{ _id: profileDoc._id },
{ $set: { interest: interest } }
);
});
You have example input:
[
{
"_id": 1,
"interests": ["sports","sing","read"]
},
{
"_id": 2,
"interests": ["tracking","travel"]
}
]
After executing above query the result in your profile collection would be something like:
[
{
"_id": 1,
"interests": ["sports","sing","read"],
"interest": [
ObjectId("5a934e000102030405000001"),
ObjectId("5a934e000102030405000002"),
ObjectId("5a934e000102030405000004")
]
},
{
"_id": 2,
"interests": ["tracking","travel"],
"interest": [
ObjectId("5a934e000102030405000000"),
ObjectId("5a934e000102030405000003")
]
}
]
Step 5:
Now you have completed all the steps and you have newly added interest field
and also old field interests field is still in safe mode, just make sure everything is working properly you can delete old interests field,
remove old field interests field from all profiles
db.getCollection('profile').updateMany(
{ interests: { $exists: true } },
{ $unset: { "interests": 1 } }
);
Playground
Warning:
Test this steps in your local/development server before executing in production server.
Take backup of your database collections before executing queries.
Field and schema names are predicted you can replace with your original name.
I have around 40k records to update and each record gets the data from querying another collection.
I have an existing query to do this, but it runs for more than an hour. It usually disconnects, then I rerun it again.
I think there is a better way to do this, I am just a noob with mongodb and this solution works but I am not happy with the execution speed.
Maybe you have a better or much faster solution.
To better illustrate the data, please see below:
accounts
[
{
"_id": ObjectId("AC101"),
"emails":null,
"name":"Account 101",
...
},
{
"_id": ObjectId("AC102"),
"emails":null,
"name":"Account 102",
...
},
{
"_id": ObjectId("AC103"),
"emails":null,
"name":"Account 103",
...
},
...
]
account_contacts
[
{
"_id": Object("ACC001"),
"account": {
"$ref" : "account",
"$id" : ObjectId("AC101")
},
"email":"acc001#test.com",
"name":"Contact 001",
...
},
{
"_id": Object("ACC002"),
"account": {
"$ref" : "account",
"$id" : ObjectId("AC102")
},
"email":"acc002#test.com",
"name":"Contact 002",
...
},
{
"_id": Object("ACC003"),
"account": {
"$ref" : "account",
"$id" : ObjectId("AC103")
},
"email":"acc003#test.com",
"name":"Contact 003",
...
},
{
"_id": Object("ACC004"),
"account": {
"$ref" : "account",
"$id" : ObjectId("AC103")
},
"email":"acc004#test.com",
"name":"Contact 004",
...
},
{
"_id": Object("ACC005"),
"account": {
"$ref" : "account",
"$id" : ObjectId("AC103")
},
"email":"acc005#test.com",
"name":"Contact 005",
...
},
...
]
Query:
db.getCollection('accounts').find({ 'emails':{ $eq:null } }).forEach(p => {
const emails = [];
db.getCollection('account_contacts').find({"account.$id": p._id}).forEach(c => {
emails.push(c.email);
});
db.getCollection('accounts').updateOne({"_id": p._id}, {$set: {"emails": emails}});
});
I have a filter to get only the accounts with null emails, so that if it gets a timeout error (1hr)... I just rerun the script and it will process those accounts with null emails.
Currently, I do not have any idea on how to improve the query... but I know it is not the best solution for this case since it takes more than an hour.
Update:
While I still cannot make the aggregate/lookUp approach work, I did tried to run the old script in mongo console, which I ran before and executes more than an hour in my ID... If you run it directly in the mongo console, it only takes 12-14 mins which is not bad.
This is what I did for now, but I still want to convert my script to use aggregation.
TIA
Using MongoDB 4.2, you can avoid pulling the documents to the client side if you are willing to use a temporary collection.
Use aggregation to match all of the documents with null email, extract just the _id and store it in a temporary collection. Note that if you have an index on {emails:1, _id:1} it will streamline this part. You may want to procedurally generate the temporary collection name so it doesn't use the same name for successive runs.
db.accounts.aggregate([
{$match: {emails: null}},
{$project: {_id: 1}},
{$out: "temporary_null_email_collection"}
])
Then aggregate the temporary collection, lookup the email from the account_contacts collection, get rid of any extraneous fields, and merge the results back with the accounts collection.
db.temporary_null_email_collection.aggregate([
{$lookup:{
from: "account_contacts",
localField: "_id",
foreignField: "$id", // verify this field name is correct
as: contacts
}},
{$project: {
_id: 1,
emails: "$contacts.emails"
}},
{$merge: "accounts"}
])
I m working with mongodb and restheart.
In my nosql db i have a unique document with this structure:
{
"_id": "docID",
"users": [
{
"userID": "12",
"elements": [
{
"elementID": "1492446877599",
"events": [
{
"event1": "one"
},
{
"event2": "two",
}
]
}
},
{
"userID": "11",
"elements": [
{
"elementID": "14924",
"events": [
{
"event1": "one"
},
{
"event2": "two",
}
]
}
}
]
}
how can i build an url-query in order to get the user with id 11?
Using mongo shell it should be something like this one:
db.getCollection('collection').find({},{'users':{'$elemMatch':{'userID':'12'}}}).pretty()
I cannot find anything similar on restheart.
Could someone help me?
Using this
http://myHost:port/documents/docID?filter={%27users%27:{%27$elemMatch%27:{%27userID%27:%2712%27}}}
restheart returns me all the documents: userID 11 and 12.
Your request is against a document resource, i.e. the URL is http://myHost:port/documents/docID
The filter query parameter applies for collection requests, i.e. URLs such as http://myHost:port/documents
In any case you need to projection (the keys query parameter) to limit the returned properties.
You should achieve it with the following request (I haven't tried it) using the $elementMatch projection operator:
http://myHost:port/documents?keys={"users":{"$elemMatch":{"userID":"12"}}}
I have a collection of students that have a name and an array of email addresses. A student document looks something like this:
{
"_id": {"$oid": "56d06bb6d9f75035956fa7ba"},
"name": "John Doe",
"emails": [
{
"label": "private",
"value": "private#johndoe.com"
},
{
"label": "work",
"value": "work#johndoe.com"
}
]
}
The label in the email subdocument is set to be unique per document, so there can't be two entries with the same label.
My problems is, that when updating a student document, I want to achieve the following:
adding an email with a new label should simply add a new subdocument with the given label and value to the array
if adding an email with a label that already exists, the value of the existing should be set to the data of the update
For example when updating with the following data:
{
"_id": {"$oid": "56d06bb6d9f75035956fa7ba"},
"emails": [
{
"label": "private",
"value": "me#johndoe.com"
},
{
"label": "school",
"value": "school#johndoe.com"
}
]
}
I would like the result of the emails array to be:
"emails": [
{
"label": "private",
"value": "me#johndoe.com"
},
{
"label": "work",
"value": "work#johndoe.com"
},
{
"label": "school",
"value": "school#johndoe.com"
}
]
How can I achieve this in MongoDB (optionally using mongoose)? Is this at all possible or do I have to check the array myself in the application code?
You could try this update but only efficient for small datasets:
mongo shell:
var data = {
"_id": ObjectId("56d06bb6d9f75035956fa7ba"),
"emails": [
{
"label": "private",
"value": "me#johndoe.com"
},
{
"label": "school",
"value": "school#johndoe.com"
}
]
};
data.emails.forEach(function(email) {
var emails = db.students.findOne({_id: data._id}).emails,
query = { "_id": data._id },
update = {};
emails.forEach(function(e) {
if (e.label === email.label) {
query["emails.label"] = email.label;
update["$set"] = { "emails.$.value": email.value };
} else {
update["$addToSet"] = { "emails": email };
}
db.students.update(query, update)
});
});
Suggestion: refactor your data to use the "label" as an actual field name.
There is one straightforward way in which MongoDB can guarantee unique values for a given email label - by making the label a single separate field in itself, in an email sub-document. Your data needs to exist in this structure:
{
"_id": ObjectId("56d06bb6d9f75035956fa7ba"),
"name": "John Doe",
"emails": {
"private": "private#johndoe.com",
"work" : "work#johndoe.com"
}
}
Now, when you want to update a student's emails you can do an update like this:
db.students.update(
{"_id": ObjectId("56d06bb6d9f75035956fa7ba")},
{$set: {
"emails.private" : "me#johndoe.com",
"emails.school" : "school#johndoe.com"
}}
);
And that will change the data to this:
{
"_id": ObjectId("56d06bb6d9f75035956fa7ba"),
"name": "John Doe",
"emails": {
"private": "me#johndoe.com",
"work" : "work#johndoe.com",
"school" : "school#johndoe.com"
}
}
Admittedly there is a disadvantage to this approach: you will need to change the structure of the input data, from the emails being in an array of sub-documents to the emails being a single sub-document of single fields. But the advantage is that your data requirements are automatically met by the way that JSON objects work.
After investigating the different options posted, I decided to go with my own approach of doing the update manually in the code using lodash's unionBy() function. Using express and mongoose's findById() that basically looks like this:
Student.findById(req.params.id, function(err, student) {
if(req.body.name) student.name = req.body.name;
if(req.body.emails && req.body.emails.length > 0) {
student.emails = _.unionBy(req.body.emails, student.emails, 'label');
}
student.save(function(err, result) {
if(err) return next(err);
res.status(200).json(result);
});
});
This way I get the full flexibility of partial updates for all fields. Of course you could also use findByIdAndUpdate() or other options.
Alternate approach:
However the way of changing the schema like Vince Bowdren suggested, making label a single separate field in a email subdocument, is also a viable option. In the end it just depends on your personal preferences and if you need strict validation on your data or not.
If you are using mongoose like I do, you would have to define a separate schema like so:
var EmailSchema = new mongoose.Schema({
work: { type: String, validate: validateEmail },
private: { type: String, validate: validateEmail }
}, {
strict: false,
_id: false
});
In the schema you can define properties for the labels you already want to support and add validation. By setting the strict: false option, you would allow the user to also post emails with custom labels. Note however, that these would not be validated. You would have to apply the validation manually in your application similar to the way I did it in my approach above for the merging.
I'm trying to update a single subelement contained within an array in a mongodb document. I want to reference the field using its array index (elements within the array don't have any fields that I can guarantee will be unique identifiers). Seems like this should be easy to do, but I can't figure out the syntax.
Here's what I want to do in pseudo-json.
Before:
{
_id : ...,
other_stuff ... ,
my_array : [
{ ... old content A ... },
{ ... old content B ... },
{ ... old content C ... }
]
}
After:
{
_id : ...,
other_stuff ... ,
my_array : [
{ ... old content A ... },
{ ... NEW content B ... },
{ ... old content C ... }
]
}
Seems like the query should be something like this:
//pseudocode
db.my_collection.update(
{_id: ObjectId(document_id), my_array.1 : 1 },
{my_array.$.content: NEW content B }
)
But this doesn't work. I've spent way too long searching the mongodb docs, and trying different variations on this syntax (e.g. using $slice, etc.). I can't find any clear explanation of how to accomplish this kind of update in MongoDB.
As expected, the query is easy once you know how. Here's the syntax, in python:
db["my_collection"].update(
{ "_id": ObjectId(document_id) },
{ "$set": { 'documents.'+str(doc_index)+'.content' : new_content_B}}
)
Update of an array element referenced by an index (e.g. 1 ) in Mongo Shell can also be done by directly indicating the index value:
db.my_collection.update(
{_id : "document_id"},
{$set : {"my_array.1.content" : "New content B"}}
)
In mongo style, using '$' positional operator.
Check out this link for details.
db.my_collection.update(
{_id: ObjectId(document_id), my_array.1 : 1 },
{ $set: { "my_array.$.content" : "NEW content B" } }
)
When it's required to update an array element without knowing it's actual index but having a unique identifier of the element:
// Modify a comment in a bucket
db.POST_COMMENT.update(
{
"_id": ObjectId("5ec424a1ed1af85a50855964"),
"bucket.commentId": "5eaf258bb80a1f03cd97a3ad_lepf4f"
},
{
$set: {
"bucket.$.text": "Comment text changed",
"bucket.$.createdDate": ISODate("2015-12-11T14:12:00.000+0000")
}
}
)
Here "bucket.commentId" is the unique identifier of an array element.
A neat way to do it in Javascript, with backticks, is:
const index = 1;
... { $set: { [`myArray.${index}.value`]: "new content"} }, ...
db.my_collection.update(
{_id: ObjectId(document_id), my_array : { ... old content A ... } },
{ $set: { "my_array.$.content" : "NEW content B" } }
)
When it's required to update an array element without knowing it's an actual index but having a unique identifier of the element
db.getCollection('profiles').update(
{
'userId':'4360a380-1540-45d9-b902-200f2d346263',
'skills.name':'css'
},
{
$set: {'skills.$.proficiencyLevel': 5}
},
{
multi: true
}
)
If you want to update the authorName of the testimonial having _id = 60c4918d74c30165ba585c14 from the following document:
"business": {
"ownerId": "60a5ebad7432d91b853c0277",
"testimonials": [
{
"_id": "60c4912877dd5664f2201b08",
"authorName": "user1",
"authorBio": "User from 10 years",
"image": "user1/img1",
"review": "asdfiuahsdfpoiuashdpfoaspdlfkjn;alsfpuoh"
},
{
"_id": "60c4918d74c30165ba585c14",
"authorName": "user2",
"authorBio": "User from 3 years",
"image": "user/img1",
"review": "asdpfuahsfljnsadfoihsf."
}
],
"createdAt": "2021-06-11T20:12:56.666Z",
"updatedAt": "2021-06-12T11:11:56.696Z",
}
Then the following mongoose query works:
await BusinessModel.updateOne(
{
'_id': Mongoose.Types.ObjectId(businessId),
'testimonials._id': Mongoose.Types.ObjectId('60c4918d74c30165ba585c14')
},
{
$set: { 'testimonials.$.authorName' : 'new author name' }
}
);
Also refer to https://docs.mongodb.com/drivers/node/fundamentals/crud/write-operations/embedded-arrays/
You can use the updateOne function of mongoDB passing the index of the element in array, if the key of old content B is "value" per example:
[
...
"value" : "old content A"
"value" : "old content B"
"value" : "old content C"
...
]
the command should be like this:
db.collection.updateOne({"_id" : "...,"},{$set: {"my_array.1.value": "NEW content B"}})
If you have a "plain" array containing simple strings, this did the trick:
db.paintings.insertMany([
{_id: 1, colors: ["red", "blue", "green"]},
{_id: 2, colors: ["red", "yellow"]}
db.paintings.updateMany(
{colors: "red"},
{$set: {"colors.$": "magenta"}})
the positional $ operator acts as a placeholder for the first element that matches the query document
Source