I have a document like this:
{
_id : ObjectID(),
title: "",
items: [
{
"itemId" : 1234678,
}
]
}
itemId is a unique index created like this:
db.allItems.createIndex( { "items.itemId" : 1 }, { unique: true});
And then everything works fine, until I set items array (not pushing one), in this case, unique index does not work. The following data in the update operation (using $set) does not throw an error and works fine, which MUST NOT. I mean it creates the sub-document without any unique error
items: [
{
itemId: 1234678
},
{
itemId: 1234678
}
]
While I expect MongoDB to throw error that itemId is not unique.
MongoDb index uniqueness is applicable for documents, not for nested arrays.
If you try to insert new document with:
items: [
{
"itemId" : 1234678,
},
...
]
MonogDB will throw E11000 duplicate key error collection
Related
I have a Profile model, that contains this field:
interests: {
type: [String],
},
My app has been running for a while. So this means for several documents, this field has already been filled with an array of strings.
In order to achieve certain goals, I need to create a model Interest with a field name and then refer to it in the Profile like this:
interests: [{
type: Schema.Types.ObjectId,
ref: "interests",
}],
The field name should contain the already existing string interests in Profile.interests.
This is the approach that I think I will follow:
Create Interest model.
Fill name field with the existing Profile.interests strings.
a. When doing this replace Profile.interests with the _ids of the newly created Interest documents.
b. Make sure Interest.name is unique.
c. Remove spaces.
Wherever interests in the app are used in the backend, use populate to fill them.
This doesn't feel like a safe operation. So I would like to hear your thoughts on it. Is there a better approach? Should I avoid doing this?
Thank you.
Step 1:
Create a Model for interests,
specify your desired fields fir interests schema and set properties for particular fields
specify collection name in options as per your requirement
create a model and specify your desired name in model
const InterestsSchema = new mongoose.Schema(
{ name: { type: String } },
{ collection: "interests" }
);
const Interests = mongoose.model("Interests", InterestsSchema);
Instead of removing interests field add new field interest (you can choose desired field), for safe side whenever you feel the current update working properly you can remove it, Update profile schema,
update interest field as per your requirement, now newly added field is interest
interests: {
type: [String]
},
interest: [{
type: Schema.Types.ObjectId,
ref: "interests"
}],
Step 2:
Wherever interests in the app are used in the backend, use interest and populate to fill them.
Step 3: (just execute the query)
Make a collection for interests and store all unique interests string from profile collection, so write a aggregation query to select unique string and store in interests collection, you can execute this query in mongo shell or any editor that you are using after specifying your original profile collection name,
$project to show interests field only because we are going to deconstruct it
$unwind to deconstruct interests array
$group by interests and select unique field, and trim white space from interests string
$project to show name field and if you want to then add your desired fields
$out will create a new collection interests and write all interests with newly generated _id field
db.getCollection('profile').aggregate([
{ $project: { interests: 1 } },
{ $unwind: "$interests" },
{ $group: { _id: { $trim: { input: "$interests" } } } },
{ $project: { _id: 0, name: "$_id" } },
{ $out: "interests" }
])
Playground
You have example input:
[
{
"_id": 1,
"interests": ["sports","sing","read"]
},
{
"_id": 2,
"interests": ["tracking","travel"]
}
]
After executing above query the output/result in interests / new collection would be something like:
[
{
"_id": ObjectId("5a934e000102030405000000"),
"name": "travel"
},
{
"_id": ObjectId("5a934e000102030405000001"),
"name": "sports"
},
{
"_id": ObjectId("5a934e000102030405000002"),
"name": "read"
},
{
"_id": ObjectId("5a934e000102030405000003"),
"name": "tracking"
},
{
"_id": ObjectId("5a934e000102030405000004"),
"name": "sing"
}
]
Step 4: (just execute the query)
Add new field interest with reference _ids from interests collection in profile collection, there are sequence to execute queries,
find profile query and project only required fields _id and interests when interest (new field) field is not exists and iterate loop using forEach
trim interests string iterating loop through map
find the interests reference _id by its name field from created interests collection
update query for add interest field that have _ids in profile collection
db.getCollection('profile').find(
{ interest: { $exists: false } },
{ _id: 1, interests: 1 }).forEach(function(profileDoc) {
// TRIM INTEREST STRING
var interests = profileDoc.interests.map(function(i){
return i.trim();
});
// FIND INTERESTS IDs
var interest = [];
db.getCollection('interests').find(
{ name: { $in: interests } },
{ _id: 1 }).forEach(function(interestDoc){
interest.push(interestDoc._id);
});
// UPDATE IDS IN PROFILE DOC
db.getCollection('profile').updateOne(
{ _id: profileDoc._id },
{ $set: { interest: interest } }
);
});
You have example input:
[
{
"_id": 1,
"interests": ["sports","sing","read"]
},
{
"_id": 2,
"interests": ["tracking","travel"]
}
]
After executing above query the result in your profile collection would be something like:
[
{
"_id": 1,
"interests": ["sports","sing","read"],
"interest": [
ObjectId("5a934e000102030405000001"),
ObjectId("5a934e000102030405000002"),
ObjectId("5a934e000102030405000004")
]
},
{
"_id": 2,
"interests": ["tracking","travel"],
"interest": [
ObjectId("5a934e000102030405000000"),
ObjectId("5a934e000102030405000003")
]
}
]
Step 5:
Now you have completed all the steps and you have newly added interest field
and also old field interests field is still in safe mode, just make sure everything is working properly you can delete old interests field,
remove old field interests field from all profiles
db.getCollection('profile').updateMany(
{ interests: { $exists: true } },
{ $unset: { "interests": 1 } }
);
Playground
Warning:
Test this steps in your local/development server before executing in production server.
Take backup of your database collections before executing queries.
Field and schema names are predicted you can replace with your original name.
im trying to add element to array in document MongoDB collection with Aggregate (not regular update) match to specific name
but i dont find the right syntax
for example:
{"name":"TEST1","department":"T1","skills":["JS","HTML"],"expY":5}
{"name":"TEST2","department":"T2","skills":["Java","CSS"],"expY":3}
i want to add to skills array "CSS" i try with sort, match, group, project, ($add, $push etc) and none of those syntax not working
db.collection.aggregate([
{ $match : { name : "TEST1" } },
HERE I DONT KNOW WHAT IS THE RIGHT SYNTAX TO ADD ELEMENT TO skills array
])
If I am understading your question correctly, you want to match all the documents with name "TEST1" and then add "CSS" to the skills array. If that's correct, here is one way to do that:
db.collection.aggregate([
{
$match: {
name: "TEST1"
}
},
{
$project: {
name: "$name",
department: "$department",
expY: "$expY",
skills: {
$concatArrays: [
"$skills",
[
"CSS"
]
]
}
}
}
])
Playground
Here is how to update your document to add "CSS" to the array:
db.your_collection.update({ name : "TEST1" }, { $push: { skills: 'CSS' } })
Consider a collection with the following documents:
{
name: "John Doe",
emails: [
{
value: "some#domain.com",
isValid: true,
isPreferred: true
}
]
},
{
name: "John Doe",
emails: [
{
value: "john.doe#gmail.com",
isValid: false,
isPreferred: false
},
{
value: "john.doe#domain.com",
isValid: true,
isPreferred: true
}
]
}
There should be no users with the same valid and preferred emails, so there is a unique index for that:
db.users.createIndex( { "emails.value": 1 }, { name: "loginEmail", unique: true, partialFilterExpression: { "emails.isValid": true, "emails.isPreferred": true } } )
Adding the following email to the first document triggers the unique constraint violation:
{
name: "John Doe",
emails: [
{
value: "john.doe#gmail.com",
isValid: false,
isPreferred: false
}
]
}
Caused by: com.mongodb.MongoCommandException: Command failed with
error 11000 (DuplicateKey): 'E11000 duplicate key error collection:
profiles.users index: loginEmail dup key: { emails.value:
"john.doe#gmail.com", emails.isValid: false, emails.isPreferred: false
}' on server profiles-db-mongodb.dev:27017. The full response is
{"ok": 0.0, "errmsg": "E11000 duplicate key error collection:
profiles.users index: loginEmail dup key: { emails.value:
"john.doe#gmail.com", emails.isValid: false, emails.isPreferred:
false }", "code": 11000, "codeName": "DuplicateKey", "keyPattern":
{"emails.value": 1, "emails.isValid": 1, "emails.isPreferred": 1},
"keyValue": {"emails.value": "john.doe#gmail.com", "emails.isValid":
false, "emails.isPreferred": false}}
As I can understand, this happens because the filter expression is applied to the collection, not to the embedded documents, so although being somewhat counterintuitive and unexpected, the index behaves as described.
My question is how can I ensure partial uniqueness without having false negatives?
TLDR: You cant.
Let's understand why it's happening first, maybe then we'll understand what can be done. The problem originates due to a combination of two Mongo features.
the dot notation syntax. The dot notation syntax allows you to query subdocuments in arrays at ease ("emails.isPreferred": true). However when you want to start using multiple conditions for subdocuments like in your case you need to use something like $elemMatch sadly the restrictions for partialFilterExpression are quite restrictive and do not give you such power.
Which means even docs with emails such as:
{
"_id": ObjectId("5f106c0e823eea49427eea64"),
"name": "John Doe",
"emails": [
{
"value": "john.doe#gmail.com",
"isValid": true,
"isPreferred": false
},
{
"value": "john.doe#domain.com",
"isValid": false,
"isPreferred": true
}
]
}
Will be indexed. So ok, We will have some extra indexed documents in the collection but still apart from (falsely) increasing index size you still hope it might work, but it doesn't due to point 2.
multikey indexes:
MongoDB uses multikey indexes to index the content stored in arrays. ... , MongoDB creates separate index entries for every element of the array.
So when you create an index on an array or on any field of a sub document in an array Mongo will "flatten" the array and create a unique entry for each of the documents. and in this case it will create a unique index for all emails in the array.
So due to all these "features" and the restrictions of the partial filter syntax usage we can't really achieve what you want.
So what can you do? I'm sure you're already thinking of possible work arounds through this. A simple solution would be to maintain an extra field that will only contain those isValid and isPreferred emails. then a unique sparse index will do the trick.
I would like to upsert an element in an array, based on doc _id and element _id. Currently it works only if the element is allready in the array (update works, insert not).
So, these collection:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c"
}
]
}]
Receiving this request:
query = { _id: '5a65fcf363e2a32531ed9f9b', 'ressources._id': '5a65fd0363e2a32531ed9f9c' };
update = { '$set': { 'ressources.$': { '_id': '5a65fd0363e2a32531ed9f9c', qt: '153', unit: 'kg' } } };
options = {upsert:true};
collection.update(query,update,options);
Will give this ok result:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c",
"qt": 153,
"unit": "kg"
}
]
}]
How to make the same request work with these initial collections:
[{
"_id": "5a65fcf363e2a32531ed9f9b"
}]
OR
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": []
}]
How to make the upsert work?
Does upsert works with entire document only?
Currently, I face this error:
The positional operator did not find the match needed from the query.
Thanks
I also tried to figure out how to do it. I found only one way:
fetch model by id
update array manually (via javascript)
save the model
Sad to know that in 2018 you still have to do the stuff like it.
UPDATE:
This will update particular element in viewData array
db.collection.update({
"_id": args._id,
"viewData._id": widgetId
},
{
$set: {
"viewData.$.widgetData": widgetDoc.widgetData
}
})
$push command will add new items
I have a document structure like
{
"_id" : ObjectId("52263922f5ebf05115bf550e"),
"Fields" : [
{
"Field" : "Lot No",
"Rules" : [ ]
},
{
"Field" : "RMA No",
"Rules" : [ ]
}
]
}
I have tried to update by using the following code to push into the Rules Array which will hold objects.
db.test.update({
"Fields.Field":{$in:["Lot No"]}
}, {
$addToSet: {
"Fields.Field.$.Rules": {
"item_name": "my_item_two",
"price": 1
}
}
}, false, true);
But I get the following error:
can't append to array using string field name [Field]
How do I do the update?
You gone too deep with that wildcard $. You match for an item in the Fields array, so you get a access on that, with: Fields.$. This expression returns the first match in your Fields array, so you reach its fields by Fields.$.Field or Fields.$.Result.
Now, lets update the update:
db.test.update({
"Fields.Field": "Lot No"
}, {
$addToSet: {
"Fields.$.Rules": {
'item_name': "my_item_two",
'price':1
}
}
}, false, true);
Please note that I've shortened the query as it is equal to your expression.