I have a document like this:
{
"InDate": "11.09.2015",
"Kst2Kst": true,
"OutDate": "11.09.2015",
"__v": 0,
"_id": ObjectId('55f2df2d7e12a9f1f52837e6'),
"accepted": true,
"inventar": [
{
"accepted": "1",
"name": "AAAA",
"isstammkost": true,
"stammkost": "IWXI"
},
{
"accepted": "1",
"name": "BBBB",
"isstammkost": false,
"stammkost": "null"
}
]
}
I want to select the data with "isstammkost": true in the inventar-array.
My query is:
Move.findOne({accepted : true, 'inventar.isstammkost' : true},
'OutDate InDate inventar.name', function(err, res)
It doesn't work -> It selects all, even with inventar.isstammkost : false.
The "normal" query works like I want (without criteria in sub-array). Whats the right way to set criteria in sub-array?
Of course it will return the "isstammkost": false part, because that is part of the same document as the "isstammkost": true. They are both objects in the array "inventar", a top-level field in a single document. Without some sort of projection, the entire document will always be returned to a mongodb query and thus nodejs will pass them on to you.
I'm not terribly up-to-speed on nodejs, but if this were the mongo shell it would look like this:
> db.MyDB.findOne({{accepted : true, "inventar.isstammkost" : true}, {"inventar.isstammkost.$": 1});
You will need to find out how to add that extra parameter to the nodejs function.
Related
I am having trouble with querying a MongoDB collection with an array inside.
Here is the structure of my collection that I am querying. This is one record:
{
"_id": "abc123def4567890",
"profile_id": "abc123def4567890",
"image_count": 2,
"images": [
{
"image_id": "ABC123456789",
"image_url": "images/something.jpg",
"geo_loc": "-0.1234,11.234567890",
"title": "A Title",
"shot_time": "01:23:33",
"shot_date": "11/22/2222",
"shot_type": "scenery",
"conditions": "cloudy",
"iso": 16,
"f": 2.4,
"ss": "1/545",
"focal": 6.0,
"equipment": "",
"instructions": "",
"upload_date": 1234567890,
"update_date": 1234567890
},
{
"image_id": "ABC123456789",
"image_url": "images/something.jpg",
"geo_loc": "-0.1234,11.234567890",
"title": "A Title",
"shot_time": "01:23:33",
"shot_date": "11/22/2222",
"shot_type": "portrait",
"conditions": "cloudy",
"iso": "16",
"f": "2.4",
"ss": "1/545",
"focal": "6.0",
"equipment": "",
"instructions": "",
"upload_date": 1234567890,
"update_date": 1234567890
}
]
}
Forgive the formatting, I didn't know how else to show this.
As you can see, it's a profile with a series of images within an array called 'images' and there are 2 images. Each of the 'images' array items contain an object of attributes for the image (url, title, type, etc).
All I want to do is to return the object element whose attributes match certain criteria:
Select object from images which has shot_type = "scenery"
I tried to make it as simple as possible so i started with:
find( { "images.shot_type": "scenery" } )
This returns the entire record and both the images within. So I tried projection but I could not isolate the single object within the array (in this case object at position 0) and return it.
I think the answer lies with projection but I am unsure.
I have gone through the MongoDB documents for hours now and can't find inspiration. I have read about $elemMatch, $, and the other array operators, nothing seems to allow you to single out an array item based on data within. I have been through this page too https://docs.mongodb.com/manual/tutorial/query-arrays/ Still can't work it out.
Can anyone provide help?
Have I made an error by using '$push' to populate my images field (making it an array) instead of using '$set' which would have made it into an embedded document? Would this have made a difference?
Using aggregation:
db.collection.aggregate({
$project: {
_id: 0,
"result": {
$filter: {
input: "$images",
as: "img",
cond: {
$eq: [
"$$img.shot_type",
"scenery"
]
}
}
}
}
})
Playground
You can use $elemMatch in this way (simplified query):
db.collection.find({
"profile_id": "1",
},
{
"images": {
"$elemMatch": {
"shot_type": 1
}
}
})
You can use two objects into find query. The first will filter all document and will only get those whose profile_id is 1. You can omit this stage and use only { } if you wnat to search into the entire collection.
Then, the other object uses $elemMatch to get only the element whose shot_type is 1.
Check an example here
Consider a collection with the following documents:
{
name: "John Doe",
emails: [
{
value: "some#domain.com",
isValid: true,
isPreferred: true
}
]
},
{
name: "John Doe",
emails: [
{
value: "john.doe#gmail.com",
isValid: false,
isPreferred: false
},
{
value: "john.doe#domain.com",
isValid: true,
isPreferred: true
}
]
}
There should be no users with the same valid and preferred emails, so there is a unique index for that:
db.users.createIndex( { "emails.value": 1 }, { name: "loginEmail", unique: true, partialFilterExpression: { "emails.isValid": true, "emails.isPreferred": true } } )
Adding the following email to the first document triggers the unique constraint violation:
{
name: "John Doe",
emails: [
{
value: "john.doe#gmail.com",
isValid: false,
isPreferred: false
}
]
}
Caused by: com.mongodb.MongoCommandException: Command failed with
error 11000 (DuplicateKey): 'E11000 duplicate key error collection:
profiles.users index: loginEmail dup key: { emails.value:
"john.doe#gmail.com", emails.isValid: false, emails.isPreferred: false
}' on server profiles-db-mongodb.dev:27017. The full response is
{"ok": 0.0, "errmsg": "E11000 duplicate key error collection:
profiles.users index: loginEmail dup key: { emails.value:
"john.doe#gmail.com", emails.isValid: false, emails.isPreferred:
false }", "code": 11000, "codeName": "DuplicateKey", "keyPattern":
{"emails.value": 1, "emails.isValid": 1, "emails.isPreferred": 1},
"keyValue": {"emails.value": "john.doe#gmail.com", "emails.isValid":
false, "emails.isPreferred": false}}
As I can understand, this happens because the filter expression is applied to the collection, not to the embedded documents, so although being somewhat counterintuitive and unexpected, the index behaves as described.
My question is how can I ensure partial uniqueness without having false negatives?
TLDR: You cant.
Let's understand why it's happening first, maybe then we'll understand what can be done. The problem originates due to a combination of two Mongo features.
the dot notation syntax. The dot notation syntax allows you to query subdocuments in arrays at ease ("emails.isPreferred": true). However when you want to start using multiple conditions for subdocuments like in your case you need to use something like $elemMatch sadly the restrictions for partialFilterExpression are quite restrictive and do not give you such power.
Which means even docs with emails such as:
{
"_id": ObjectId("5f106c0e823eea49427eea64"),
"name": "John Doe",
"emails": [
{
"value": "john.doe#gmail.com",
"isValid": true,
"isPreferred": false
},
{
"value": "john.doe#domain.com",
"isValid": false,
"isPreferred": true
}
]
}
Will be indexed. So ok, We will have some extra indexed documents in the collection but still apart from (falsely) increasing index size you still hope it might work, but it doesn't due to point 2.
multikey indexes:
MongoDB uses multikey indexes to index the content stored in arrays. ... , MongoDB creates separate index entries for every element of the array.
So when you create an index on an array or on any field of a sub document in an array Mongo will "flatten" the array and create a unique entry for each of the documents. and in this case it will create a unique index for all emails in the array.
So due to all these "features" and the restrictions of the partial filter syntax usage we can't really achieve what you want.
So what can you do? I'm sure you're already thinking of possible work arounds through this. A simple solution would be to maintain an extra field that will only contain those isValid and isPreferred emails. then a unique sparse index will do the trick.
I would like to upsert an element in an array, based on doc _id and element _id. Currently it works only if the element is allready in the array (update works, insert not).
So, these collection:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c"
}
]
}]
Receiving this request:
query = { _id: '5a65fcf363e2a32531ed9f9b', 'ressources._id': '5a65fd0363e2a32531ed9f9c' };
update = { '$set': { 'ressources.$': { '_id': '5a65fd0363e2a32531ed9f9c', qt: '153', unit: 'kg' } } };
options = {upsert:true};
collection.update(query,update,options);
Will give this ok result:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c",
"qt": 153,
"unit": "kg"
}
]
}]
How to make the same request work with these initial collections:
[{
"_id": "5a65fcf363e2a32531ed9f9b"
}]
OR
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": []
}]
How to make the upsert work?
Does upsert works with entire document only?
Currently, I face this error:
The positional operator did not find the match needed from the query.
Thanks
I also tried to figure out how to do it. I found only one way:
fetch model by id
update array manually (via javascript)
save the model
Sad to know that in 2018 you still have to do the stuff like it.
UPDATE:
This will update particular element in viewData array
db.collection.update({
"_id": args._id,
"viewData._id": widgetId
},
{
$set: {
"viewData.$.widgetData": widgetDoc.widgetData
}
})
$push command will add new items
I have a collection of students that have a name and an array of email addresses. A student document looks something like this:
{
"_id": {"$oid": "56d06bb6d9f75035956fa7ba"},
"name": "John Doe",
"emails": [
{
"label": "private",
"value": "private#johndoe.com"
},
{
"label": "work",
"value": "work#johndoe.com"
}
]
}
The label in the email subdocument is set to be unique per document, so there can't be two entries with the same label.
My problems is, that when updating a student document, I want to achieve the following:
adding an email with a new label should simply add a new subdocument with the given label and value to the array
if adding an email with a label that already exists, the value of the existing should be set to the data of the update
For example when updating with the following data:
{
"_id": {"$oid": "56d06bb6d9f75035956fa7ba"},
"emails": [
{
"label": "private",
"value": "me#johndoe.com"
},
{
"label": "school",
"value": "school#johndoe.com"
}
]
}
I would like the result of the emails array to be:
"emails": [
{
"label": "private",
"value": "me#johndoe.com"
},
{
"label": "work",
"value": "work#johndoe.com"
},
{
"label": "school",
"value": "school#johndoe.com"
}
]
How can I achieve this in MongoDB (optionally using mongoose)? Is this at all possible or do I have to check the array myself in the application code?
You could try this update but only efficient for small datasets:
mongo shell:
var data = {
"_id": ObjectId("56d06bb6d9f75035956fa7ba"),
"emails": [
{
"label": "private",
"value": "me#johndoe.com"
},
{
"label": "school",
"value": "school#johndoe.com"
}
]
};
data.emails.forEach(function(email) {
var emails = db.students.findOne({_id: data._id}).emails,
query = { "_id": data._id },
update = {};
emails.forEach(function(e) {
if (e.label === email.label) {
query["emails.label"] = email.label;
update["$set"] = { "emails.$.value": email.value };
} else {
update["$addToSet"] = { "emails": email };
}
db.students.update(query, update)
});
});
Suggestion: refactor your data to use the "label" as an actual field name.
There is one straightforward way in which MongoDB can guarantee unique values for a given email label - by making the label a single separate field in itself, in an email sub-document. Your data needs to exist in this structure:
{
"_id": ObjectId("56d06bb6d9f75035956fa7ba"),
"name": "John Doe",
"emails": {
"private": "private#johndoe.com",
"work" : "work#johndoe.com"
}
}
Now, when you want to update a student's emails you can do an update like this:
db.students.update(
{"_id": ObjectId("56d06bb6d9f75035956fa7ba")},
{$set: {
"emails.private" : "me#johndoe.com",
"emails.school" : "school#johndoe.com"
}}
);
And that will change the data to this:
{
"_id": ObjectId("56d06bb6d9f75035956fa7ba"),
"name": "John Doe",
"emails": {
"private": "me#johndoe.com",
"work" : "work#johndoe.com",
"school" : "school#johndoe.com"
}
}
Admittedly there is a disadvantage to this approach: you will need to change the structure of the input data, from the emails being in an array of sub-documents to the emails being a single sub-document of single fields. But the advantage is that your data requirements are automatically met by the way that JSON objects work.
After investigating the different options posted, I decided to go with my own approach of doing the update manually in the code using lodash's unionBy() function. Using express and mongoose's findById() that basically looks like this:
Student.findById(req.params.id, function(err, student) {
if(req.body.name) student.name = req.body.name;
if(req.body.emails && req.body.emails.length > 0) {
student.emails = _.unionBy(req.body.emails, student.emails, 'label');
}
student.save(function(err, result) {
if(err) return next(err);
res.status(200).json(result);
});
});
This way I get the full flexibility of partial updates for all fields. Of course you could also use findByIdAndUpdate() or other options.
Alternate approach:
However the way of changing the schema like Vince Bowdren suggested, making label a single separate field in a email subdocument, is also a viable option. In the end it just depends on your personal preferences and if you need strict validation on your data or not.
If you are using mongoose like I do, you would have to define a separate schema like so:
var EmailSchema = new mongoose.Schema({
work: { type: String, validate: validateEmail },
private: { type: String, validate: validateEmail }
}, {
strict: false,
_id: false
});
In the schema you can define properties for the labels you already want to support and add validation. By setting the strict: false option, you would allow the user to also post emails with custom labels. Note however, that these would not be validated. You would have to apply the validation manually in your application similar to the way I did it in my approach above for the merging.
Here i quoted my code for multiple search filtering. I could not find the mistakes in that. please give a right code to make it work well.
Employee document:
{
"_id": "527c8d9327c6f27f17df0d2e17000530",
"_rev": "24-276a8dc913559901897fd601d2f9654f",
"proj_role": "TeamMember",
"work_total_experience": "3",
"personal": {
"languages_known": [
"English","Telugu"
]},
"skills": [
{
"skill_set": "Webservices Framework",
"skill_exp": 1,
"skill_certified": "yes",
"skill_rating": 3,
},
{
"skill_set": "Microsoft",
"skill_exp": 1,
"skill_certified": "yes",
"skill_rating": 3,
}
]
"framework_competency": "Nasscom",
"type": "employee-docs"
}
Design Document:
{
"_id": "_design/sample",
"_rev": "86-1250f792e6e84f6f33447a00cf64d61d",
"views": {},
"language": "javascript",
"indexes": {
"search": {
"index": "function(doc){\n index(\"default\", doc._id);if(doc.type=='employee-docs'){\nif (doc.proj_role){index(\"project_role\", doc.proj_role);}if(doc.work_total_experience){\nindex(\"work_experience\", doc.work_total_experience);}\nif(doc.personal.languages_known){for(c in doc.personal.languages_known){ \n index(\"languages_known\",doc.personal.languages_known[c]);}} if(doc.skills){for (var i=0;i<doc.skills.length;i++){\nindex('skill_set',doc.skills[i].skill_set);}}}}"
}
}
}
Run using below URL : https://ideyeah4.cloudant.com/opteamize_new/_design/sample/_search/search?q=project_role:TeamMember%20AND%20work_experience:%223%22%20AND%20languages_known:Telugu%20AND%20skill_set:Microsoft&include_docs=true
A simple way to debug this is to query the top 100 results in your index:
https://ideyeah4.cloudant.com/opteamize_new/_design/sample/_search/search?q=*:*&limit=100
This will at least tell you whether there are any documents in your index at all.
Your current query (without URL encoding) looks like:
project_role:TeamMember AND work_experience:"3" AND languages_known:Telugu AND skill_set:Microsoft
I'd suggest that some of these search values require quotes - always true when you are searching string values. Next, you could try:
project_role:"TeamMember"
see if you get any results and refine from there.
Debugging this might also be easier if you store the values as well as index them (so you can see exactly what is indexed). To do this, add an object to each index call { "store": true }. For example,
index("languages_known", doc.personal.languages_known[c], { "store": true });
Now, when you query the index it will return a list of fields which were stored with each match.