Details
I develop survey application with express and struggle with some getting of data.
The case:
you can get all surveys by "GET /surveys". And every survey doc has to contains hasVoted:mongoose.Bool and optionsVote:mongoose.Map if the user has voted for the survey. (SurveySchema is bellow)
you can vote for survey by "POST /surveys/vote"
you can see the results of any survey only if you vote for it
new Schema({
question: {
type: mongoose.Schema.Types.String,
required: true,
},
options: {
type: [{
type: mongoose.Schema.Types.String,
required: true,
}]
},
optionsVote: {
type: mongoose.Schema.Types.Map,
of: mongoose.Schema.Types.Number,
},
votesCount: {
type: mongoose.Schema.Types.Number,
},
votes: {
type: [{
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
option: mongoose.Schema.Types.Number,
}]
},
})
Target:
So the target of the question is how to add fields hasVoted and optionsVote if there is "Vote" sub document in votes array where user===req.user.id ?
I believe you got the idea so if you have an idea how to change the schema to achieve the desired result I'm open!
Example:
Data:
[{
id:"surveyId1
question:"Question",
options:["op1","op2"],
votes:[{user:"userId1", option:0}]
votesCount:1,
optionsVote:{"0":1,"1":0}
},{
id:"surveyId2
question:"Question",
options:["op1","op2"],
votes:[{user:"userId2", option:0}]
votesCount:1,
optionsVote:{"0":1,"1":0}
}]
Route handler:
Where req.user.id='userId1' and then make the desired query.
The result
[{ // Voted for this survey
id:"surveyId1
question:"Question",
options:["op1","op2"],
votes:[{user:"userId1", option:0}]
votesCount:1,
optionsVote:{"0":1,"1":0},
hasVoted:true,
},{ // No voted for this survey
id:"surveyId2
question:"Question",
options:["op1","op2"],
votesCount:1,
}]
In MongoDB, you can search for sub document as follows
//Mongodb query to search for survey filled by a user
db.survey.find({ 'votes.user': myUserId })
So with this when you can get results only where user has voted, do you really need hasVoted field?
To have optionsVote field, first I would prefer schema of optionsVote as {option: "a", count:1}. You can choose any of the following approach.
A. manage to update optionsVote field at the time of update by incrementing the count of the voted option when you POST /survey/vote.
B. Another approach would be to calculate the optionsVote based on votes entries at the time of GET /survey. You can do this via aggregate
//Mongodb query to get optionsVote:{option: "a", count:1} from votes: { user:"x", option:"a"}
db.survey.aggregate([
{ $unwind: "$votes" },
{ $group: {
"_id": { "id": "_id", "option": "$votes.option" },
optionCount: { $sum: 1 }
}
},
{
$group: { "_id": "$_id.id" },
optionsVote: { $push : { option: "$_id.option", count: "$optionCount" } },
votes: { $push : '$votes'}
}
])
//WARNING: I haven't tested this query, this is just to show the approach -> group based on votes.option and count all votes for that option for each document and then create optionsVote field by pushing all option with their count using $push into the field `optionsVote`
I recommend approach A because I assume POST operations would be quite less than GET operations. Also it's easier to implement. Having said that, keeping query in B handy will help you with sanity check.
Related
In my chat app (+20m users for example) i need to get last updated chats but there's no optimal solution.
Chats
{
id: 1,
title: 'My Group',
updated_at: 137974654
},
{
id: 2,
title: 'Gamers',
updated_at: 137973654
}
Members
{
chat_id: 1,
user_id: 'A'
},
{
chat_id: 2,
user_id: 'B'
}
I don't want to embed members because a user can join thousands of chats so document size is beyond 16MB.
The maximum BSON document size is 16 megabytes.
A potential problem with the embedded document pattern is that it can lead to large documents, especially if the embedded field is unbounded. In this case, you can use the subset pattern to only access data which is required by the application, instead of the entire set of embedded data.
https://docs.mongodb.com/manual/tutorial/model-embedded-one-to-many-relationships-between-documents/#subset-pattern
My solution but not efficient
I first tried to get a list of chat_ids then query chats:
const members = Members.find({ user_id: 'A' }).project({ chat_id: 1 });
const chat_ids = members.map(item => item.chat_id);
const chats = Chats.find({
id: {
$in: chat_ids
}
}).sort({ updated_at: -1 }).limit(20);
But as i said, What if a user has joined +100000 chats? so the first query is too large + the last query is too slow.
What should i do? Do i need a relational database?
Making some assumptions that you are only interested in user_id: 'A', and keeping in mind that I am a MongoDB noob, and I only tested this with the example data you provided, does this do what you want?
db.members.aggregate([
{
$match: {
user_id: "A"
}
},
{
$lookup: {
from: "chats",
localField: "chat_id",
foreignField: "id",
as: "thechats"
}
},
{
"$unwind": "$thechats"
},
{
"$replaceRoot": {
"newRoot": "$thechats"
}
},
{
"$sort": {
updated_at: -1
}
},
{
"$limit": 20
}
])
Try it at mongoplayground.net.
I have a Profile model, that contains this field:
interests: {
type: [String],
},
My app has been running for a while. So this means for several documents, this field has already been filled with an array of strings.
In order to achieve certain goals, I need to create a model Interest with a field name and then refer to it in the Profile like this:
interests: [{
type: Schema.Types.ObjectId,
ref: "interests",
}],
The field name should contain the already existing string interests in Profile.interests.
This is the approach that I think I will follow:
Create Interest model.
Fill name field with the existing Profile.interests strings.
a. When doing this replace Profile.interests with the _ids of the newly created Interest documents.
b. Make sure Interest.name is unique.
c. Remove spaces.
Wherever interests in the app are used in the backend, use populate to fill them.
This doesn't feel like a safe operation. So I would like to hear your thoughts on it. Is there a better approach? Should I avoid doing this?
Thank you.
Step 1:
Create a Model for interests,
specify your desired fields fir interests schema and set properties for particular fields
specify collection name in options as per your requirement
create a model and specify your desired name in model
const InterestsSchema = new mongoose.Schema(
{ name: { type: String } },
{ collection: "interests" }
);
const Interests = mongoose.model("Interests", InterestsSchema);
Instead of removing interests field add new field interest (you can choose desired field), for safe side whenever you feel the current update working properly you can remove it, Update profile schema,
update interest field as per your requirement, now newly added field is interest
interests: {
type: [String]
},
interest: [{
type: Schema.Types.ObjectId,
ref: "interests"
}],
Step 2:
Wherever interests in the app are used in the backend, use interest and populate to fill them.
Step 3: (just execute the query)
Make a collection for interests and store all unique interests string from profile collection, so write a aggregation query to select unique string and store in interests collection, you can execute this query in mongo shell or any editor that you are using after specifying your original profile collection name,
$project to show interests field only because we are going to deconstruct it
$unwind to deconstruct interests array
$group by interests and select unique field, and trim white space from interests string
$project to show name field and if you want to then add your desired fields
$out will create a new collection interests and write all interests with newly generated _id field
db.getCollection('profile').aggregate([
{ $project: { interests: 1 } },
{ $unwind: "$interests" },
{ $group: { _id: { $trim: { input: "$interests" } } } },
{ $project: { _id: 0, name: "$_id" } },
{ $out: "interests" }
])
Playground
You have example input:
[
{
"_id": 1,
"interests": ["sports","sing","read"]
},
{
"_id": 2,
"interests": ["tracking","travel"]
}
]
After executing above query the output/result in interests / new collection would be something like:
[
{
"_id": ObjectId("5a934e000102030405000000"),
"name": "travel"
},
{
"_id": ObjectId("5a934e000102030405000001"),
"name": "sports"
},
{
"_id": ObjectId("5a934e000102030405000002"),
"name": "read"
},
{
"_id": ObjectId("5a934e000102030405000003"),
"name": "tracking"
},
{
"_id": ObjectId("5a934e000102030405000004"),
"name": "sing"
}
]
Step 4: (just execute the query)
Add new field interest with reference _ids from interests collection in profile collection, there are sequence to execute queries,
find profile query and project only required fields _id and interests when interest (new field) field is not exists and iterate loop using forEach
trim interests string iterating loop through map
find the interests reference _id by its name field from created interests collection
update query for add interest field that have _ids in profile collection
db.getCollection('profile').find(
{ interest: { $exists: false } },
{ _id: 1, interests: 1 }).forEach(function(profileDoc) {
// TRIM INTEREST STRING
var interests = profileDoc.interests.map(function(i){
return i.trim();
});
// FIND INTERESTS IDs
var interest = [];
db.getCollection('interests').find(
{ name: { $in: interests } },
{ _id: 1 }).forEach(function(interestDoc){
interest.push(interestDoc._id);
});
// UPDATE IDS IN PROFILE DOC
db.getCollection('profile').updateOne(
{ _id: profileDoc._id },
{ $set: { interest: interest } }
);
});
You have example input:
[
{
"_id": 1,
"interests": ["sports","sing","read"]
},
{
"_id": 2,
"interests": ["tracking","travel"]
}
]
After executing above query the result in your profile collection would be something like:
[
{
"_id": 1,
"interests": ["sports","sing","read"],
"interest": [
ObjectId("5a934e000102030405000001"),
ObjectId("5a934e000102030405000002"),
ObjectId("5a934e000102030405000004")
]
},
{
"_id": 2,
"interests": ["tracking","travel"],
"interest": [
ObjectId("5a934e000102030405000000"),
ObjectId("5a934e000102030405000003")
]
}
]
Step 5:
Now you have completed all the steps and you have newly added interest field
and also old field interests field is still in safe mode, just make sure everything is working properly you can delete old interests field,
remove old field interests field from all profiles
db.getCollection('profile').updateMany(
{ interests: { $exists: true } },
{ $unset: { "interests": 1 } }
);
Playground
Warning:
Test this steps in your local/development server before executing in production server.
Take backup of your database collections before executing queries.
Field and schema names are predicted you can replace with your original name.
I would like to upsert an element in an array, based on doc _id and element _id. Currently it works only if the element is allready in the array (update works, insert not).
So, these collection:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c"
}
]
}]
Receiving this request:
query = { _id: '5a65fcf363e2a32531ed9f9b', 'ressources._id': '5a65fd0363e2a32531ed9f9c' };
update = { '$set': { 'ressources.$': { '_id': '5a65fd0363e2a32531ed9f9c', qt: '153', unit: 'kg' } } };
options = {upsert:true};
collection.update(query,update,options);
Will give this ok result:
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": [
{
"_id": "5a65fd0363e2a32531ed9f9c",
"qt": 153,
"unit": "kg"
}
]
}]
How to make the same request work with these initial collections:
[{
"_id": "5a65fcf363e2a32531ed9f9b"
}]
OR
[{
"_id": "5a65fcf363e2a32531ed9f9b",
"ressources": []
}]
How to make the upsert work?
Does upsert works with entire document only?
Currently, I face this error:
The positional operator did not find the match needed from the query.
Thanks
I also tried to figure out how to do it. I found only one way:
fetch model by id
update array manually (via javascript)
save the model
Sad to know that in 2018 you still have to do the stuff like it.
UPDATE:
This will update particular element in viewData array
db.collection.update({
"_id": args._id,
"viewData._id": widgetId
},
{
$set: {
"viewData.$.widgetData": widgetDoc.widgetData
}
})
$push command will add new items
First, let me say that this question is slightly different to others I've seen regarding nested arrays in mongo.
Most of them ask how to get a specific element in a nested array. I want to get all elements of an array that has itself another array containing a given value.
I have documents, that look like this:
{
_id: something,
value: something,
items: [{
name: someItem,
price: somePrice,
stores:[
{name: storeName, url: someUrl },
{name: anotherStoreName, url: someOtherUrl}
]
},
{
name: someOtherItem,
price: someOtherPrice,
stores:[
{name: storeName, url: someUrl },
{name: yetAnotherStoreName, url: someOtherUrl}
]
}]
}
What I want is to get only the items elements that have a given store in the stores array.
This is, if I ask storeName, I should get both items in the example. But if I ask for anotherStoreName, I should only get the first element of the array items.
Searching for similar questions and trying the solutions I can only get the first matching element of items, but I want all the matching elements.
I'd appreciate any help.
You should use mongo aggregation to get result in following way.
First use $unwind to separate array of items and then add match condition.
db.collection.aggregate([{
$unwind: "$items"
}, {
$match: {
"items.stores.name": "storeName"
}
}, {
$project: {
_id: 0,
items: "$items" //you can add multiple fields here
}
}]);
You should use the aggregation framework to unwind elements on the items array and then treat them as individual documents.
db.data.aggregate([
{
$unwind: "$items"
},
{
$project: {
_id: 0,
items: "$items"
}
},
{
$match: {
"items.stores.name": "yetAnotherStoreName"
}
}
]);
$project just let you work with the important part of the document.
This works on the mongoshell be carefully when using your driver.
The result.
I have a Mongodb document that contains an an array that is deeply imbedded inside the document. In one of my action, I would like to return the entire document but filter out the elements of that array that don't match that criteria.
Here is some simplified data:
{
id: 123 ,
vehicles : [
{name: 'Mercedes', listed: true},
{name: 'Nissan', listed: false},
...
]
}
So, in this example I want the entire document but I want the vehicles array to only have objects that have the listed property set to true.
Solutions
Ideally, I'm looking for a solution using mongo's queries (e.g. `$unwind, $elemMatch, etc...) but I'm also using mongoose so solution that uses Mongoose is OK.
You could use aggregation framework like this:
db.test312.aggregate(
{$unwind:"$vehicles"},
{$match:{"vehicles.name":"Nissan"}},
{$group:{_id:"$_id",vehicles:{$push:"$vehicles"}}}
)
You can use $addToSet on the group after unwinding and matching by listed equals true.
Sample shell query:
db.collection.aggregate([
{
$unwind: "$vehicles"
},
{
$match: {
"vehicles.listed": {
$eq: true
}
}
},
{
$group: {
_id: "$id",
vehicles: {
"$addToSet": {
name: "$vehicles.name",
listed: "$vehicles.listed"
}
}
}
},
{
$project: {
_id: 0,
id: "$_id",
vehicles: 1
}
}
]).pretty();