MongoDB Aggregate pipeline with $group and $count on a Pointer reference returns wrong data - database

I have a set of data in MongoDB with parse-server in the following format-
Rating => objectId, user<_User>, rating...
_User => objectId, gender<m|f|nb|na>
I have been trying to group the data based on the user's gender to find out how many male, female, non-binary or N/A users have rated. user field in a pointer reference to _User. I am using the following aggregate pipeline.
const pipeline = [
{
lookup: {
from: '_User',
localField: 'user',
foreignField: 'objectId',
as: 'user'
}
},
{
unwind: { path: '$user' }
},
{
group: {
objectId: '$user.gender',
count: {
$sum: 1
}
}
}
]
const data = await new Query('Rating').aggregate(pipeline)
Result =>
[
{
"count": 54,
"objectId": "na"
},
{
"count": 405,
"objectId": null
},
{
"count": 27,
"objectId": "f"
},
{
"count": 540,
"objectId": "m"
}
],
However, returned data count doesn't match with actual data. The actual database has only 27 ratings with 1 f, 2 na, 24 m.
For MongoDB developers, objectId is equavalent to _id.
I am a novice to aggregation framework. What am I doing wrong?
Server Environment-
parse-server: 3.2.3
mongodb: 4.0.2

It is tricky because you need to understand how Parse Server stores the data inside the MongoDB. The following query should solve your problem:
const query = new Parse.Query('Rating');
const pipeline = [
{
project: {
objectId: 1,
userId: { $substr: ['$_p_user', '_User$'.length, -1] }
}
},
{
lookup: {
from: '_User',
localField: 'userId',
foreignField: '_id',
as: 'user'
}
},
{
unwind: { path: '$user' }
},
{
group: {
objectId: '$user.gender',
count: {
$sum: 1
}
}
}
];
return await query.aggregate(pipeline, { useMasterKey: true });

Related

MongoDB using skip and distinct in a query based on values inside an array

So I have document that is structure like this
_id: ObjectId('62bbe17d8fececa06b91873d')
clubName: 'test'
staff:[
'62bbe47f8fececa06b9187d8'
'624f4b56ab4f5170570cdba3' //IDS of staff members
]
A single staff can be assigned to multiple clubs so what I'm trying to achieve is to get all staff that has been assigned to at least one club and display them on a table on the front end, I followed this solution since distinct and skip can't be used on a single query but it just returned this:
[
{ _id: [ '624f5054ab4f5170570cdd16', '624f5054ab4f5170570cdd16' ] } //staff from club 1,
{ _id: [ '624f5054ab4f5170570cdd16', '624f9194ab4f5170570cded1' ] } //staff from club 2,
{ _id: [ '624f4b56ab4f5170570cdba3' ]} //staff from club 3
]
my desired outcome would be like this:
[ _id : ['624f5054ab4f5170570cdd16', '624f9194ab4f5170570cded1', '624f4b56ab4f5170570cdba3'] ]
here's my query:
const query = this.clubModel.aggregate(
[{ $group: { _id: '$staff' } }, { $skip: 0}, { $limit: 10}],
(err, results) => {
console.log(results);
},
);
the values returned are not distinct at all, is there an operation that can evaluate the value inside an array and make them distinct?
Here's my new query after adding the 'createdAt' field in my document structure:
const query = this.clubModel.aggregate([
{ $sort: { createdAt: -1 } },
{
$unwind: '$drivers',
},
{
$project: {
isActive: true,
},
},
{
$group: {
_id: 'null',
ids: {
$addToSet: '$drivers',
},
},
},
{
$project: {
_id: 0,
},
},
{
$skip: skip,
},
{
$limit: limit,
},
]);
Does this works for you, first UNWIND the staff array, and then group on "_id" as null and add staff values using $addToSet:
db.collection.aggregate([
{
"$unwind": "$staff"
},
{
"$group": {
"_id": "null",
"ids": {
"$addToSet": "$staff"
}
}
},
{
"$project": {
"_id": 0,
}
},
{
$skip: 0
},
{
$limit: 10
}
])
Here's the working link.

Looping through array to count in mongodb/mongoose

I have a user schema that contains a value called amputationInfo:
amputationInfo: [
{
type: String,
},
],
Here is an example of what that might look like in the database:
amputationInfo: [
"Double Symes/Boyd",
"Single Above-Elbow"
]
I have a review Schema that allows a user to leave a review, it contains a reference to the user who left it:
user: {
type: mongoose.Schema.ObjectId,
ref: 'User',
require: [true, 'Each review must have an associated user!'],
},
When a user leaves a review, I want to create an aggregate function that looks up the user on the review, finds their amputationInfo, loops through the array and adds up the total amount of users that contain "Double Symes/Boyd", "Single Above-Elbow"
So if we have 3 users and their amputationInfo is as follows:
amputationInfo: [
"Double Symes/Boyd",
"Single Above-Elbow"
]
amputationInfo: [
"Single Above-Elbow"
]
amputationInfo: []
The return from the aggregate function will count each term and add one to the corresponding value and look something like this:
[
{
doubleSymesBoyd: 1,
singleAboveElbow: 2
}
]
Here is what I have tried, but I just don't know enough about mongoDB to solve the issue:
[
{
'$match': {
'prosthetistID': new ObjectId('6126ca6148f34c00189f86f5')
}
}, {
'$lookup': {
'from': 'users',
'localField': 'user',
'foreignField': '_id',
'as': 'userInfo'
}
}, {
'$unwind': {
'path': '$userInfo'
}
}
]
After the $unwind, the resulting object has a userInfo key, that contains an amputationInfo array nested:
You can have following stages
$unwind to deconstruct the array
first $group to get the sum of each category
second $group to push into one document and make it as key value pair
$arrayToObject to get the desired output
$replaceRoot to make the data output into root
Here is the code
db.collection.aggregate([
{ "$unwind": "$userInfo.amputationInfo" },
{
"$group": {
"_id": "$userInfo.amputationInfo",
"count": { "$sum": 1 }
}
},
{
$group: {
_id: null,
data: { $push: {
k: "$_id",
v: "$count"
}
}
}
},
{ $project: { data: { "$arrayToObject": "$data" } } },
{ "$replaceRoot": { "newRoot": "$data" } }
])
Working Mongo playground

Get number of followers for specific page in Mongodb

let's say I have a collection called pages as
{
_id: "pageid",
name: "Mongodb"
},
{
_id: "pageid2",
name: "Nodejs"
}
and user collection as follows
{
_id : "userid1",
following: ["pageid"],
...
},
{
_id : "userid2",
following: ["pageid", "pageid2"],
...
}
how could I make a query to retrieve the pages information along with the number of users follow each page in mongodb, expected result as follows
[
{
_id: "pageid",
name: "MongoDB",
followers: 2
},
{
_id: "pageid2",
name: "Nodejs",
followers: 1
},
]
You can use $lookup and $size to count total followers,
db.pages.aggregate([
{
$lookup: {
from: "user",
localField: "_id",
foreignField: "following",
as: "followers"
}
},
{
$addFields: {
followers: { $size: "$followers" }
}
}
])
Playground

How to retrieve in the same query user's followers and users I already follow in mongodb

I'm using mongodb with this collections: User and Follow. User collection has users and follow has the followers mapping.
Try to run this query but only get the user's followers and who I'm following too:
users = User.collection.aggregate([{
"$lookup": {
from: "follows",
localField: "_id",
foreignField: "followable_id",
as: "follow"
}
},
{
"$match": {
"follow.followable_type": "User",
"follow.user_id": BSON::ObjectId.from_string(userid)
}
},
{
"$group": {
_id: "$follow.followable_id",
count: {
"$sum": 1
},
data: {
"$addToSet": "$$ROOT"
},
}
},
{
"$project": {
_id: 1,
data: 1,
count: 1,
follow: {
'$cond': {
'if': {
'$eq': ["$count", 2]
},
then: true,
else: false
}
}
}
},
{
"$skip": 0 * 10
},
{
"$limit": 10
}
])
users.each do |user|
puts "#{user['data'].first['name']}: #{user['follow']}: #{user['count']}"
end
How to return in the same query the users I'm following?
the output:
Diego_Lukmiller: false: 1
Marianno: false: 1
kah: false: 1
Fausto Torres: false: 1
I solved/fixed it by doing in three steps (written in ruby/mongoid):
//get users the user X follows
users = Follow.collection.aggregate([
{ "$match": { "followable_id": BSON::ObjectId.from_string(userid) } },
{ "$lookup": { from: "users", localField: "user_id", foreignField: "_id", as: "user" } },
{ "$unwind": "$user" },
{ "$sort": {"created_at":-1} },
{ "$skip": page * 10 },
{ "$limit": 10 },
{ "$group":{ _id: "$followable_id", data: {"$addToSet": "$user._id"} } }
])
userids=users.blank? ? [] : users.first['data'].to_a
//get users I follow
f=Follow.collection.aggregate([
{"$match":{user_id: BSON::ObjectId.from_string(meid), followable_id: {"$in":userids}}},
{"$group":{ _id: nil, data: {"$addToSet": "$followable_id"}}}
])
//make intersection
users = User.collection.aggregate([
{ "$lookup": { from: "follows", localField: "_id", foreignField: "user_id", as: "follow" } },
{ "$unwind": "$follow" },
{ "$match": { "follow.followable_type": "User","follow.followable_id": BSON::ObjectId.from_string(userid) } },
{ "$group":{ _id: "$_id", data:{"$addToSet": "$$ROOT"}} },
{ "$unwind": "$data" },
{ "$project": {
_id: 1,data: 1,count: 1,
follow: {
'$cond': {
"if": { '$in': [ "$_id", f.first['data'] ] }, "then": true,
"else": false
}
}
}
},
{ "$skip": page * 10 },
{ "$limit": 10 }
])
The output:
name: Fausto Torres; Do I follow too? false; Counter that's make the magic: 1
name: kah; Do I follow too? false; Counter that's make the magic: 1
name: Marianno; Do I follow too? true; Counter that's make the magic: 2
name: Diego_Lukmiller; Do I follow too? true; Counter that's make the magic: 2
The query performs pretty fast and looks simple. Is there way / How do I solve it in a single query?!?

mongodb - using join on a local variable

I'm using node.js and mongodb, I have an array of objects which holds the names of an id. Let's say below is my array
let names = [
{ value: 1, text: 'One' },
{ value: 2, text: 'Two' },
{ value: 3, text: 'Three' },
{ value: 4, text: 'Gour' }
]
And this is my query result of a collection using $group which gives me the distinct values.
[
{ _id: { code: '1', number: 5 } },
{ _id: { code: '2', number: 5 } },
{ _id: { code: '3', number: 2 } },
{ _id: { code: '4', number: 22 } },
]
$lookup let's us to join the data from a different collection, but in my case I have an array which holds the text value for each of the codes which I got from the query.
Is there a way we can map the text from the array to the results from mongodb?
Any help will be much appreciated.
EDIT
MongoDB query which I was trying
db.collection.aggregate([
{
$match: {
_Id: id
}
},
{
$lookup: {
localField: "code",
from: names,
foreignField: "value",
as: "renderedNames"
}
},
{
"$group" : {
"_id": {
code: "$code",
number: "$number"
}
}
}
]);
Local variable lives in nodejs app, and mongodb knows nothing about it.
It looks like it belongs to representation layer, where you want to show codes as meaningful names. The mapping should be done there. I believe find is the most suitable here:
names.find(name => name.code === doc._id.code).text
If the names are not truly variable but quite constant, you can move it to own collection, e.g. codeNames:
db.codeNames.insert([
{ _id: "1", text: 'One' },
{ _id: "2", text: 'Two' },
{ _id: "3", text: 'Three' },
{ _id: "4", text: 'Gour' }
]);
and use $lookup as following:
db.collection.aggregate([
{
$match: {
_Id: id
}
},
{
"$group" : {
"_id": {
code: "$code",
number: "$number"
}
}
},
{
$lookup: {
localField: "_id.code",
from: "codeNames",
foreignField: "_id",
as: "renderedNames"
}
}
]);
If none of the above suit your usecase, you can pass the names to the database in each request to map names db-side, but you must be really really sure you cannot use 2 previous options:
db.collection.aggregate([
{
$match: {
_Id: id
}
},
{
"$group" : {
"_id": {
code: "$code",
number: "$number"
}
}
},
{
$project: {
renderedNames: { $filter: {
input: [
{ value: "1", text: 'One' },
{ value: "2", text: 'Two' },
{ value: "3", text: 'Three' },
{ value: "4", text: 'Gour' }
],
as: "name",
cond: { $eq: [ "$$name.value", "$_id.code" ] }
}
}
}
},
]);
As a side note, I find $match: {_Id: id} quite confusing, especially followed by $group. If _Id is _id, it is unique. You can have no more than 1 document after this stage, so there is not too much to group really.

Resources