MongoDB aggregate nested objects - arrays

I have two collections in my mongoDb database, one called media, one called users.
Documents in the media collection:
{"_id":{"$oid":"6379204a8cf5677554c26c1b"},"_partition":"6378eb74f6613d5d4192da79","name":"silas test"}
{"_id":{"$oid":"6378eb74f6613d5d4192da79"},"_partition":"6378eb74f6613d5d4192da79","name":"test media"}
Document in the users collection:
{
"_id":{
"$oid":"6379ee6c770a8f43afc8e3e4"},
"_partition":"6378eb74f6613d5d4192da79",
"types":[
{
"mediaReference":[
{"$oid":"6379204a8cf5677554c26c1b"}
],
"mediatype":"podcast"
},
{
"mediaReference":[
{"$oid":"6378eb74f6613d5d4192da79"}
],
"mediatype":"movie"
}
],
"username":"silas"
}
Now im looking for a way to use the mongoDb aggregation pipeline to get the following result:
{
"_id":{
"$oid":"6379ee6c770a8f43afc8e3e4"},
"_partition":"6378eb74f6613d5d4192da79",
"types":[
{
"mediaReference":[
{
"_id": {
"$oid":"6379204a8cf5677554c26c1b"
},
"_partition":"6378eb74f6613d5d4192da79",
"name":"silas test"
}
],
"mediatype":"podcast"
},
{
"mediaReference":[
{
"_id": {
"$oid":"6378eb74f6613d5d4192da79"
},
"_partition":"6378eb74f6613d5d4192da79",
"name":"test media"
}
],
"mediatype":"movie"
}
],
"username":"silas"
}
Basically im searching for a way to paste the refferenced document into the mediaReference object inside the users document.
i would appreciate any help
EDIT:
I've found the solution:
db.users.aggregate([{
$unwind: {
path: '$types'
}
}, {
$lookup: {
from: 'media',
localField: 'types.mediaReference',
foreignField: '_id',
as: 'types.mediaReference'
}
}, {
$group: {
_id: '$_id',
merged: {
$first: '$$ROOT'
},
types: {
$push: '$types'
}
}
}, {
$addFields: {
'merged.types': '$types'
}
}, {
$replaceRoot: {
newRoot: '$merged'
}
}])

Related

Mongo Aggregation with nested array of Id´s

I'm having a Model which is structured similar to this:
{
"_id": ObjectId("5c878c5c18a4ff001b981zh5"),
"books": [
ObjectId("5d963a7544ec1b122ab2ddc"),
ObjectId("5d963be01f663d168f8ea4dc"),
ObjectId("5d963bcb1f663d168f8ea2f4"),
ObjectId("5d963bdf1f663d16858ea7c9"),
}
Now I want to use the aggregation framework to get a list of only the populated books, like:
{ _id: ObjectId("5d963a7544ec1b122ab2ddc"), title: ...., ... },
..
.aggregate([
{
$lookup: {
from: 'books',
let: { books: '$books' },
pipeline: [{ $match: { $expr: { _id: { $in: ['_id', '$$books'] } } } }],
as: 'bookInfos'
}
},
{ $unwind: '$bookInfos' },
{ $replaceRoot: { newRoot: '$bookInfos' } }
])
I am not too sure about your question, but I think this might be what you're looking for.
So this query worked for me:
{
$match: {
_id: user._id,
},
},
{
$lookup: {
from: "books",
localField: "books",
foreignField: "_id",
as: "booksInfo",
},
},
{ $unwind: "$booksInfo" },
{
$replaceRoot: {
newRoot: "$booksInfo",
},
},
Thanks #zishone. Somehow your query returned all the books available in the db and not only the ones from the User Model, but it works as desired when looking up the documents with local and foreignField.

aggregation in nodejs resulting in nested json, can I get it without nesting, taking only one data _id from all collections

aggregation in nodejs resulting in nested json, can I get it without nesting, taking only one data _id from all collections. Is there any possibility to get the data without a nested json
I was trying aggregation in nodejs with the below code. I got the output as given in output session below. But I would like to get the output as expected output, since I cant use looping on looping
Student.aggregate([
{
$match: { name: 'abcd'}
},
{
$lookup:{
from:'teachers',
pipeline: [
{
$match: { name: 'pqrs' }
},
{
$project:{
"_id":1
}
}
],
as: "teacherLookup"
}
},
{
$lookup:
{
from:'subjects',
pipeline: [
{
$match: { name: 'computer' }
},
{
$project:{
"_id":1
}
}
],
as: "subjectLookup"
}
}
])
output
[
{
_id: '52301c7878965455d2a4',
teacherLookup: [ '5ea737412589688930' ],
subjectLookup: [ '5ea745821369999917' ]
}
]
I am expecting the output as (without nested json)
[
{
studentId: '5ea1c7878965455d2a4',
teacherId: '5ea737412589688930' ,
subjectId: '5ea745821369999917'
}
]
You can use $arrayElemAt to get the first element from the array.
Student.aggregate([
{
$match: { name: "abcd" },
},
{
$lookup: {
from: "teachers",
pipeline: [
{
$match: { name: "pqrs" },
},
{
$project: {
_id: 1,
},
},
],
as: "teacherId",
},
},
{
$lookup: {
from: "subjects",
pipeline: [
{
$match: { name: "computer" },
},
{
$project: {
_id: 1,
},
},
],
as: "subjectId",
},
},
{
$project: {
teacherId: { $arrayElemAt: ["$teacherId", 0] },
subjectId: { $arrayElemAt: ["subjectId", 0] },
},
}
]);

Lookup VS Lookup with pipeline MongoDB (Performace & How it internally works)

I'm making a blog and have an query about which would give me better performace, simple lookup or lookup with pipeline because sometime simple lookup gave me fast result and sometime pipleline lookup. So, I am bit confused now which one to use or where to use. Suppose I have 2 collection, user and comment collection.
// Users Collection
{
_id: "MONGO_OBJECT_ID",
userName: "Web Alchemist"
}
// Comments Collection
{
_id: "MONGO_OBJECT_ID",
userId: "USER_MONGO_OBJECT_ID",
isActive: "YES", // YES or NO
comment: "xyz"
}
Now I want to Lookup from users collection to comments, which one would be better for this. I made two query which giving me same result.
[
{
$match: { _id: ObjectId("5d68c019c7d56410cc33b01a") }
},
{
$lookup: {
from: "comments",
as: "comments",
localField: "_id",
foreignField: "userId"
}
},
{
$unwind: "$comments"
},
{
$match: {
"comments.isActive": "YES"
}
},
{ $limit: 5},
{
_id: 1, userName: 1, comments: { _id: "$comments._id", comment: "$comments.comment"}
},
{
$group: {
_id: "$_id",
userName: { '$first': '$userName' },
comments: { $addToSet: "comments"}
}
}
]
OR
[
{
$match: { _id: ObjectId("5d68c019c7d56410cc33b01a") }
},
{
$lookup: {
from: "comments",
as: "comments",
let: { userId: "$_id" },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $eq: ['$userId', '$$userId'] },
{ $eq: ['$isActive', 'YES'] }
]
}
}
},
{ limit: 5 },
{
$project: { _id: 1, comment: 1 }
}
]
}
}
]

MongoDB Aggregate ObjectId inside Array of embedded Documents

I have seen a ton of question about the $lookup aggregator for arrays of ObjectIds but I can't seem to find anything about when the ObjectId is inside an array of embedded documents.
I have the follow document inside a mongodb database:
_id: ObjectId('...')
Alarms: [
{
Gateway: ObjectId('...')
Port: 1
},
{
Gateway: ObjectId('...')
Port: 2
}
]
I would like to have the following:
_id: ObjectId('...')
Alarms [
{
Gateway: ...(Gateway Object),
Port: 1
},
{
Gateway: ...(Gateway Object),
Port: 2
}
]
I have tried the following with no success:
$lookup: {
from: 'Gateway',
localField: 'Alarms.Gateway',
foreignField: '_id',
as: 'Alarms.Gateway'
}
But this gives me the following result:
_id: ObjectId('...')
Alarms [
{
Gateway: {
...(Gateway Object)
}
Port: 1
}
]
Please try the below queries :
If you don't want the object which doesn't have match in Gateway collection exist in Alarms array in final result :
db.Alarms.aggregate([{ $unwind: '$Alarms' }, {
$lookup: {
from: 'Gateway',
localField: 'Alarms.Gateway',
foreignField: '_id',
as: 'Alarms.Gateway'
}
}, { $match: { 'Alarms.Gateway': { $ne: [] } } },
{ $addFields: { 'Alarms.Gateway': { $arrayElemAt: ['$Alarms.Gateway', 0] } } },
{ $group: { _id: '$_id', Alarms: { $push: '$Alarms' } } }
])
Test : MongoDB-Playground
Otherwise, if you want all objects in Alarms array to be returned irrespective of whether there is a match in Gateway or not :
db.Alarms.aggregate([{ $unwind: '$Alarms' }, {
$lookup: {
from: 'Gateway',
localField: 'Alarms.Gateway',
foreignField: '_id',
as: 'Alarms.GatewayObj'
}
}, { $addFields: { 'Alarms.Gateway': { $cond: [{ $ne: ['$Alarms.GatewayObj', []] }, { $arrayElemAt: ['$Alarms.GatewayObj', 0] }, '$Alarms.Gateway'] } } },
{ $project: { 'Alarms.GatewayObj': 0 } },
{ $group: { _id: '$_id', Alarms: { $push: '$Alarms' } } }
])
Test : MongoDB-Playground
Difference between two queries would be one will return below object in Alarms array (Vs) one don't.
{
"Gateway": ObjectId("5e2b5425d02e05b6940de2fb"),
"Port": 2
}

How to use $text search inside $lookup pipeline

I have the next collection for exaple:
// vehicles collection
[
{
"_id": 321,
manufactor: SOME-OBJECT-ID
},
{
"_id": 123,
manufactor: ANOTHER-OBJECT-ID
},
]
And I have a collection named tables:
// tables collection
[
{
"_id": SOME-OBJECT-ID,
title: "Skoda"
},
{
"_id": ANOTHER-OBJECT-ID,
title: "Mercedes"
},
]
As you can see, the vehicles collection's documents are pulling data from the
tables's collection ducments - the first document in the vehicles collection has a manufactor
id which is getting pulled from the tables collection and named Skoda.
That is great.
When I am querying the DB using aggregate I can able to easily pull the remote data from the remote collections
respectively - without any problem.
I can also easily make rules and limitations like $project, $sort, $skip, $limit and others.
But I want to display to the user only those vehicles that are manufcatord by Mercedes.
Since Mercedes is not mentioned in the vehicles collection, but only its ID, the $text $search would not
return with the right results.
This is the aggregate pipeline that I provide:
[
{
$match: {
$text: {
$search: "Mercedes"
}
}
},
{
$lookup: {
from: "tables",
let: {
manufactor: "$manufactor"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$_id", "$$manufactor"
]
}
}
},
{
$project: {
title: 1
}
}
],
as: "manufactor"
},
},
{
$unwind: "$manufactor"
},
{
$lookup: {
from: "tables",
let: {
model: "$model"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$_id", "$$model"
]
}
}
},
{
$project: {
title: 1
}
}
],
as: "model"
},
},
{
$unwind: "$model"
},
{
$lookup: {
from: "users",
let: {
joined_by: "$_joined_by"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$_id", "$$joined_by"
]
}
}
},
{
$project: {
personal_info: 1
}
}
],
as: "joined_by"
},
},
{
$unwind: "$joined_by"
}
]
As you can see I am using the $text and $search $match at the first stage in the pipleline - otherwise
MongoDB will throw an error.
But this $text $search object searhed only in the origin collection - the vehicles collection.
Is there a way to tell MongoDB to search in the remote collection with the $text and $search method
and then put in the aggregate only results that are matching both?
UPDATE
When I am doing this instead:
{
$lookup: {
from: "tables",
pipeline: [
{
$match: {
$text: {
$search: "Mercedes"
}
}
},
{
$project: {
title: 1
}
}
],
as: "manufactor"
},
},
This is what I receive:
MongoError: pipeline requires text score metadata, but there is no text score available
if you are using one of the affected versions in this thread, you need to update your mongodb server.
As you can see the issue was fixed in version 4.1.8

Resources