I'm running a query on Mongodb to get the combine data from two different collections: User and Store.
User collection has a property named as store_ids, which is an array that contains a list of ObjectIds of each store that the User has access to.
I'm trying to add the name of each store in the query result.
Example:
User Document:
{
_id: '58ebf8f24d52e9ab59b5538b',
store_ids: [
ObjectId("58dd4bb10e2898b0057be648"),
ObjectId("58ecd57d1a2f48e408ea2a30"),
ObjectId("58e7a0766de7403f5118afea"),
]
}
Store Documents:
{
_id: "58dd4bb10e2898b0057be648",
name: "Store A",
},
{
_id: "58ecd57d1a2f48e408ea2a30",
name: "Store B",
},
{
_id: "58e7a0766de7403f5118afea",
name: "Store C"
}
I'm looking for a query that returns an output like this:
{
_id: '58ebf8f24d52e9ab59b5538b',
stores: [
{
_id: ObjectId("58dd4bb10e2898b0057be648"),
name: "Store A"
},
{
id: ObjectId("58ecd57d1a2f48e408ea2a30"),
name: "Store B"
},
{
_id: ObjectId("58e7a0766de7403f5118afea"),
name: "Store C"
}
]
}
I've already tried operations like $map and $set. I don't know if I'm applying them in the right way because they didn't work for my case.
You can use an aggregate query:
db.users.aggregate([
{
$lookup: {
from: "stores", //Your store collection
localField: "store_ids",
foreignField: "_id",
as: "stores"
}
},
{
$project: {
store_ids: 0
}
}
])
You can see a working example here: https://mongoplayground.net/p/ICsEEsmRcg0
We can achieve this with a simple $lookup and with $project.
db.user.aggregate({
"$lookup": {
"from": "store",
"localField": "store_ids",
"foreignField": "_id",
"as": "stores"
}
},
{
"$project": {
store_ids: 0
}
})
$lookup will join with store table on with the store_ids array where the _id matches
$project removes the store_ids array from the resulting objects
Playground
Related
I am having this mongo DB query which queries a collection called songs and for each song, returns the respective album associated:
db.songs.aggregate([{
$lookup: {
from: "albums",
let: { album: '$album' },
as: "album",
pipeline: [{
$match: {
$expr: {
$and: [
{ $eq: ['$albumId', '$$album._id'] },
{ $eq: ['$status', 'Draft'] },
]
}
}
}]
}
}])
In the above query, my intention was to return a song only if the album was in Draft status, but in contrast, it returns all songs, and for the ones for which the album is not in Draft, it just returns an empty array inside the lookup. How can I not return the song document at all if the album is not in Draft?
Additionally, is it possible to flatten the results in the document? ie, merge all the fields of albums into the song document?
Once you perform the $lookup you can filter out the documents with an empty array:
{ $match: { album: { $ne: [] } }}
Then there is an example in the MongoDB documentation for the $mergeObjects operator that is very similar to your case. Assuming that each song belongs to one album, put together your aggregation pipeline may look like this:
db.songs.aggregate([
{
$lookup: {
from: "albums",
let: { album: '$album' },
as: "album",
pipeline: [{
$match: {
$expr: {
$and: [
{ $eq: ['$albumId', '$$album._id'] },
{ $eq: ['$status', 'Draft'] },
]
}
}
}]
}
},
{ $match: { album: { $ne: [] } }},
{
$replaceRoot: { newRoot: { $mergeObjects: [ { $arrayElemAt: [ "$album", 0 ] }, "$$ROOT" ] } }
},
{ $project: { album: 0 } }
])
You may want to experiment going in the other direction: find albums in status = Draft then get the songs:
db.album.aggregate([
{$match: {"status":"Draft"}}
,{$lookup: {from: "song",
localField: "album", foreignField: "album",
as: "songs"}}
// songs is now an array of docs. Run $map to turn that into an
// array of just the song title, and overwrite it (think x = x + 1):
,{$addFields: {songs: {$map: {
input: "$songs",
in: "$$this.song"
}} }}
]);
If you have a LOT of material in the song document, you can use the fancier $lookup to cut down the size of the docs in the lookup array -- but you still need the $map to turn it into an array of strings.
db.album.aggregate([
{$match: {"status":"Draft"}}
,{$lookup: {from: "song",
let: { aid: "$album" },
pipeline: [
{$match: {$expr: {$eq:["$album","$$aid"]}}},
{$project: {song:true}}
],
as: "songs"}}
,{$addFields: {songs: {$map: {
input: "$songs",
in: "$$this.song"
}} }}
]);
Mongo V5.03
I am using Compass for building a pipeline. And I am stuck here.
collection: hello
{
"_id" : "...",
"collection_name" : "world"
}
collection: world
{
"_id" : "..."
}
while building a pipeline with mongodb aggregation, to call another collection, we can use $lookup operator. Syntax of $lookup looks like this :
{
* from: The target collection.
* localField: The local join field.
* foreignField: The target join field.
* as: The name for the results.
* pipeline: The pipeline to run on the joined collection.
* let: Optional variables to use in the pipeline field stages.
}
For one time use, I can directly write { from : 'world' , ...}. But I want to do this instead { from : '$collection_name', ... } so that I can keep calling field value because that collection_names field is an array which I $unwind it.
Comeon tips, suggestions, solution
We probably do not have method to $lookup from dynamic collection name as of now. However, if we have already known all possibilities of the collection names, we may use multiple $lookup to perform conditional $lookup and use $setUnion to join the lookup results together.
Here is an example of knowing all 2 possibilities, world and foo; syntax in MongoDB 5.0+:
db.hello.aggregate([
{
"$lookup": {
"from": "world",
"localField": "key",
"foreignField": "_id",
"let": {
c: "$collection_name"
},
"pipeline": [
{
$match: {
$expr: {
$eq: [
"$$c",
"world"
]
}
}
}
],
"as": "worldLookup"
}
},
{
"$lookup": {
"from": "foo",
"localField": "key",
"foreignField": "_id",
"let": {
c: "$collection_name"
},
"pipeline": [
{
$match: {
$expr: {
$eq: [
"$$c",
"foo"
]
}
}
}
],
"as": "fooLookup"
}
},
{
"$project": {
collection_name: 1,
key: 1,
allLookup: {
"$setUnion": [
"$worldLookup",
"$fooLookup"
]
}
}
}
])
Here is the Mongo playground for your reference.
let's say I have a collection called pages as
{
_id: "pageid",
name: "Mongodb"
},
{
_id: "pageid2",
name: "Nodejs"
}
and user collection as follows
{
_id : "userid1",
following: ["pageid"],
...
},
{
_id : "userid2",
following: ["pageid", "pageid2"],
...
}
how could I make a query to retrieve the pages information along with the number of users follow each page in mongodb, expected result as follows
[
{
_id: "pageid",
name: "MongoDB",
followers: 2
},
{
_id: "pageid2",
name: "Nodejs",
followers: 1
},
]
You can use $lookup and $size to count total followers,
db.pages.aggregate([
{
$lookup: {
from: "user",
localField: "_id",
foreignField: "following",
as: "followers"
}
},
{
$addFields: {
followers: { $size: "$followers" }
}
}
])
Playground
Considering the following document "Backpack", each slots is a piece of said backpack, and each slot has a contents describing various items and a count of them.
{
_id: "backpack",
slots: [
{
slot: "left-pocket",
contents: [
{
item: "pen",
count: 3
},
{
item: "pencil",
count: 2
},
]
},
{
slot: "right-pocket",
contents: [
{
item: "bottle",
count: 1
},
{
item: "eraser",
count: 1
},
]
}
]
}
The item field is the _id of an item of another collection, e.g.:
{
_id: "pen",
color: "red"
(...)
},
Same for pen, pencil, bottle, eraser, etc.
I want to make a $lookup so I can fill in the item's data, but I'm not finding a way of having the lookup's as be the same place as the item. That is:
db.collection.aggregate({
{
$lookup: {
from: 'items',
localField: 'slots.contents.item',
foreignField: '_id',
as: 'convertedItems', // <=== ISSUE
},
},
})
Problem is that as being named convertedItems means the document gets an array of items in the root of the document called 'convertedItems', like this:
{
_id: "backpack",
slots: [ (...) ],
convertedItems: [ (...) ]
}
How can I tell $lookup to actually use the localField as the place to append the data?
That is, make document become:
{
_id: "backpack",
slots: [
{
slot: "left-pocket",
contents: [
{
item: "pen", // <== NOTE
count: 3, // <== NOTE
_id: "pen",
color: "red"
(...)
},
{
item: "pencil", // <== NOTE
count: 2, // <== NOTE
_id: "pencil",
color: "blue"
(...)
},
]
},
(...)
Note: At this point, if have entire data of item, doesn't matter if item property is kept, but count must remain.
I can't manually do $addFields with arrayElemAt because the number of items in slots is not fixed.
Extra Info: I'm using MongoAtlas Free so assume MongoDB 4.2+ (no need to unwind arrays for $lookup).
PS: I thought now of just leaving as root item (e.g. "convertedItems") and on the code that receives the API, when looping through the items, I do Array.find on the "convertedItems" per the the _id using the item. I'll keep the question as I'm curious on how to do on MongoDB side
When you use $lookup, there is a single query in the related collection for each document in the source pipeline, not a query per value in the source document.
If you want each item looked up separately, you'll need to unwind the arrays so each document in the pipeline contains a single item, do the lookup, and then group to rebuild the arrays.
db.collection.aggregate([
{$unwind: "$slots"},
{$unwind: "$slots.contents"},
{$lookup: {
from: "items",
localField: "slots.contents.item",
foreignField: "_id",
as: "convertedItems"
}},
{$group: {
_id: "$slots.slot",
root: {$first: "$$ROOT"},
items: {
$push: {
$mergeObjects: [
"$slots.contents",
{$arrayElemAt: ["$convertedItems", 0]}
]
}},
}},
{$addFields: {"root.slots.contents": "$items"}},
{$replaceRoot: {newRoot: "$root"}},
{$group: {
_id: "$_id",
root: {$first: "$$ROOT"},
slots: {$push: "$slots"}
}},
{$addFields: {"root.slots": "$slots"}},
{$replaceRoot: {newRoot: "$root"}},
{$project: { convertedItems: 0}}
])
Playground
unwind makes your collection explode, Also you can't specify in place of
'as', So you need to add additional stages like addFields, filters to
get required o/p
As I've commented, your requirement has a bit to do in order to match main doc's elements with $lookup result, maybe this can be easily done by code, but if it has to be done by query, using this query you'll be working on same no.of docs as what you've in collection quiet opposite to unwind as it would explode you docs when having nested arrays like what you've now, As in general this is a bit complex try to use $match as first stage to filter docs if needed for better performance. Additionally you can use $explain to get to know about your query performance.
Query :
db.Backpack.aggregate([
/** lookup on items collection & get matched docs to items array */
{
$lookup: {
from: "items",
localField: "slots.contents.item",
foreignField: "_id",
as: "items"
}
},
/** Iterate on slots & contents & internally filter on items array to get matched doc for a content object &
* merge the objects back to respective objects to form the same structure */
{
$project: {
slots: {
$map: {
input: "$slots",
in: {
$mergeObjects: [
"$$this",
{
contents: {
$map: {
input: "$$this.contents",
as: "c",
in: {
$mergeObjects: [
"$$c",
{
$let: {
vars: {
matchedItem: {
$arrayElemAt: [
{
$filter: {
input: "$items",
as: "i",
cond: {
$eq: [
"$$c.item",
"$$i._id"
]
}
}
},
0
]
}
},
in: {
color: "$$matchedItem.color"
}
}
}
]
}
}
}
}
]
}
}
}
}
}
])
Test : MongoDB-Playground
The idea here is to return an array of documents of the users' followers with the information if this user is a friend of that follower or not.
So far I have:
db.getCollection('users').aggregate([
{ $match: { _id: ObjectId("588877d82523b4395039910a") } },
{ $lookup: {
from: 'users',
localField: 'followers',
foreignField: '_id',
as: 's_followers'
}
},
{
$project: {
"s_followers._id": 1,
"s_followers.isFriend": {
$in: ["s_followers.id",
{ $setIntersection: ["$friends", "$followers"] }
]}
}
}
])
But the "s_followers.id" used in the $in operator doesn't seem to retrieve the _id information from the follower, so it always returns false.
When I use a ObjectId directly, I got the result I want:
"s_followers.isFriend": {
$in: [ObjectId("588877d82523b4395039910a"),
{ $setIntersection: ["$friends", "$followers"] }
]}
But I really need this ID to be a reference to the follower _id.
Expected result would be something like:
{
"_id" : ObjectId("588877d82523b4395039910a"),
"s_followers" : [
{
"_id" : ObjectId("5888687e56be8f172844d96f"),
"isFriend" : true
},
{
"_id" : ObjectId("5888ca27d79b8b03949a6e8c"),
"isFriend" : false
}
]
}
Thanks for your help!
UPD: A different approach (maybe easier), would be to use the ID of the user that I have (the one used on $match), but I would still need to get the reference for the follower's follower array
db.getCollection('users').aggregate([
{ $match: { _id: ObjectId("588877d82523b4395039910a") } },
{ $lookup: {
from: 'users',
localField: 'followers',
foreignField: '_id',
as: 's_followers'
}
}, {
$project: {
"firstName": 1,
"s_followers._id": 1,
"s_followers.firstName": 1,
"s_followers.followers": 1,
"s_followers.isFriend": { $in: [ObjectId("588877d82523b4395039910a"), "$s_followers.followers"] }
}
}
])
UPD2: The user data structure (the part that matters)
{
followers: [{ type: mongoose.Schema.Types.ObjectId, ref: "User" }],
friends: [{ type: mongoose.Schema.Types.ObjectId, ref: "User" }],
}
FOR VERSION 3.4.0+
Ok, just got it, I'll post here the code and my understanding of it:
db.getCollection('users').aggregate([
{ $match: { _id: ObjectId("588877d82523b4395039910a") } },
{ $lookup: {
from: 'users',
localField: 'followers',
foreignField: '_id',
as: 's_followers'
}
}, {
$project: {
"firstName": 1,
"s_followers._id": 1,
"s_followers.firstName": 1,
"s_followers.followers": 1,
}
}, {
$unwind: "$s_followers"
}, {
$project: {
"firstName": "$s_followers.firstName",
"isFriend": { $in: [ObjectId("588877d82523b4395039910a"), "$s_followers.followers"] }
}
}
])
My understanding of it:
$match: match the user I'm intended to get the followers of.
$lookup: found each follower detail
$project: select the information I want to return, also get the follower list of each follower
$unwind: create a different document for each follower
With the array unwinded, I can refer to the follower's followers array, and find my object id in it :)
In my example use followers friends list to check current user is friend or not. as {$arrayElemAt:["$s_followers.friends",0]} if want to find in followers then can use "$s_followers.followers"
You can try it.
db.getCollection('user').aggregate([
{ $match: { _id: ObjectId("5714d190e6128b7e7f8d9008") } },
{$unwind:"$followers"},
{ $lookup: {
from: 'user',
localField: 'followers',
foreignField: '_id',
as: 's_followers'
}
},
{$project:{
firstName:1,
s_followers:{$arrayElemAt:["$s_followers",0]},
isFriend:{$cond:[{
$anyElementTrue:{
$map: {"input": {$arrayElemAt:["$s_followers.friends",0]},
"as": "el",
"in": { "$eq": [ "$$el", "$_id" ] }
}
}
},true,false]}
}
},
{$group:{
_id:"$_id",
s_followers:{$push:{_id:"$s_followers._id",isFriend:"$isFriend"}}
}
}
])