I have a Mongodb document that contains an an array that is deeply imbedded inside the document. In one of my action, I would like to return the entire document but filter out the elements of that array that don't match that criteria.
Here is some simplified data:
{
id: 123 ,
vehicles : [
{name: 'Mercedes', listed: true},
{name: 'Nissan', listed: false},
...
]
}
So, in this example I want the entire document but I want the vehicles array to only have objects that have the listed property set to true.
Solutions
Ideally, I'm looking for a solution using mongo's queries (e.g. `$unwind, $elemMatch, etc...) but I'm also using mongoose so solution that uses Mongoose is OK.
You could use aggregation framework like this:
db.test312.aggregate(
{$unwind:"$vehicles"},
{$match:{"vehicles.name":"Nissan"}},
{$group:{_id:"$_id",vehicles:{$push:"$vehicles"}}}
)
You can use $addToSet on the group after unwinding and matching by listed equals true.
Sample shell query:
db.collection.aggregate([
{
$unwind: "$vehicles"
},
{
$match: {
"vehicles.listed": {
$eq: true
}
}
},
{
$group: {
_id: "$id",
vehicles: {
"$addToSet": {
name: "$vehicles.name",
listed: "$vehicles.listed"
}
}
}
},
{
$project: {
_id: 0,
id: "$_id",
vehicles: 1
}
}
]).pretty();
Related
I'm struggling to write a Mongo UpdateMany statement that can reference and update an object within an array.
Here I create 3 documents. Each document has an array called innerArray always containing a single object, with a single date field.
use test;
db.innerArrayExample.insertOne({ _id: 1, "innerArray": [ { "originalDateTime" : ISODate("2022-01-01T01:01:01Z") } ]});
db.innerArrayExample.insertOne({ _id: 2, "innerArray": [ { "originalDateTime" : ISODate("2022-01-02T01:01:01Z") } ]});
db.innerArrayExample.insertOne({ _id: 3, "innerArray": [ { "originalDateTime" : ISODate("2022-01-03T01:01:01Z") } ]});
I want to add a new date field, based on the original date field, to end up with this:
{ _id: 1, "innerArray": [ { "originalDateTime" : ISODate("2022-01-01T01:01:01Z"), "copiedDateTime" : ISODate("2022-01-01T12:01:01Z") } ]}
{ _id: 2, "innerArray": [ { "originalDateTime" : ISODate("2022-01-02T01:01:01Z"), "copiedDateTime" : ISODate("2022-01-02T12:01:01Z") } ]}
{ _id: 3, "innerArray": [ { "originalDateTime" : ISODate("2022-01-03T01:01:01Z"), "copiedDateTime" : ISODate("2022-01-03T12:01:01Z") } ]}
In pseudo code I am saying take the originalDateTime, run it through a function and add a related copiedDateTime value.
For my specific use-case, the function I want to run strips the timezone from originalDateTime, then overwrites it with a new one, equivalent to the Java ZonedDateTime function withZoneSameLocal. Aka 9pm UTC becomes 9pm Brussels (therefore effectively 7pm UTC). The technical justification and methodology were answered in another Stack Overflow question here.
The part of the query I'm struggling with, is the part that updates/selects data from an element inside an array. In my simplistic example, for example I have crafted this query, but unfortunately it doesn't work:
This function puts copiedDateTime in the correct place... but doesn't evaluate the commands to manipulate the date:
db.innerArrayExample.updateMany({ "innerArray.0.originalDateTime" : { $exists : true }}, { $set: { "innerArray.0.copiedDateTime" : { $dateFromString: { dateString: { $dateToString: { "date" : "$innerArray.0.originalDateTime", format: "%Y-%m-%dT%H:%M:%S.%L" }}, format: "%Y-%m-%dT%H:%M:%S.%L", timezone: "Europe/Paris" }}});
// output
{
_id: 1,
innerArray: [
{
originalDateTime: ISODate("2022-01-01T01:01:01.000Z"),
copiedDateTime: {
'$dateFromString': {
dateString: { '$dateToString': [Object] },
format: '%Y-%m-%dT%H:%M:%S.%L',
timezone: 'Europe/Paris'
}
}
}
]
}
This simplified query, also has the same issue:
b.innerArrayExample.updateMany({ "innerArray.0.originalDateTime" : { $exists : true }}, { $set: { "innerArray.0.copiedDateTime" : "$innerArray.0.originalDateTime" }});
//output
{
_id: 1,
innerArray: [
{
originalDateTime: ISODate("2022-01-01T01:01:01.000Z"),
copiedDateTime: '$innerArray.0.originalDateTime'
}
]
}
As you can see this issue looks to be separate from the other stack overflow question. Instead of being able changing timezones, it's about getting things inside arrays to update.
I plan to take this query, create 70,000 variations of it with different location/timezone combinations and run it against a database with millions of records, so I would prefer something that uses updateMany instead of using Javascript to iterate over each row in the database... unless that's the only viable solution.
I have tried putting $set in square brackets. This changes the way it interprets everything, evaluating the right side, but causing other problems:
test> db.innerArrayExample.updateMany({ "_id" : 1 }, [{ $set: { "innerArray.0.copiedDateTime" : "$innerArray.0.originalDateTime" }}]);
//output
{
_id: 1,
innerArray: [
{
'0': { copiedDateTime: [] },
originalDateTime: ISODate("2022-01-01T01:01:01.000Z")
}
]
}
Above it seems to interpret .0. as a literal rather than an array element. (For my needs I know the array only has 1 item at all times). I'm at a loss finding an example that meets my needs.
I have also tried experimenting with the arrayFilters, documented on my mongo updateMany documentation but I cannot fathom how it works with objects:
test> db.innerArrayExample.updateMany(
... { },
... { $set: { "innerArray.$[element].copiedDateTime" : "$innerArray.$[element].originalDateTime" } },
... { arrayFilters: [ { "originalDateTime": { $exists: true } } ] }
... );
MongoServerError: No array filter found for identifier 'element' in path 'innerArray.$[element].copiedDateTime'
test> db.innerArrayExample.updateMany(
... { },
... { $set: { "innerArray.$[0].copiedDateTime" : "$innerArray.$[element].originalDateTime" } },
... { arrayFilters: [ { "0.originalDateTime": { $exists: true } } ] }
... );
MongoServerError: Error parsing array filter :: caused by :: The top-level field name must be an alphanumeric string beginning with a lowercase letter, found '0'
If someone can help me understand the subtleties of the Mongo syntax and help me back on to the right path I'd be very grateful.
You want to be using pipelined updates, the issue you're having with the syntax you're using is that it does not allow the usage of aggregation operators and document field values.
Here is a quick example on how to do it:
db.collection.updateMany({},
[
{
"$set": {
"innerArray": {
$map: {
input: "$innerArray",
in: {
$mergeObjects: [
"$$this",
{
copiedDateTime: "$$this.originalDateTime"
}
]
}
}
}
}
}
])
Mongo Playground
I need to check if an ObjectId exists in a non nested array and in multiple nested arrays, I've managed to get very close using the aggregation framework, but got stuck in the very last step.
My documents have this structure:
{
"_id" : ObjectId("605ce5f063b1c2eb384c2b7f"),
"name" : "Test",
"attrs" : [
ObjectId("6058e94c3994d04d28639616"),
ObjectId("6058e94c3994d04d28639627"),
ObjectId("6058e94c3994d04d28639622"),
ObjectId("6058e94c3994d04d2863962e")
],
"variations" : [
{
"varName" : "Var1",
"attrs" : [
ObjectId("6058e94c3994d04d28639616"),
ObjectId("6058e94c3994d04d28639627"),
ObjectId("6058e94c3994d04d28639622"),
ObjectId("60591791d4d41d0a6817d23f")
],
},
{
"varName" : "Var2",
"attrs" : [
ObjectId("60591791d4d41d0a6817d22a"),
ObjectId("60591791d4d41d0a6817d255"),
ObjectId("6058e94c3994d04d28639622"),
ObjectId("60591791d4d41d0a6817d23f")
],
},
],
"storeId" : "9acdq9zgke49pw85"
}
Let´s say I need to check if this if this _id exists "6058e94c3994d04d28639616" in all arrays named attrs.
My aggregation query goes like this:
db.product.aggregate([
{
$match: {
storeId,
},
},
{
$project: {
_id: 0,
attrs: 1,
'variations.attrs': 1,
},
},
{
$project: {
attrs: 1,
vars: '$variations.attrs',
},
},
{
$unwind: '$vars',
},
{
$project: {
attr: {
$concatArrays: ['$vars', '$attrs'],
},
},
},
]);
which results in this:
[
{
attr: [
6058e94c3994d04d28639616,
6058e94c3994d04d28639627,
6058e94c3994d04d28639622,
6058e94c3994d04d2863962e,
6058e94c3994d04d28639616,
6058e94c3994d04d28639627,
6058e94c3994d04d28639622,
60591791d4d41d0a6817d23f,
60591791d4d41d0a6817d22a,
60591791d4d41d0a6817d255,
6058e94c3994d04d28639622,
60591791d4d41d0a6817d23f
]
},
{
attr: [
60591791d4d41d0a6817d22a,
60591791d4d41d0a6817d255,
6058e94c3994d04d28639622,
60591791d4d41d0a6817d23f,
6058e94c3994d04d28639624,
6058e94c3994d04d28639627,
6058e94c3994d04d28639628,
6058e94c3994d04d2863963e
]
}
]
Assuming I have two products in my DB, I get this result. Each element in the outermost array is a different product.
The last bit, which is checking for this key "6058e94c3994d04d28639616", I could not find a way to do it with $group, since I dont have keys to group on.
Or with $match, adding this to the end of the aggregation:
{
$match: {
attr: "6058e94c3994d04d28639616",
},
},
But that results in an empty array. I know that $match does not query arrays like this, but could not find a way to do it with $in as well.
Is this too complicated of a Schema? I cannot have the original data embedded, since it is mutable and I would not be happy to change all products if something changed.
Will this be very expensive if I had like 10000 products?
Thanks in advance
You are trying to compare string 6058e94c3994d04d28639616 with ObjectId. Convert the string to ObjectId using $toObjectId operator when perform $match operation like this:
{
$match: {
$expr: {
$in: [{ $toObjectId: "6058e94c3994d04d28639616" }, "$attr"]
}
}
}
I am new to MongoDB and trying to execute a query. I have a company collection and company IDs array. I would like to get the results where attributes.0.ccode exist and attributes.0.ccode is not empty and will be checked within the ids provided in an array( cdata)
var query = Company.find({ _id: { $in: cdata } },{ "attributes.0.ccode": { $exists: true }, $and: [ { "attributes.0.ccode": { $ne: "" } } ] }).select({"attributes": 1}).sort({});
The error I am getting is
"$err": "Can't canonicalize query: BadValue Unsupported projection option: attributes.0.ccode: { $exists: true }",
"code": 17287
I think it's a bracketing issue but can't figure it out where.
Any help is highly appreciated.
In your code { _id: { $in: cdata } } is interpreted as query, and everything else, starting from ,{ "attributes.0.ccode": { $e.. as a Projection (which field to display). Try to refactor your code so _id: {$in ...} and the rest of the query belong to the same higher - level object. Something like this:
var query = Company.find({
_id: {
$in: cdata
},
"attributes.0.ccode": {
$exists: true
},
$and: [
{
"attributes.0.ccode": {
$ne: ""
}
}
]
}).select({"attributes": 1}).sort({});
The documents in my collection("users") look like this:
{
_id: ObjectID(...),
verifiedFields:
{
email: ["user#example.com", "othermail#example.com"],
phone: [...]
},
profiles:
[
{
_id: ObjectID(...),
fields:
{
email: { value: "user#example.com" }
}
}
]
}
I would like to now select all users who's profiles.field.email.value is contained within their verifiedFields.email. I would like to avoid having to store a boolean verified as the user should be allowed to have multiple profiles with the same fields and this would cause complications later on.
Thanks in advance!
Your query is:
db.users.find({
$expr: {$gt: [{$size: {$setIntersection: ['$profiles.fields.email.value', '$verifiedFields.email']}}, 0]}
})
This query doesn't use indexes, so on performance issue you will need add verified field.
Please, check aggregation pipeline and $expr for better understanding this query. Pipeline used for testing your query was:
db.test.aggregate([
{ $addFields: {a : '$profiles.fields.email.value'}},
{ $addFields: {b: { $setIntersection: ['$a', '$verifiedFields.email']}}},
{ $match: {$expr: {$gt: [{$size: '$b'}, 0]}}}
])
something like this should work:
collection('users').find({
'profiles[0].fields.email.value': {$in: 'verifiedFields.email'}
})
First, let me say that this question is slightly different to others I've seen regarding nested arrays in mongo.
Most of them ask how to get a specific element in a nested array. I want to get all elements of an array that has itself another array containing a given value.
I have documents, that look like this:
{
_id: something,
value: something,
items: [{
name: someItem,
price: somePrice,
stores:[
{name: storeName, url: someUrl },
{name: anotherStoreName, url: someOtherUrl}
]
},
{
name: someOtherItem,
price: someOtherPrice,
stores:[
{name: storeName, url: someUrl },
{name: yetAnotherStoreName, url: someOtherUrl}
]
}]
}
What I want is to get only the items elements that have a given store in the stores array.
This is, if I ask storeName, I should get both items in the example. But if I ask for anotherStoreName, I should only get the first element of the array items.
Searching for similar questions and trying the solutions I can only get the first matching element of items, but I want all the matching elements.
I'd appreciate any help.
You should use mongo aggregation to get result in following way.
First use $unwind to separate array of items and then add match condition.
db.collection.aggregate([{
$unwind: "$items"
}, {
$match: {
"items.stores.name": "storeName"
}
}, {
$project: {
_id: 0,
items: "$items" //you can add multiple fields here
}
}]);
You should use the aggregation framework to unwind elements on the items array and then treat them as individual documents.
db.data.aggregate([
{
$unwind: "$items"
},
{
$project: {
_id: 0,
items: "$items"
}
},
{
$match: {
"items.stores.name": "yetAnotherStoreName"
}
}
]);
$project just let you work with the important part of the document.
This works on the mongoshell be carefully when using your driver.
The result.