Using aggregation framework i would like to filter some elements inside an array based on some condition involving childs of the given array
Here there is an example of the documents
/* 1 */
{
"_id" : 1,
"articles" : [
{
"suppliers" : [
{
"exports" : [
{
"channelId" : 'A'
}
]
}
]
},
{
"suppliers" : [
{
"exports" : []
}
]
}
]
}
/* 2 */
{
"_id" : 2,
"articles" : [
{
"suppliers" : [
{
"exports" : [
{
"channelId" : 'A'
}
]
}
]
}
]
}
/* 3 */
{
"_id" : 3,
"articles" : [
{
"suppliers" : [
{
"exports" : [
{
"channelId" : 'B'
}
]
}
]
}
]
}
Let's say i want filter out all elements in articles array if 'articles.suppliers.exports.channelId' = 'A'.
Here is an example result
/* 1 */
{
"_id" : 1,
"articles" : [
{
"suppliers" : [
{
"exports" : []
}
]
}
]
}
/* 2 */
{
"_id" : 2,
"articles" : []
}
/* 3 */
{
"_id" : 3,
"articles" : [
{
"suppliers" : [
{
"exports" : [
{
"channelId" : 'B'
}
]
}
]
}
]
}
EDIT
Difference from that question is that here we want to remove all the child of the first array if the condition is verified inside the last array, in the old question we wanted to remove just the element in the last array
Related
So by mistake, I migrated 68k documents from a table to another, but the problem is ( i'm a SQL Server DBA learing mongodb):
I need a field ( array ) that has the value:
something : [ 000 ]
but during the project, I used another [ and ]:
$project:
{
field_name:[$field_value]
}
but the field is already array so the result I got was:
Vieweing with JASON:
"targetMarketCountryCode" : [
[
"076"
]
],
is there a way to update;replace this value to remove this quotes [[]] and use only one normal array quote []??
I would like to update ALL documents.
I'm trying to replace but with no success:
db.rep.replaceOne( {"field":[[079]]}), {"Idate":[079]} )
this is just a test, not the oficial table.
Tests:
db.teste.insert({name:"butter",value:[["077"]]})
db.teste.insert({name:"butter",value:[["077"]]})
db.teste.insert({name:"butter",value:[["077"]]})
db.teste.find()
/* 1 createdAt:14/08/2019 13:08:15*/
{
"_id" : ObjectId("5d5431ef44f85649486e4100"),
"name" : "butter",
"value" : [
[
"077"
]
]
},
/* 2 createdAt:14/08/2019 13:08:21*/
{
"_id" : ObjectId("5d5431f544f85649486e4101"),
"name" : "butter",
"value" : [
[
"077"
]
]
},
/* 3 createdAt:14/08/2019 13:08:22*/
{
"_id" : ObjectId("5d5431f644f85649486e4102"),
"name" : "butter",
"value" : [
[
"077"
]
]
}
///////////////////////////////////////////////////////
db.teste.aggregate({$unwind: "$value"})
/* 1 createdAt:14/08/2019 13:08:15*/
{
"_id" : ObjectId("5d5431ef44f85649486e4100"),
"name" : "butter",
"value" : [
"077"
]
},
/* 2 createdAt:14/08/2019 13:08:21*/
{
"_id" : ObjectId("5d5431f544f85649486e4101"),
"name" : "butter",
"value" : [
"077"
]
},
/* 3 createdAt:14/08/2019 13:08:22*/
{
"_id" : ObjectId("5d5431f644f85649486e4102"),
"name" : "butter",
"value" : [
"077"
]
}
////////////////////////////////////////////////////
db.teste.aggregate([{$unwind:"$value"}, {$out:"teste" }])
db.teste.find()
/* 1 createdAt:14/08/2019 13:08:15*/
{
"_id" : ObjectId("5d5431ef44f85649486e4100"),
"name" : "butter",
"value" : "077"
},
/* 2 createdAt:14/08/2019 13:08:21*/
{
"_id" : ObjectId("5d5431f544f85649486e4101"),
"name" : "butter",
"value" : "077"
},
/* 3 createdAt:14/08/2019 13:08:22*/
{
"_id" : ObjectId("5d5431f644f85649486e4102"),
"name" : "butter",
"value" : "077"
}
What I can see is:
1) I have an array with [["value"]]
2) Using $unwind I can replace [["value"]] to ["value"].
3) Uing it to "update" the document, it removes both arrays and replace the value with a simple string.
For the MongoDB version: 4.2+
db.rep.updateMany(
{},
[
{
$set: {
"targetMarketCountryCode": {
$arrayElemAt: ["$targetMarketCountryCode",0]
}
}
}
]
)
It would replace the 'targetMarketCountryCode' array with its first element in all the documents.
For lower versions: 2.6+
db.rep.aggregate([
{
$unwind:"$targetMarketCountryCode"
},
{
$out:"rep"
}
])
It performs aggregation on the 'rep' collection. The 'targetMarketCountryCode' is unwinded and old data of the collection is replaced with the output of the aggregation query.
Sample data:
{
"_id" : ObjectId("5d5430aa7c780d119a01a6ce"),
"targetMarketCountryCode" : [
[
"076"
]
]
}
Output:
{
"_id" : ObjectId("5d5430aa7c780d119a01a6ce"),
"targetMarketCountryCode" : [
"076"
]
}
Thank you guys for the answers. As i'm using mongo 4.0 I could make this:
db.collection.update({ field: [["value"]] }, {
$set: {
"field": [ "value" ]},
{ multi: true, upsert: false}})
My Document Structure:
{
"_id" : ObjectId("59edc58af33e9b5988b875fa"),
"Agent" : {
"Name" : "NomanAgent",
"Location" : "Lahore",
"AgentId" : 66,
"Reward" : "Thumb Up",
"Suggestion" : [
"Knowledge",
"Professionalisn"
]
}
}
What I want to achieve in this query:
I want to find the count of each suggestion given by a customer to every agent, it should look something like,
{
"AgentName": "Xyz",
"SuggestionCounts": {
"Knowledge": 2,
"Professionalism": 3,
"Friendliness": 1
}
}
What I have done so far,
db.getCollection('_survey.response').aggregate([
{
$group:{
_id: "$Agent.Name",
Suggestions: {$push:"$Agent.Suggestion"}
}
}
]);
Output:
/* 1 */
{
"_id" : "GhazanferAgent",
"Suggestions" : [
[
"Clarity",
"Effort"
],
[
"Friendliness"
]
]
}
/* 2 */
{
"_id" : "NomanAgent",
"Suggestions" : [
[
"Knowledge",
"Professionalisn"
]
]
}
How I want it to be(As Suggestion in the document is an array and when when i group documents by Agent.Name so it become array of arrays as shown in my output, it want to merge all arrays into single with duplication and then i will find the count of each element in array):
/* 1 */
{
"_id" : "GhazanferAgent",
"SuggestionsCombined" : [
[
"Clarity",
"Effort",
"Friendliness"
]
]
}
/* 2 */
{
"_id" : "NomanAgent",
"SuggestionsCombined" : [
[
"Knowledge",
"Professionalisn"
]
]
}
Thanks in advance!!
One way would be like this - the output structure is not identical to what you suggested but probably close enough:
db.getCollection('_survey.response').aggregate([
{
$unwind: "$Agent.Suggestion" // flatten "Suggestion" array
}, {
$group:{ // group by agent and suggestion
_id: { "AgentName": "$Agent.Name", "Suggestion": "$Agent.Suggestion" },
"Count": { $sum: 1} // calculate count of occurrencs
}
}, {
$group:{
_id: "$_id.AgentName", // group by agent only
"Suggestions": { $push: { "Suggestion": "$_id.Suggestion", "Count": "$Count" } } // create array of "Suggestion"/"Count" pairs per agent
}
}
]);
My schema is as follows
{
product:[{
structure :[{
version:[{
type:[{
values:[{}]
}]
}]
}]
}]
}
Need to search fields in values array.
Actually I need to update the fields in values array. I'm unable to write query to get sub array values. How can I do that. Kindly let me know, as soon as possible.
You can use this types of query:
// match a one name
db.getCollection('test').find({"product.structure.version.type.values.name":'test'})
// match some value from an array
db.getCollection('test').find({"product.structure.version.type.values.name": {$in: [1, 'test']}})
Last query will find 3 items in the next collection:
/* 1 */
{
"_id" : ObjectId("595b9e17c482ca1b99db23a6"),
"product" : [
{
"structure" : [
{
"version" : [
{
"type" : [
{
"values" : [
{
"name" : 1
}
]
}
]
}
]
}
]
}
]
}
/* 2 */
{
"_id" : ObjectId("595b9e68f9f2fe8d79ca4c82"),
"product" : [
{
"structure" : [
{
"version" : [
{
"type" : [
{
"values" : [
{
"name" : 3
},
{
"name" : 4
},
{
"name" : 1
}
]
}
]
}
]
}
]
}
]
}
/* 3 */
{
"_id" : ObjectId("595b9e71f9f2fe8d79ca4c84"),
"product" : [
{
"structure" : [
{
"version" : [
{
"type" : [
{
"values" : [
{
"name" : 5
}
]
}
]
}
]
}
]
}
]
}
/* 4 */
{
"_id" : ObjectId("595ba08af9f2fe8d79ca4cd9"),
"product" : [
{
"structure" : [
{
"version" : [
{
"type" : [
{
"values" : [
{
"name" : 6
}
]
}
]
}
]
}
]
}
]
}
/* 5 */
{
"_id" : ObjectId("595ba0f1f9f2fe8d79ca4d00"),
"product" : [
{
"structure" : [
{
"version" : [
{
"type" : [
{
"values" : [
{
"name" : "test"
}
]
}
]
}
]
}
]
}
]
}
UPDATE:
To do an update query, you need to do something like this:
db.getCollection('test').update({"product.structure.version.type.values.name": {$in: [1, 'test']}}, {$set: {"product.$.testProp": 'some test string'}}, {multi: true})
My collection coll is
/* 1 */
{
"_id" : ObjectId("566121aa4b88d840eb7d1c50"),
"batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"),
"array" : [
{
"id" : 1
},
{
"id" : 2
},
{
"id" : 3
},
{
"id" : 4
}
]
}
/* 2 */
{
"_id" : ObjectId("5661224a4b88d840eb7d1c51"),
"batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"),
"array" : [
{
"id" : 1
},
{
"id" : 7
},
{
"id" : 3
},
{
"id" : 5
}
]
}
what i need is to pull objects in array 'array' where
batchCourseId = ObjectId("566122ab94b792fbdf81bcf3")
and 2<array.id<=5
expected output is
/* 1 */
{
"_id" : ObjectId("566121aa4b88d840eb7d1c50"),
"array" : [
{
"id" : 3
},
{
"id" : 4
}
]
}
/* 2 */
{
"_id" : ObjectId("5661224a4b88d840eb7d1c51"),
"array" : [
{
"id" : 3
},
{
"id" : 5
}
]
}
already tried
db.coll.find({"batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3")},
{ array: { $elemMatch: { id: { $gt: 2,$lte: 5} } } })
the output is like
/* 1 */
{
"_id" : ObjectId("566121aa4b88d840eb7d1c50"),
"array" : [
{
"id" : 3
}
]
}
/* 2 */
{
"_id" : ObjectId("5661224a4b88d840eb7d1c51"),
"array" : [
{
"id" : 3
}
]
}
close but only the first matching object in array is in result
FYI this only a sample set of data the original data is more complex and big in count
so pls let me know the best practice to do this, performance is also important
thanks in advance
You can use aggregation for achieving the same. A sample is shown below:
db.coll.aggregate(
{$match: {"batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3")}},
{$unwind: '$array'},
{$match: {'array.id': { $gt: 2,$lte: 5}}},
{$group: {_id: '$_id', array: {$push : '$array'}}}
)
Result:
{ "_id" : ObjectId("5661224a4b88d840eb7d1c51"), "array" : [ { "id" : 3 }, { "id" : 5 } ] }
{ "_id" : ObjectId("566121aa4b88d840eb7d1c50"), "array" : [ { "id" : 3 }, { "id" : 4 } ] }
In MongoDB aggregation $unwind creates Cartesian_product problem so in large data set is good way to avoid $unwind.
Let's check with your example if you use $unwind in aggregation then result looks like this
db.collectionName.aggregate([
{ "$match": { "batchCourseId": ObjectId("566122ab94b792fbdf81bcf3") }},
{ "$unwind": "$array" }
])
so result of above query is :
{ "_id" : ObjectId("566121aa4b88d840eb7d1c50"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 1 } }
{ "_id" : ObjectId("566121aa4b88d840eb7d1c50"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 2 } }
{ "_id" : ObjectId("566121aa4b88d840eb7d1c50"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 3 } }
{ "_id" : ObjectId("566121aa4b88d840eb7d1c50"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 4 } }
{ "_id" : ObjectId("5661224a4b88d840eb7d1c51"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 1 } }
{ "_id" : ObjectId("5661224a4b88d840eb7d1c51"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 7 } }
{ "_id" : ObjectId("5661224a4b88d840eb7d1c51"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 3 } }
{ "_id" : ObjectId("5661224a4b88d840eb7d1c51"), "batchCourseId" : ObjectId("566122ab94b792fbdf81bcf3"), "array" : { "id" : 5 } }
this create multiple documents and in large documents in collections it slow the performance and increase processing time.
Instead of $unwind use $map in aggregation with aggregation-set operator and the query is as below :
db.collection.aggregate([{
"$match": {
"batchCourseId": ObjectId("566122ab94b792fbdf81bcf3")
}
}, {
"$project": {
"array": {
"$setDifference": [{
"$map": {
"input": "$array",
"as": "el",
"in": {
"$cond": {
"if": {
"$and": [{
"$gt": ["$$el.id", 2]
}, {
"$lte": ["$$el.id", 5]
}]
},
"then": "$$el",
"else": false
}
}
}
},
[false]
]
}
}
}])
docs:
order1.filter = ['tag1','tag2']
order2.filter = ['tag1','tag2','tag3']
want to get:
query ['tag1','tag2'] -> (only order1)
query ['tag1','tag2','tag3'] -> (order1 and order2)
query ['tag1','tag2','tag3','tag4', etc ] -> (order1 and order2)
and
query ['tag1','tag3'] -> (null)
query ['tag2','tag3'] -> (null)
All values order.filter should be necessarily in the query array
How to do it? Tried directives $all, $in :(
You can do this with aggregation framework (there is no way to do this with regular find that I know of).
I think this is basically a duplicate so adjusting that code for your fields you get something like:
//sample documents:
> db.docs.find({},{_id:0})
{ "order" : 1, "filter" : [ "t1", "t2" ] }
{ "order" : 2, "filter" : [ "t1", "t2", "t3" ] }
var tagArray = [ "t1", "t2" ]; // array to "match"
db.docs.aggregate( [
{
"$project" : {
"order" : 1,
"filter" : 1,
"killFlag" : {
"$const" : [
true,
false
]
}
}
},
{
"$unwind" : "$filter"
},
{
"$unwind" : "$killFlag"
},
{
"$match" : {
"$nor" : [
{
"filter" : {
"$in" : tagArray
},
"killFlag" : true
}
]
}
},
{
"$group" : {
"_id" : "$order",
"filter" : {
"$addToSet" : "$filter"
},
"killFlag" : {
"$max" : "$killFlag"
}
}
},
{
"$match" : {
"killFlag" : false
}
},
{
"$project" : {
"_id" : 1,
"filter" : 1
}
}
]);