I am new to MongoDB. I have a collection that has multiple documents in which a particular document has a nested array structure. I need to iterate through this nested array and change the data type of the value of the iterated field.
Nested Array structure:
[
{
"identifier":{
"type":"xxxx",
"value":"1111"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":"yyyyy",
"value":"222"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":"zzzzz",
"value":"3333"
},
"origin":"https://olkdghf.com",
"score":8.0
}
]
The problem is I need to change the datatype without replacing the existing field value. But I am getting a new empty value instead of the original value.
My query:
db.SourceEntityv8test.find({"hasIdentifier.identifier.type": {$exists:true}}).sort({_id:1}).skip(0).limit(100).forEach(function(x)
{
db.SourceEntityv8test.update({_id: x._id}, {$set:{"hasIdentifier.$[].identifier.type":[]}} );
});
Expected output:
[
{
"identifier":{
"type":[xxxx],
"value":"1111"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[yyyyy],
"value":"222"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[zzzzz],
"value":"3333"
},
"origin":"example.com",
"score":8.0
}
]
Achieved output:
[
{
"identifier":{
"type":[],
"value":"1111"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[],
"value":"222"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[],
"value":"3333"
},
"origin":"example.com",
"score":8.0
}
]
A bit complex. Expect that you need to achieve it by update with aggregation pipeline.
Concept:
Update whole hasIdentifier array by 1.1.
1.1. Merge each document in hasIdentifier array with 1.1.1.
1.1.1. Merge identifier object with the document with type array.
db.SourceEntityv8test.update({_id: x._id},
[
{
$set: {
hasIdentifier: {
$map: {
input: "$hasIdentifier",
in: {
$mergeObjects: [
"$$this",
{
"identifier": {
$mergeObjects: [
"$$this.identifier",
{
type: [
"$$this.identifier.type"
]
}
]
}
}
]
}
}
}
}
}
])
Sample Mongo Playground
Related
So suppose I have a document like:
{
_id: 1,
items: ["aaa", "bbb", "ccc", "ddd", "eee"...]
}
I would like to shuffle the items list once, with this order saved in the table - i.e. I don't want to call random or something for every query, since there are about 200,000 items in this array (not huge, but still, calling $rand every time I want to retrieve an item would be inefficient)
So I'm really looking for some kind of manual script that I can run once - it would then update this document, so it became something like:
{
_id: 1,
items: ["ddd", "bbb", "aaa", "eee", "ccc"...]
}
If anyone knows if this is possible, I'd appreciate it. Thanks
Otherwise, I'd probably fetch the data, shuffle it using another language, then save it back into Mongo
I'm not sure this is the better way to do this
https://mongoplayground.net/p/4AH8buOXudQ
db.collection.aggregate([
{
$unwind: {
path: "$items"
}
},
{
$sample: {
size: 100 //to shuffle values upto particular index
}
},
{
$group: {
_id: "$_id",
item: {
$push: "$items"
}
}
}
]);
If you're Mongo version 5.2+ I would do this using an aggregation pipeline update with the new $sortArray operator and $rand.
Essentially we add a random value for each item, sort the array and then transform it back, You can run this update on demand whenever you want to reshuffle the array.
db.collection.updateMany(
{},
[
{
$addFields: {
items: {
$map: {
input: {
$sortArray: {
input: {
$map: {
input: "$items",
in: {
value: "$$this",
sortVal: {
$rand: {}
}
}
}
},
sortBy: {
"sortVal": 1
}
}
},
in: "$$this.value"
}
}
}
}
])
Mongo Playground
If you're on a lesser version, you can generate some kind of pseudo random sort using $reduce ( you can actually do a bubble sort as well but that n^2 performance on such a large array is not recommend ), here is an example of how to generate some sort of randomness:
The approach is to iterate over the items array with the $reduce operator, if the random generated value is less than 0.3 then we push the item to be in the start of the array, if that value is less than 0.6 we append it to the end of the new array and if that value is between 0.6 and 1 and push it in the middle of the array.
Obviously you can choose whatever random logic you want and add more switch cases, as mentioned even an actual sort is possible but at the cost of performance.
db.collection.update({},
[
{
$addFields: {
items: {
$map: {
input: {
$reduce: {
input: {
$map: {
input: "$items",
in: {
value: "$$this",
sortVal: {
$rand: {}
}
}
}
},
initialValue: [],
in: {
$switch: {
branches: [
{
case: {
$lt: [
"$$this.sortVal",
0.333
]
},
then: {
$concatArrays: [
"$$value",
[
"$$this"
]
]
},
},
{
case: {
$lt: [
"$$this.sortVal",
0.6666
]
},
then: {
$concatArrays: [
[
"$$this"
],
"$$value",
]
}
}
],
default: {
$concatArrays: [
{
$slice: [
"$$value",
{
$round: {
$divide: [
{
$size: "$$value"
},
2
]
}
}
]
},
[
"$$this"
],
{
$slice: [
"$$value",
{
$round: {
$divide: [
{
$size: "$$value"
},
2
]
}
},
{
$add: [
{
$size: "$$value"
},
1
]
}
]
}
]
}
}
}
}
},
in: "$$this.value"
}
}
}
}
])
Mongo Playground
I am having the below document structure:
[
{
"network_type": "ex",
"rack": [
{
"xxxx": {
"asn": 111111,
"nodes": {
"business": [
"sk550abcc1eb01.abc.com",
"sk550abcc1eb10.abc.com",
"sk550abcc1eb19.abc.com",
"sk550abcc1eb28.abc.com"
]
},
"region": "ex-01",
"zone": "01a"
}
}
]
}
]
I need to rename/update the key array element "xxxx" to "details".
I tried the below command, but it doesn't seem to work.
db.collection.update({},
{
$rename: {
"rack.xxxx": "details"
}
})
Link: https://mongoplayground.net/p/9dcDP-VKZ55
Please help me.
You can't direct $rename the field name which is within the array.
Instead,
Iterate with document(s) in the rank array, create the details field with the value of xxxx and next append this field to each document.
Remove the path with $rank.xxxx to remove the xxxx field from the document(s) in the rank array.
db.collection.update({},
[
{
$set: {
rack: {
$map: {
input: "$rack",
in: {
$mergeObjects: [
{
"details": "$$this.xxxx"
},
"$$this"
]
}
}
}
}
},
{
$unset: "rack.xxxx"
}
])
Sample Mongo Playground
I have a script creating a document, updating it and cleaning up.
db.getCollection('things').insert( { _id: 1001,
elemo: { a: "A", b: "B" },
histo: [ ] } } )
db.getCollection('things').update( { _id: 1001 },
[ { $set: {
histo: { $concatArrays: [ "$histo", ["$elemo"] ] } } } ] )
db.getCollection("things").find({ _id: 1001})
db.getCollection('things').remove({ _id: 1001 })
For certain reasons, I'd like to retain the functionality but can't guarantee that the originally empty array actually exists. I need to perform my update in such a way so that an existing array will get an additional element, while a non-existing (yet) one will get created (including said element).
db.getCollection('things').insert( { _id: 1001,
elemo: { a: "A", b: "B" } } )
db.getCollection('things').update( { _id: 1001 },
[ { $set: {
histo: { $concatArrays: [ "$histo", ["$elemo"] ] } } } ] )
db.getCollection("things").find({ _id: 1001})
db.getCollection('things').remove({ _id: 1001 })
The above only creates the field but its value is null, and so additional amendments to it result in null. I'm rather certain that it needs something more around $concatArrays but I can't figure out what. First, I thought I could go $ifnull but it didn't recognize that command (no error, no insertion, no coalescing, nothing).
You can make use of $cond or $ifNull (as you guessed) to check if the key exists or not inside the $concatArrays operator.
Using $cond Method
db.collection.update({
_id: 1001
},
[
{
$set: {
histo: {
"$concatArrays": [
{
"$cond": {
"if": {
"$not": [
"$histo"
]
},
"then": [],
"else": "$histo",
}
},
[
"$elemo"
],
],
}
}
}
])
Mongo Playground Sample Execution
Using $ifNull Method
db.collection.update({
_id: 1001
},
[
{
$set: {
histo: {
"$concatArrays": [
{
"$ifNull": [
"$histo",
[]
],
},
[
"$elemo"
],
],
}
}
}
])
Mongo Playground Sample Execution
Here is my Json in Mongo DB Compass. I am just querying greater than rating products from each collection.
Note: if I am doing with pageCount it is working fine because that is not inside a collection.
{PageCount:{gte:2}} -- works.
Problem with inner arrays collection of collection if anyone matches it displays all.
When we are doing the below query if anyone of the index have greater than 99 it shows all the values.
{"ProductField.ProductDetailFields.ProductDetailInfo.ProductScore.Rating": {$exists:true, $ne: null , $gte: 99}}
----- if I perform above query, I am getting this output.
How to iterate like foreach kind of things and check the condition in MongoDB querying
{
"_id":{
"$oid":"5fc73a7b3fb52d00166554b9"
},
"ProductField":{
"PageCount":2,
"ProductDetailFields":[
{
"PageNumber":1,
"ProductDetailInfo":[
{
"RowIndex":0,
"ProductScore":{
"Name":"Samsung",
"Rating":99
},
},
{
"RowIndex":1,
"ProductScore":{
"Name":"Nokia",
"Rating":96
},
},
{
"RowIndex":2,
"ProductScore":{
"Name":"Apple",
"Rating":80
},
}
]
}
]
}
},
{
"_id":{
"$oid":"5fc73a7b3fb52d0016655450"
},
"ProductField":{
"PageCount":2,
"ProductDetailFields":[
{
"PageNumber":1,
"ProductDetailInfo":[
{
"RowIndex":0,
"ProductScore":{
"Name":"Sony",
"Rating":93
}
},
{
"RowIndex":1,
"ProductScore":{
"Name":"OnePlus",
"Rating":93
}
},
{
"RowIndex":2,
"ProductScore":{
"Name":"BlackBerry",
"Rating":20
}
}
]
}
]
}
}
#Misky How to run this query execute:
While run this query in Mongo Shell - no sql client throws below error. we are using 3.4.9 https://www.nosqlclient.com/demo/
Is this somewhat close to your idea
db.collection.aggregate({
$addFields: {
"ProductField.ProductDetailFields": {
$map: {
"input": "$ProductField.ProductDetailFields",
as: "pdf",
in: {
$filter: {
input: {
$map: {
"input": "$$pdf.ProductDetailInfo",
as: "e",
in: {
$cond: [
{
$gte: [
"$$e.ProductScore.Rating",
99
]
},
{
$mergeObjects: [
"$$e",
{
PageNumber: "$$pdf.PageNumber"
}
]
},
null
]
}
}
},
as: "i",
cond: {
$ne: [
"$$i",
null
]
}
}
}
}
}
}
},
{
$addFields: {
"ProductField.ProductDetailFields": {
"$arrayElemAt": [
"$ProductField.ProductDetailFields",
0
]
}
}
})
LIVE VERSION
I have changed one of the fields of my collection in mongoDB from an array of strings to an array of object containing 2 strings. New documents get inserted without any problem, but when a get method is called to get , querying all the documents I get this error:
Failed to decode 'Students'. Decoding 'photoAddresses' errored
with: readStartDocument can only be called when CurrentBSONType is
DOCUMENT, not when CurrentBSONType is STRING.
photoAddresses is the field that was changed in Students.
I was wondering is there any way to update all the records so they all have the same data type, without losing any data.
The old version of photoAdresses:
"photoAddresses" : ["something","something else"]
This should be updated to the new version like this:
"photoAddresses" : [{photoAddresses:"something"},{photoAddresses:"something else"}]
The following aggregation queries update the string array to object array, only if the array has string elements. The aggregation operator $map is used to map the string array elements to objects. You can use any of the two queries.
db.test.aggregate( [
{
$match: {
$expr: { $and: [ { $isArray: "$photo" },
{ $gt: [ { $size: "$photo" }, 0 ] }
]
},
"photo.0": { $type: "string" }
}
},
{
$project: {
photo: {
$map: {
input: "$photo",
as: "ph",
in: { addr: "$$ph" }
}
}
}
},
] ).forEach( doc => db.test.updateOne( { _id: doc._id }, { $set: { photo: doc.photo } } ) )
The following query works with MongoDB version 4.2+ only. Note the update operation is an aggregation instead of an update. See updateMany.
db.test.updateMany(
{
$expr: { $and: [ { $isArray: "$photo" },
{ $gt: [ { $size: "$photo" }, 0 ] }
]
},
"photo.0": { $type: "string" }
},
[
{
$set: {
photo: {
$map: {
input: "$photo",
as: "ph",
in: { addr: "$$ph" }
}
}
}
}
]
)
[EDIT ADD]: The following query works with version MongoDB 3.4:
db.test.aggregate( [
{
$addFields: {
matches: {
$cond: {
if: { $and: [
{ $isArray: "$photoAddresses" },
{ $gt: [ { $size: "$photoAddresses" }, 0 ] },
{ $eq: [ { $type: { $arrayElemAt: [ "$photoAddresses", 0 ] } }, "string" ] }
] },
then: true,
else: false
}
}
}
},
{
$match: { matches: true }
},
{
$project: {
photoAddresses: {
$map: {
input: "$photoAddresses",
as: "ph",
in: { photoAddresses: "$$ph" }
}
}
}
},
] ).forEach( doc => db.test.updateOne( { _id: doc._id }, { $set: { photoAddresses: doc.photoAddresses } } ) )