I want to insert a string in an array of strings that is present in a mongodb collection.
the collection looks like:
{
"_id": some id,
"urls": [url1,url2]
}
I want to insert a string on every insert call to the urls array
How can I do it?
The $push operator changed some time back to allow an array of 1 to n items to be inserted into an existing array at an arbitrary position:
db.foo.find();
{ "_id" : 0, "items" : [ "url1", "url2" ] }
// Position starts at 0. Let's use position 1 for the demo:
db.foo.update({_id:0},{$push: {"items": {$each: ["foo","bar"], $position: 1}}});
db.foo.find();
{ "_id" : 0, "items" : [ "url1", "foo", "bar", "url2" ] }
Note that position can also be negative. -1 is the end of the array, -2 is 1 item in from the end, etc. To insert something into the items array in every doc, provide the {} predicate and multi:true option:
db.foo.update({},{$push: {"items": {$each: ["foo","bar"], $position: 1}}},{multi:true});
your post request, with express.js for example, will look something like this:
const urlRouter = express.Router();
urlRouter.route('/your_route_here').post((req, res, next) => {
yourMongoCollectionSchema.create(req.body)
.then(url => {
console.log("Url added", url);
res.setHeader('Content-Type', 'application/json');
res.json(url);
}, err => next(err))
.catch(err => next(err));
})
Related
I have the following collection
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
"myarray" : [
{
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
},
{
userId : ObjectId("613ca5e48dbe673802c2d521"),
point : 2
},
]
}
These are my questions
I want to push into myarray if userId doesn't exist, it should be appended to myarray. If userId exists, it should be updated to point.
I found this
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId" : ObjectId("570ca5e48dbe673802c2d035")
}, {
$set: { "myarray.$.point": 10 }
})
But if userId doesn't exist, nothing happens.
and
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408")
}, {
$push: {
"myarray": {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
})
But if userId object already exists, it will push again.
What is the best way to do this in MongoDB?
Try this
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $push: {"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}}
)
Explination:
in the first statment $pull removes the element with userId= ObjectId("570ca5e48dbe673802c2d035") from the array on the document where _id = ObjectId("57315ba4846dd82425ca2408")
In the second one $push inserts
this object { userId:ObjectId("570ca5e48dbe673802c2d035"), point: 10 } in the same array.
The accepted answer by Flying Fisher is that the existing record will first be deleted, and then it will be pushed again.
A safer approach (common sense) would be to try to update the record first, and if that did not find a match, insert it, like so:
// first try to overwrite existing value
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
// you probably need to modify the following if-statement to some async callback
// checking depending on your server-side code and mongodb-driver
if(!result.nMatched)
{
// record not found, so create a new entry
// this can be done using $addToSet:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408")
},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
// OR (the equivalent) using $push:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": {$ne: ObjectId("570ca5e48dbe673802c2d035"}}
},
{
$push: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
}
This should also give (common sense, untested) an increase in performance, if in most cases the record already exists, only the first query will be executed.
There is a option called update documents with aggregation pipeline starting from MongoDB v4.2,
check condition $cond if userId in myarray.userId or not
if yes then $map to iterate loop of myarray array and check condition if userId match then merge with new document using $mergeObjects
if no then $concatArrays to concat new object and myarray
let _id = ObjectId("57315ba4846dd82425ca2408");
let updateDoc = {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
};
db.collection.update(
{ _id: _id },
[{
$set: {
myarray: {
$cond: [
{ $in: [updateDoc.userId, "$myarray.userId"] },
{
$map: {
input: "$myarray",
in: {
$mergeObjects: [
"$$this",
{
$cond: [
{ $eq: ["$$this.userId", updateDoc.userId] },
updateDoc,
{}
]
}
]
}
}
},
{ $concatArrays: ["$myarray", [updateDoc]] }
]
}
}
}]
)
Playground
Unfortunately "upsert" operation is not possible on embedded array. Operators simply do not exist so that this is not possible in a single statement.Hence you must perform two update operations in order to do what you want. Also the order of application for these two updates is important to get desired result.
I haven't found any solutions based on a one atomic query. Instead there are 3 ways based on a sequence of two queries:
always $pull (to remove the item from array), then $push (to add the updated item to array)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{
$push: {
"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
)
try to $set (to update the item in array if exists), then get the result and check if the updating operation successed or if a $push needs (to insert the item)
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
if(!result.nMatched){
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
);
always $addToSet (to add the item if not exists), then always $set to update the item in array
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
myarray: { $not: { $elemMatch: {userId: ObjectId("570ca5e48dbe673802c2d035")} } } },
{
$addToSet : {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
},
{ multi: false, upsert: false});
db.collection.update({
_id: ObjectId("57315ba4846dd82425ca2408"),
"myArray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{ $set : { myArray.$.point: 10 } },
{ multi: false, upsert: false});
1st and 2nd way are unsafe, so transaction must be established to avoid two concurrent requests could push the same item generating a duplicate.
3rd way is safer. the $addToSet adds only if the item doesn't exist, otherwise nothing happens. In case of two concurrent requests, only one of them adds the missing item to the array.
Possible solution with aggregation pipeline:
db.collection.update(
{ _id },
[
{
$set: {
myarray: { $filter: {
input: '$myarray',
as: 'myarray',
cond: { $ne: ['$$myarray.userId', ObjectId('570ca5e48dbe673802c2d035')] },
} },
},
},
{
$set: {
myarray: {
$concatArrays: [
'$myarray',
[{ userId: ObjectId('570ca5e48dbe673802c2d035'), point: 10 },
],
],
},
},
},
],
);
We use 2 stages:
filter myarray (= remove element if userId exist)
concat filtered myarray with new element;
When you want update or insert value in array try it
Object in db
key:name,
key1:name1,
arr:[
{
val:1,
val2:1
}
]
Query
var query = {
$inc:{
"arr.0.val": 2,
"arr.0.val2": 2
}
}
.updateOne( { "key": name }, query, { upsert: true }
key:name,
key1:name1,
arr:[
{
val:3,
val2:3
}
]
In MongoDB 3.6 it is now possible to upsert elements in an array.
array update and create don't mix in under one query, if you care much about atomicity then there's this solution:
normalise your schema to,
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
}
You could use a variation of the .forEach/.updateOne method I currently use in mongosh CLI to do things like that. In the .forEach, you might be able to set all of your if/then conditions that you mentioned.
Example of .forEach/.updateOne:
let medications = db.medications.aggregate([
{$match: {patient_id: {$exists: true}}}
]).toArray();
medications.forEach(med => {
try {
db.patients.updateOne({patient_id: med.patient_id},
{$push: {medications: med}}
)
} catch {
console.log("Didn't find match for patient_id. Could not add this med to a patient.")
}
})
This may not be the most "MongoDB way" to do it, but it definitely works and gives you the freedom of javascript to do things within the .forEach.
I have a mongoose schema like this:
{
"_id" : ObjectId("5a7acda13b808dbed05d6505"),
"symbol" : "#AKS",
"counter" : 4
},
{
"_id" : ObjectId("5a7acda13b808dbed05d6506"),
"symbol" : "#AKD",
"counter" : 5
}
Now if I want to update multiple column i.e "symbol" values on one query then MongoDB updateMany Query will do. This is Query:
MyCollectionName.updateMany({ 'symbol' : { $in : req.body.array}}, { $inc : { 'counter': 1 } }, function(err, data) {
if(err) {
console.log('error')
} else {
console.log('value incremented')
}
});
By this Query If I give the Array value i.e
var array = ["#AKS", "#AKD"];
Then it will Increment the counter of Both. My Question is If I will Provide the Same value on an array then I want Two increment not one. Like this:
var array = ["#AKS", "#AKS"];
//I want the Increment of 2.
var array = [#AKS", "#AKS", "#AKS", "#AKS"]
//at this stage I want the counter of Increment is 4
But currently it will do Only One. Is this possible????
Any help is really appreciated.
You can use the bulkWrite API for this update as it allows you to compose an array of bulkWrite() write operations in which you can use the updateOne operation for each element in the array. The following example shows how you can apply it in your case:
let ops = [];
const handleResult = res => console.log(res);
const handleError = err => console.error(err);
req.body.array.forEach(function(item) {
ops.push({
"updateOne": {
"filter": { "symbol": item },
"update": { "$inc": { "counter": 1 } }
}
});
if (ops.length === 1000) {
MyCollectionName.bulkWrite(ops).then(handleResult).catch(handleError);
ops = [];
}
})
if (ops.length > 0) MyCollectionName.bulkWrite(ops).then(handleResult).catch(handleError);
what if instead of using updateMany you just forEach your array:
const yourArray = req.body.array;
yourArray.forEach(function(item) {
MyCollectionName.update({ 'symbol': { $in: item }}, { $inc: { 'counter': 1 }, function (err, data) {
// additional code
});
});
I have a mongodb Document like the one mentioned below
{
"_id" : ObjectId("213122423423423"),
"eventDateTimes" :
[
ISODate("2015-05-26T12:18:15.000Z"),
ISODate("2015-05-26T12:18:14.000Z"),
ISODate("2015-05-26T12:00:37.000Z"),
ISODate("2015-05-26T12:00:36.000Z")
],
"parseFlags" :
[
false,
"True",
false,
false
],
"eventMessages" : [
"Erro1 ",
"Error2",
"Error3",
"Error4"
]
}
}
I have to fetch the eventMessages based on the parseFlags array.
I have to get the index of elements in the parseFlags array where the value is "false" and then get the event messages mapping to those indexes.
The result should be the document with following parameters where parseFlag is false:
{
id,
EventDateTimes:[date1,date3,date4],
ParseFlags :[false,false,false]
eventMessages :[Error1,Error3,Error4]
}
Can you please let me know how to get this output? I am using mongodb 3.2.
Mongodb 3.4 has new array operator zip that groups the array elements based on the index
db.collectionName.aggregate({
$project: {
transposed: {
$zip: {inputs: ["$eventDateTimes", "$parseFlags", "$eventMessages"]
}
}
},
{$unwind: '$transposed'},
{$project: {eventDateTime: {$arrayElemAt: ['$tranposed',0]}, parseFlag:
{$arrayElemAt: ['$transposed', 1]}}}
{$match: {parseFlag: false}}
)
For mongo 3.2 there isn't any direct way to handle the expected query. You can instead use mapReduce with custom code
db.collection.mapReduce(function(){
var transposed = [];
for(var i=0;i<this.eventDateTimes.length;i++){
if(this.parseFlags[i]===false){
transposed.push({eventDateTime: this.eventDateTimes[i], parseFlag: this.parseFlags[i]});
}
}
emit(this._id, transposed);
}, function(key, values){
return [].concat.apply([], values);
}, {out: {inline:1}})
PS: I haven't actually executed the query, but above one should give you an idea how it needs to be done.
I have a mongo document that contains an array called history:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another": 3
}
]
}
I want to add fields to each history object or update fields IF the name AND organisation match, however if they don't, I want to add a new object to the array with the queried name and organisation and add/update the other fields to the object when necessary.
So:
This query, finds one that matches:
db.users.find({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
})
However, I'm struggling to get the update/upsert to work IF that combination of history.name and history.organisation dont exist in the array.
What I think I need to do is a :
"If this history name does not equal 'Test123' AND the history organisation does not equal 'Rat' then add an object to the array with those fields and any other field provided in the update query."
I tried this:
db.users.update({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
}, {
history: { name: "Test123"},
history: { organisation: "Rat"}
}, {upsert:true})
But that gave me E11000 duplicate key error index: db.users.$email_1 dup key: { : null }
Any help greatly appreciated.
Thanks community!
Not possible with a single atomic update I'm afraid, you would have to do a couple of update operations that satisfy both conditions.
Break down the update logic into two distinct update operations, the first one would require using the positional $ operator to identify the element in the history array you want and the $set to update the existing fields. This operation follows the logic update fields IF the name AND organisation match
Now, you'd want to use the findAndModify() method for this operation since it can return the updated document. By default, the returned document does not include the modifications made on the update.
So, armed with this arsenal, you can then probe your second logic in the next operation i.e. update IF that combination of "history.name" and "history.organisation" don't exist in the array. With this second
update operation, you'd need to then use the $push operator to add the elements.
The following example demonstrates the above concept. It initially assumes you have the query part and the document to be updated as separate objects.
Take for instance when we have documents that match the existing history array, it will just do a single update operation, but if the documents do not match, then the findAndModify() method will return null, use this logic in your second update operation to push the document to the array:
var doc = {
"name": "Test123",
"organisation": "Rat"
}, // document to update. Note: the doc here matches the existing array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
After this operation for documents that match, querying the collection will yield the same
db.users.find({ "email": "email#address" });
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
}
]
}
Now consider documents that won't match:
var doc = {
"name": "foo",
"organisation": "bar"
}, // document to update. Note: the doc here does not matches the current array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
Querying this collection for this document
db.users.find({ "email": "email#address" });
would yield
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
},
{
"name" : "foo",
"organisation" : "bar"
}
]
}
I am trying to query a single embedded document in an array in MongoDB. I don't know what I am doing wrong. Programmatically, I will query this document and insert new embedded documents into the currently empty trips arrays.
{
"_id" : ObjectId("564b3300953d9d51429163c3"),
"agency_key" : "DDOT",
"routes" : [
{
"route_id" : "6165",
"route_type" : "3",
"trips" : [ ]
},
{
"route_id" : "6170",
"route_type" : "3",
"trips" : [ ]
},
...
]
}
Following queries -I run in mongo shell- return empty:
db.tm_routes.find( { routes : {$elemMatch: { route_id:6165 } } } ).pretty();
db.tm_routes.find( { routes : {$elemMatch: { route_id:6165,route_type:3 } } } ).pretty();
db.tm_routes.find({'routes.route_id':6165}).pretty()
also db.tm_routes.find({'routes.route_id':6165}).count() is 0.
The following query returns every document in the array
db.tm_routes.find({'routes.route_id':'6165'}).pretty();
{
"_id" : ObjectId("564b3300953d9d51429163c3"),
"agency_key" : "DDOT",
"routes" : [
{
"route_id" : "6165",
"route_type" : "3",
"trips" : [ ]
},
{
"route_id" : "6170",
"route_type" : "3",
"trips" : [ ]
},
...
]}
but db.tm_routes.find({'routes.route_id':'6165'}).count() returns 1.
And finally, here is how I inserted data in the first place -in Node.JS-:
async.waterfall([
...
//RETRIEVE ALL ROUTEIDS FOR EVERY AGENCY
function(agencyKeys, callback) {
var routeIds = [];
var routesArr = [];
var routes = db.collection('routes');
//CALL GETROUTES FUNCTION FOR EVERY AGENCY
async.map(agencyKeys, getRoutes, function(err, results){
if (err) throw err;
else {
callback(null, results);
}
});
//GET ROUTE IDS
function getRoutes(agencyKey, callback){
var cursor = routes.find({agency_key:agencyKey});
cursor.toArray(function(err, docs){
if(err) throw err;
for(i in docs){
routeIds.push(docs[i].route_id);
var routeObj = {
route_id:docs[i].route_id,
route_type:docs[i].route_type,
trips:[]
};
routesArr.push(routeObj);
/* I TRIED 3 DIFFERENT WAYS TO PUSH DATA
//1->
collection.update({agency_key:agencyKey}, {$push:{"routes":{
'route_id':docs[i].route_id,
'route_type':docs[i].route_type,
'trips':[]
}}});
//2->
collection.update({agency_key:agencyKey}, {$push:{"routes":routeObj}});
*/
}
// 3->
collection.update({agency_key:agencyKey}, {$push:{routes:{$each:routesArr}}});
callback(null, routeIds);
});
};
},
...
var collection = newCollection(db, 'tm_routes',[]);
function newCollection(db, name, options){
var collection = db.collection(name);
if (collection){
collection.drop();
}
db.createCollection(name, options);
return db.collection(name);
}
Note: I am not using Mongoose and don't want to use if possible.
Melis,
I see what you are asking for, and what you need is help understanding how things are stored in mongodb. Things to understand:
A document is the basic unit of data for MongoDB and can be roughly compared to a row in a relational database.
A collection can be thought of as a table with a dynamic schema
So documents are stored in collections.Every document has a special _id, that is unique within a collection. What you showed us above in the following format is One document.
{
"_id" : ObjectId("564b3300953d9d51429163c3"),
"agency_key" : "DDOT",
"routes" : [
{
"route_id" : "6165",
"route_type" : "3",
"trips" : [ ]
},
{
"route_id" : "6170",
"route_type" : "3",
"trips" : [ ]
},
...
]}
If you run a query in your tm_routes collection. The find() will return each document in the collection that matches that query. Therefore when you run the query db.tm_routes.find({'routes.route_id':'6165'}).pretty(); it is returning the entire document that matches the query. Therefore this statement is wrong:
The following query returns every document in the array
If you need to find a specific route in that document, and only return that route, depending on your use, because its an array, you may have to use the $-Positional Operator or the aggregation framework.
For Node and Mongodb users using Mongoose, this is one of the ways to write the query to the above problem:
db.tm_routes.updateOne(
{
routes: {
$elemMatch: {
route_id: 6165 (or if its in a route path then **6165** could be replaced by **req.params.routeid**
}
}
},
{
$push: {
"routes.$.trips":{
//the content you want to push into the trips array goes here
}
}
}
)