Update array in mongo and upsert - arrays

I'm trying to update an array within a document and it works correctly, but when I want to add a new element with upsert fails how to run an error. I've been searching on google for a few hours and the mongodb documentation and what I have tried I cannot operate.
The structure of the collection is:
{
"name" : String,
"providerId": Number,
"description": String,
"providers": [
{
"merchantId": String,
"name": String,
"valid": Boolean,
"data": String
},
{
"merchantId": String,
"name": String,
"valid": Boolean,
"data": String
},
{
"merchantId": String,
"name": String,
"valid": Boolean,
"data": String
}
]
}
Use this query to update existing data:
db.collection.update( { "providerId": ID, "providers": { $elemMatch: { "merchantId": MERCHANTID }}}, { $set: {
"providers.$.merchantId": MERCHANTID,
"providers.$.name": NAME,
"providers.$.valid": true,
"providers.$.data": DATA
}});
This working properly and correctly updated me the elements of the array. I want to when an element does not exist add it, without knowing if there are items or not, but not is if possible, probe to add upsert ( { upsert: true } ) parameter but gives me the following error. I think it is because it does not return any object search.
This is the error:
MongoError: The positional operator did not find the match needed from the query. Unexpanded update: providers.$.name
Is there any way to update the data in the subdocuments in the array and is compatible with add new ones if they don't exist? I've tried to search with the operator $in and it gives me error; also probe to search in different ways ( { "providers.merchantId": MERCHANTID } ) and others.
Thanks

There is an option to achieve what you want.
// step 1
var writeResult = db.collection.update({
"providerId" : ID,
"providers" : {
$elemMatch : {
"merchantId" : MERCHANTID
}
}
}, {
$set : {
"providers.$.merchantId" : MERCHANTID,
"providers.$.name" : NAME,
"providers.$.valid" : true,
"providers.$.data" : DATA
}
});
// step 2
if (!writeResult.nModified) { // if step 1 has succeeded on update, nModified == 1, else nModified == 0
db.collection.update({
"providerId" : ID,
"providers.merchantId" : {
$ne : MERCHANTID // this criteria is necessary to avoid concurrent issue
}
}, {
"$push" : {
"prividers" : {
"merchantId" : MERCHANTID,
"name" : NAME,
"valid" : true,
"data" : DATA
}
}
});
}

Related

is there any possible way with upsert the document of array in mongodb [duplicate]

I have the following collection
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
"myarray" : [
{
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
},
{
userId : ObjectId("613ca5e48dbe673802c2d521"),
point : 2
},
]
}
These are my questions
I want to push into myarray if userId doesn't exist, it should be appended to myarray. If userId exists, it should be updated to point.
I found this
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId" : ObjectId("570ca5e48dbe673802c2d035")
}, {
$set: { "myarray.$.point": 10 }
})
But if userId doesn't exist, nothing happens.
and
db.collection.update({
_id : ObjectId("57315ba4846dd82425ca2408")
}, {
$push: {
"myarray": {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
})
But if userId object already exists, it will push again.
What is the best way to do this in MongoDB?
Try this
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $push: {"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}}
)
Explination:
in the first statment $pull removes the element with userId= ObjectId("570ca5e48dbe673802c2d035") from the array on the document where _id = ObjectId("57315ba4846dd82425ca2408")
In the second one $push inserts
this object { userId:ObjectId("570ca5e48dbe673802c2d035"), point: 10 } in the same array.
The accepted answer by Flying Fisher is that the existing record will first be deleted, and then it will be pushed again.
A safer approach (common sense) would be to try to update the record first, and if that did not find a match, insert it, like so:
// first try to overwrite existing value
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
// you probably need to modify the following if-statement to some async callback
// checking depending on your server-side code and mongodb-driver
if(!result.nMatched)
{
// record not found, so create a new entry
// this can be done using $addToSet:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408")
},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
// OR (the equivalent) using $push:
db.collection.update(
{
_id: ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": {$ne: ObjectId("570ca5e48dbe673802c2d035"}}
},
{
$push: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
);
}
This should also give (common sense, untested) an increase in performance, if in most cases the record already exists, only the first query will be executed.
There is a option called update documents with aggregation pipeline starting from MongoDB v4.2,
check condition $cond if userId in myarray.userId or not
if yes then $map to iterate loop of myarray array and check condition if userId match then merge with new document using $mergeObjects
if no then $concatArrays to concat new object and myarray
let _id = ObjectId("57315ba4846dd82425ca2408");
let updateDoc = {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
};
db.collection.update(
{ _id: _id },
[{
$set: {
myarray: {
$cond: [
{ $in: [updateDoc.userId, "$myarray.userId"] },
{
$map: {
input: "$myarray",
in: {
$mergeObjects: [
"$$this",
{
$cond: [
{ $eq: ["$$this.userId", updateDoc.userId] },
updateDoc,
{}
]
}
]
}
}
},
{ $concatArrays: ["$myarray", [updateDoc]] }
]
}
}
}]
)
Playground
Unfortunately "upsert" operation is not possible on embedded array. Operators simply do not exist so that this is not possible in a single statement.Hence you must perform two update operations in order to do what you want. Also the order of application for these two updates is important to get desired result.
I haven't found any solutions based on a one atomic query. Instead there are 3 ways based on a sequence of two queries:
always $pull (to remove the item from array), then $push (to add the updated item to array)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{ $pull: {"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")}}
)
db.collection.update(
{ _id : ObjectId("57315ba4846dd82425ca2408")},
{
$push: {
"myarray": {
userId:ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
}
)
try to $set (to update the item in array if exists), then get the result and check if the updating operation successed or if a $push needs (to insert the item)
var result = db.collection.update(
{
_id : ObjectId("57315ba4846dd82425ca2408"),
"myarray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{
$set: {"myarray.$.point": {point: 10}}
}
);
if(!result.nMatched){
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
{
$addToSet: {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
);
always $addToSet (to add the item if not exists), then always $set to update the item in array
db.collection.update({_id: ObjectId("57315ba4846dd82425ca2408")},
myarray: { $not: { $elemMatch: {userId: ObjectId("570ca5e48dbe673802c2d035")} } } },
{
$addToSet : {
myarray: {
userId: ObjectId("570ca5e48dbe673802c2d035"),
point: 10
}
}
},
{ multi: false, upsert: false});
db.collection.update({
_id: ObjectId("57315ba4846dd82425ca2408"),
"myArray.userId": ObjectId("570ca5e48dbe673802c2d035")
},
{ $set : { myArray.$.point: 10 } },
{ multi: false, upsert: false});
1st and 2nd way are unsafe, so transaction must be established to avoid two concurrent requests could push the same item generating a duplicate.
3rd way is safer. the $addToSet adds only if the item doesn't exist, otherwise nothing happens. In case of two concurrent requests, only one of them adds the missing item to the array.
Possible solution with aggregation pipeline:
db.collection.update(
{ _id },
[
{
$set: {
myarray: { $filter: {
input: '$myarray',
as: 'myarray',
cond: { $ne: ['$$myarray.userId', ObjectId('570ca5e48dbe673802c2d035')] },
} },
},
},
{
$set: {
myarray: {
$concatArrays: [
'$myarray',
[{ userId: ObjectId('570ca5e48dbe673802c2d035'), point: 10 },
],
],
},
},
},
],
);
We use 2 stages:
filter myarray (= remove element if userId exist)
concat filtered myarray with new element;
When you want update or insert value in array try it
Object in db
key:name,
key1:name1,
arr:[
{
val:1,
val2:1
}
]
Query
var query = {
$inc:{
"arr.0.val": 2,
"arr.0.val2": 2
}
}
.updateOne( { "key": name }, query, { upsert: true }
key:name,
key1:name1,
arr:[
{
val:3,
val2:3
}
]
In MongoDB 3.6 it is now possible to upsert elements in an array.
array update and create don't mix in under one query, if you care much about atomicity then there's this solution:
normalise your schema to,
{
"_id" : ObjectId("57315ba4846dd82425ca2408"),
userId : ObjectId("570ca5e48dbe673802c2d035"),
point : 5
}
You could use a variation of the .forEach/.updateOne method I currently use in mongosh CLI to do things like that. In the .forEach, you might be able to set all of your if/then conditions that you mentioned.
Example of .forEach/.updateOne:
let medications = db.medications.aggregate([
{$match: {patient_id: {$exists: true}}}
]).toArray();
medications.forEach(med => {
try {
db.patients.updateOne({patient_id: med.patient_id},
{$push: {medications: med}}
)
} catch {
console.log("Didn't find match for patient_id. Could not add this med to a patient.")
}
})
This may not be the most "MongoDB way" to do it, but it definitely works and gives you the freedom of javascript to do things within the .forEach.

ElasticSearch - why ScriptDocValues returns unique values?

I have simple mapping - one string field and one string[] field.
The array of strings contains duplicate values, and I get those duplicate values in query:
{ "query" : { "term" : {"id" : "579a252585b8c5c428fa0a3c"} } }
Returns a single valid hit:
{
"id" : "579a252585b8c5c428fa0a3c",
"touches" : [ "5639abfb5cba47087e8b4571", "5639abfb5cba47087e8b4571", "5639abfb5cba47087e8b4571", "5639abfb5cba47087e8b457b", "5639abfb5cba47087e8b457b"
}
But in metric script aggregation:
"aggs": {
"path": {
"scripted_metric": {
"map_script": "_agg['result'] = doc['touches'].values"
}
}
}
retuns
"aggregations" : {
"path" : {
"value" : [ { }, {
"result" : [ "5639abfb5cba47087e8b4571", "5639abfb5cba47087e8b457b" ]
}, { }, { }, { } ]
}
}
that element is org.elasticsearch.index.fielddata.ScriptDocValues$Strings, casting it toString() returns a json-encoded 2-element array.
So, the question:
Why does ScriptDocValues$Strings return only unique array values and how to get the initial array in script aggregation?
Thanks.
UPD
I found that for numerical values (in particular floats) everything works perfect.

MongoDB Update array in a document

I try to update arrays of multiple document with this query :
db.BusinessRequest.update({"DealTypes": { $exists: true }, "DealTypes.DisplayName": "Minority trade sale" }, {$set:{"DealTypes.$.DisplayName":"Minority"}}, false,true );
but when there is a match, it only updates the first row of my array whereas the displayName does not match with the first.
I use IntelliShell of MongoChef software.
My document looks like this :
{
"_id" : BinData(4, "IKC6QJRGSIywmKTKKRfTHA=="),
"_t" : "InvestorBusinessRequest",
"Title" : "Business Request 000000002",
"DealTypes" : [
{
"_id" : "60284B76-1F45-49F3-87B5-5278FF49A304",
"DisplayName" : "Majority",
"Order" : "001"
},
{
"_id" : "64A52AFE-2FF5-426D-BEA7-8DAE2B0E59A6",
"DisplayName" : "Majority trade sale",
"Order" : "002"
},
{
"_id" : "C07AE70D-4F62-470D-BF65-06AF93CCEBFA",
"DisplayName" : "Minority trade sale",
"Order" : "003"
},
{
"_id" : "F5C4390A-CA7D-4AC8-873E-2DC43D7F4158",
"DisplayName" : "Equity fund raising",
"Order" : "004"
}
]
}
How can I achieve this please ? Thanks in advance
EDIT :
This line works :
db.BusinessRequest.update({"DealTypes": { $exists: true }, "DealTypes": { $elemMatch: {"DisplayName": "Majority trade sale"}}}, {$set:{"DealTypes.$.DisplayName":"Majority"}}, false,true );
Please try this :
db.BusinessRequest.find().forEach( function(doc) {
do {
db.BusinessRequest.update({{"DealTypes": { $exists: true }, "DealTypes.DisplayName": "Minority trade sale" },
{$set:{"DealTypes.$.DisplayName":"Minority"}});
} while (db.getPrevError().n != 0);
})
or
You cannot modify multiple array elements in a single update operation. Thus, you'll have to repeat the update in order to migrate documents which need multiple array elements to be modified. You can do this by iterating through each document in the collection, repeatedly applying an update with $elemMatch until the document has all of its relevant comments replaced.
db.BusinessRequest.update({"DealTypes": { $exists: true }, "DealTypes": { $elemMatch: {"DisplayName": "Majority trade sale"}}}, {$set:{"DealTypes.$.DisplayName":"Majority"}}, false,true );
If you need efficiency in the search then I suggest you to normalise schema where each row is kept in separate document.
Please execute the following script in your mongo shell :
db.BusinessRequest.find({"DealTypes":{$exists:true}}).forEach(function(item)
{
for(i=0;i < item.DealTypes.length;i++)
{
if(item.DealTypes[i].DisplayName === 'Minority trade sale'){
item.DealTypes[i].DisplayName = 'Minority';
}
}
db.BusinessRequest.save(item);
});
Last two arguments in your update have a problem.
This is the form of update() method in mongodb
db.collection.update(
<query>,
<update>,
{
upsert: <boolean>,
multi: <boolean>,
writeConcern: <document>
}
)
I believe your update should be like this;
db.BusinessRequest.update
( {"DealTypes": { $exists: true }, "DealTypes.DisplayName": "Minority trade sale" }
, {$set:{"DealTypes.$.DisplayName":"Minority"}}
{ upsert : false, multi : true });

mongo add to nested array if entry does not contain two fields that match

I have a mongo document that contains an array called history:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another": 3
}
]
}
I want to add fields to each history object or update fields IF the name AND organisation match, however if they don't, I want to add a new object to the array with the queried name and organisation and add/update the other fields to the object when necessary.
So:
This query, finds one that matches:
db.users.find({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
})
However, I'm struggling to get the update/upsert to work IF that combination of history.name and history.organisation dont exist in the array.
What I think I need to do is a :
"If this history name does not equal 'Test123' AND the history organisation does not equal 'Rat' then add an object to the array with those fields and any other field provided in the update query."
I tried this:
db.users.update({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
}, {
history: { name: "Test123"},
history: { organisation: "Rat"}
}, {upsert:true})
But that gave me E11000 duplicate key error index: db.users.$email_1 dup key: { : null }
Any help greatly appreciated.
Thanks community!
Not possible with a single atomic update I'm afraid, you would have to do a couple of update operations that satisfy both conditions.
Break down the update logic into two distinct update operations, the first one would require using the positional $ operator to identify the element in the history array you want and the $set to update the existing fields. This operation follows the logic update fields IF the name AND organisation match
Now, you'd want to use the findAndModify() method for this operation since it can return the updated document. By default, the returned document does not include the modifications made on the update.
So, armed with this arsenal, you can then probe your second logic in the next operation i.e. update IF that combination of "history.name" and "history.organisation" don't exist in the array. With this second
update operation, you'd need to then use the $push operator to add the elements.
The following example demonstrates the above concept. It initially assumes you have the query part and the document to be updated as separate objects.
Take for instance when we have documents that match the existing history array, it will just do a single update operation, but if the documents do not match, then the findAndModify() method will return null, use this logic in your second update operation to push the document to the array:
var doc = {
"name": "Test123",
"organisation": "Rat"
}, // document to update. Note: the doc here matches the existing array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
After this operation for documents that match, querying the collection will yield the same
db.users.find({ "email": "email#address" });
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
}
]
}
Now consider documents that won't match:
var doc = {
"name": "foo",
"organisation": "bar"
}, // document to update. Note: the doc here does not matches the current array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
Querying this collection for this document
db.users.find({ "email": "email#address" });
would yield
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
},
{
"name" : "foo",
"organisation" : "bar"
}
]
}

Mongodb: Querying array of subdocuments

I have users' collection whose schema is like:
{
_id: unique number,
name: 'asdf',
age: '12',
gender: 'm',
address: [
{area: 'sdf',
city: 'sdq',
state: 'wfw'},
{area: 'asdf',
city: 'sdfs',
state: 'vfdwd'}
]
}
I want to find out the users for whom all the values of state in address should be the value I pass. If even one of the state value doesn't match with the value I pass the user shouldn't be returned.
I tried simple find, aggregation framework with $unwind, $match but nothing seemed to get solution. Can you please help me out...
Thanks
P.S. please bear with multiple addresses for the sake of question. :)
To find out if all array entries match the state "wfw", do an aggregation like the following:
db.users.aggregate([
{ "$project" : {
"test" : {
"$allElementsTrue" : [{
"$map" : {
"input" : "$address",
"as" : "a",
"in" : { "$eq" : ["wfw", "$$a.state"] }
}
}]
}
} },
{ "$match" : { "test" : true } }
])
This aggregation takes each document, maps "state equals 'wfw'" over the address array to get a boolean array, and tests if the entire array is true, storing the result in `test, and then filtering the results based on test. You will need MongoDB 2.6 for support of some of the operators.
I don't know if I understand.
I replicated your document. When you want to retrieve an user by state you can do in many ways
If you search with single value you can do
db.g.find({ "address.state": "wfw" })
and retrieve an user
You can use $all
db.g.find( { "address.state": { $all: ["wfw","vfdwd"] } } ) // retrieve User
db.g.find( { "address.state": { $all: ["wfw","vfdwd","foo"] } } ) // don't retrieve User
or you can use $and
db.g.find( { $and: [ { "address.state":"wfw" },{ "address.state":"vfdwd" }] } )
But I don't know if I understand your question
Update and the correct answer
db.g.find( { "address.state": { $nin: ["wfw"] } } )
Let me Know

Resources