I seem to have an issue in retrieving a selected few documents from a PouchDB using startkey and endkey. I need to get the documents back with key starting with "profile" (in this example profile41 & profile48).
When using the chrome PouchDB extension, this works fine when doing a query using startkey:"profile" and endkey:"profile\0ffff", but for some reason, this doesn't work when running my code (angular/ionic).
My code returns an empty doc list. When I set the startkey:"profile41" and endkey:"profile41" I do get the doc back, so I know it connects and can retrieve documents from the DB.
PS: First use of PouchDB, so I might have overlooked something simple here.
Some documents in my db
{
"_id": "animaltypes",
"_rev": "7-e413c314272a62a6a14ed293f5f934cf",
"value": {
"rev": "7-e413c314272a62a6a14ed293f5f934cf"
},
"key": "animaltypes"
}
{
"_id": "profile41",
"_rev": "3-f4065b825d304d79479e3576409ce744",
"value": {
"rev": "3-f4065b825d304d79479e3576409ce744"
},
"key": "profile41"
}
{
"_id": "profile48",
"_rev": "3-5e62a6e33f022a8ac30d46b80126dedd",
"value": {
"rev": "3-5e62a6e33f022a8ac30d46b80126dedd"
},
"key": "profile48"
}
My javascript that retrieves docs
this.getData = function(keystart,keyend){
var deferred = $q.defer();
localDB.allDocs({include_docs: true,
descending: true,
startkey:keystart,
endkey:keyend}, function(err, doc) {
if (err) {
deferred.reject(err);
} else {
var rows = [];
for (var x in doc.rows) {
rows.push(doc.rows[x].doc.data);
}
deferred.resolve(rows);
}
});
return deferred.promise;
};
This returns an empty array with the following params
startkey = "profile"
endkey = "profile\0ffff"
This returns the correct single doc when parameters are set to
startkey = "profile41"
endkey = "profile41"
If you want to use "descending:true" you have to change the order of startkey and endkey.
Indeed, if you don't use "descending:true" you B-Tree looks like that:
1- animaltypes
2 - profile41
3 - profile48
startkey="profile" will be between id 1 and id 2
endkey="profile\0ffff" will be after id 3
In result you will have 2 records
If you use "descending=true" you B-Tree looks like that:
1- profile48
2 - profile41
3 - animaltypes
startkey="profile" will be between id 2 and id 3
endkey="profile\0ffff" will be after id 2 and id 3
in result you will have 0 record.
Related
I have a basic mongoose model with an attribute instruments which represents an array. Therefore it consists multiple items, which each have the following attributes: name, counter. The document itself has an autogenerated _id of type ObjectID.
Model
var ParticipationSchema = new Schema({
instruments: [{
name: String,
counter: Number
}],
// etc.
},
{
timestamps: true
});
I'd like now to change exactly 1 item within the instruments array, only if it matches the following requirements:
The document id has to equal 58f3c77d789330486ccadf40
The instruments-item's name which should be changed has to equal 'instrument-1'
The instrument-item's counter has to be lower than 3
Query
let someId = '58f3c77d789330486ccadf40';
let instrumentName = 'instrument-1'
let counter = 3;
Participation.update(
{
$and: [
{ _id: mongoose.Types.ObjectId(someId) },
{ 'instruments.name': instrumentName },
{ 'instruments.counter': { $lt: counter } }
]},
{
$set: {
'instruments.$.currentcounter' : counter
}
},
(err, raw) => {
// ERROR HANDLING
}
});
Let's assume I have 3 entries within the instruments-attribute:
"instruments": [
{
"name": "instrument-1",
"counter": 2
},
{
"name": "instrument-1",
"counter": 2
},
{
"name": "instrument-1",
"counter": 2
}
]
Desired behaviour: change the first element's counter attribute to 3, no matter, when running the update code 1 time, do no action when running it more times.
Actual behaviour:
it changes the 1st element's counter attribute to 3 when running it the 1st time
it changes the 2nds element's counter attribute to 3 when running it the 2nd time
it changes the 3rds element's counter attribute to 3 when running it the 3rd time
Although the queries are anded, they don't seem to run element-wise. To resolve the issue, $elemMatch can be used:
Participation.update({
_id: mongoose.Types.ObjectId(someId),
'instruments': {
$elemMatch: {
'name': instrumentName,
'counter': { $lt: counter } }}
},
// etc.
more info:
API reference on $elemMatch
Thanks to #VEERAM's for pointing out that this kind of behaviour is also documented the mongodb homepage.
I'm trying to find all users except for a few, like this:
// get special user IDs
var special = db.special.find({}, { _id: 1 }).toArray();
// get all users except for the special ones
var users = db.users.find({_id: {$nin: special}});
This doesn't work because the array that I'm passing to $nin is not and array of ObjectId but an array of { _id: ObjectId() }
Variable special looks like this after the first query:
[ { _id: ObjectId(###) }, { _id: ObjectId(###) } ]
But $nin in the second query needs this:
[ ObjectId(###), ObjectId(###) ]
How can I get just the ObjectId() in an array from the first query so that I can use them in the second query?
Or, is there a better way of achieving what I'm trying to do?
Use the cursor.map() method returned by the find() function to transform the list of { _id: ObjectId(###) } documents to an array of ObjectId's as in the following
var special = db.special.find({}, { _id: 1 }).map(function(doc){
return doc._id;
});
Another approach you can consider is using the $lookup operator in the aggregation framework to do a "left outer join" on the special collection and filtering the documents on the new "joined" array field. The filter should match on documents whose array field is empty.
The following example demonstrates this:
db.users.aggregate([
{
"$lookup": {
"from": "special",
"localField": "_id",
"foreignField": "_id",
"as": "specialUsers" // <-- this will produce an arry of "joined" docs
}
},
{ "$match": { "specialUsers.0": { "$exists": false } } } // <-- match on empty array
])
I have a mongo document that contains an array called history:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another": 3
}
]
}
I want to add fields to each history object or update fields IF the name AND organisation match, however if they don't, I want to add a new object to the array with the queried name and organisation and add/update the other fields to the object when necessary.
So:
This query, finds one that matches:
db.users.find({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
})
However, I'm struggling to get the update/upsert to work IF that combination of history.name and history.organisation dont exist in the array.
What I think I need to do is a :
"If this history name does not equal 'Test123' AND the history organisation does not equal 'Rat' then add an object to the array with those fields and any other field provided in the update query."
I tried this:
db.users.update({
email:"email#address",
$and: [
{ "history.name": "Test123", "history.organisation": "Rat"}
]
}, {
history: { name: "Test123"},
history: { organisation: "Rat"}
}, {upsert:true})
But that gave me E11000 duplicate key error index: db.users.$email_1 dup key: { : null }
Any help greatly appreciated.
Thanks community!
Not possible with a single atomic update I'm afraid, you would have to do a couple of update operations that satisfy both conditions.
Break down the update logic into two distinct update operations, the first one would require using the positional $ operator to identify the element in the history array you want and the $set to update the existing fields. This operation follows the logic update fields IF the name AND organisation match
Now, you'd want to use the findAndModify() method for this operation since it can return the updated document. By default, the returned document does not include the modifications made on the update.
So, armed with this arsenal, you can then probe your second logic in the next operation i.e. update IF that combination of "history.name" and "history.organisation" don't exist in the array. With this second
update operation, you'd need to then use the $push operator to add the elements.
The following example demonstrates the above concept. It initially assumes you have the query part and the document to be updated as separate objects.
Take for instance when we have documents that match the existing history array, it will just do a single update operation, but if the documents do not match, then the findAndModify() method will return null, use this logic in your second update operation to push the document to the array:
var doc = {
"name": "Test123",
"organisation": "Rat"
}, // document to update. Note: the doc here matches the existing array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
After this operation for documents that match, querying the collection will yield the same
db.users.find({ "email": "email#address" });
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
}
]
}
Now consider documents that won't match:
var doc = {
"name": "foo",
"organisation": "bar"
}, // document to update. Note: the doc here does not matches the current array
query = { "email": "email#address" }; // query document
query["history.name"] = doc.name; // create the update query
query["history.organisation"] = doc.organisation;
var update = db.users.findAndModify({
"query": query,
"update": {
"$set": {
"history.$.name": doc.name,
"history.$.organisation": doc.organisation
}
}
}); // return the document modified, if there's no matched document update = null
if (!update) {
db.users.update(
{ "email": query.email },
{ "$push": { "history": doc } }
);
}
Querying this collection for this document
db.users.find({ "email": "email#address" });
would yield
Output:
{
"_id" : ObjectId("575fe85bfe98c1fba0a6e535"),
"email" : "email#address",
"__v" : 0,
"history" : [
{
"name" : "Test123",
"organisation" : "Rat",
"field" : 4,
"another" : 3
},
{
"name" : "foo",
"organisation" : "bar"
}
]
}
I am trying to query a single embedded document in an array in MongoDB. I don't know what I am doing wrong. Programmatically, I will query this document and insert new embedded documents into the currently empty trips arrays.
{
"_id" : ObjectId("564b3300953d9d51429163c3"),
"agency_key" : "DDOT",
"routes" : [
{
"route_id" : "6165",
"route_type" : "3",
"trips" : [ ]
},
{
"route_id" : "6170",
"route_type" : "3",
"trips" : [ ]
},
...
]
}
Following queries -I run in mongo shell- return empty:
db.tm_routes.find( { routes : {$elemMatch: { route_id:6165 } } } ).pretty();
db.tm_routes.find( { routes : {$elemMatch: { route_id:6165,route_type:3 } } } ).pretty();
db.tm_routes.find({'routes.route_id':6165}).pretty()
also db.tm_routes.find({'routes.route_id':6165}).count() is 0.
The following query returns every document in the array
db.tm_routes.find({'routes.route_id':'6165'}).pretty();
{
"_id" : ObjectId("564b3300953d9d51429163c3"),
"agency_key" : "DDOT",
"routes" : [
{
"route_id" : "6165",
"route_type" : "3",
"trips" : [ ]
},
{
"route_id" : "6170",
"route_type" : "3",
"trips" : [ ]
},
...
]}
but db.tm_routes.find({'routes.route_id':'6165'}).count() returns 1.
And finally, here is how I inserted data in the first place -in Node.JS-:
async.waterfall([
...
//RETRIEVE ALL ROUTEIDS FOR EVERY AGENCY
function(agencyKeys, callback) {
var routeIds = [];
var routesArr = [];
var routes = db.collection('routes');
//CALL GETROUTES FUNCTION FOR EVERY AGENCY
async.map(agencyKeys, getRoutes, function(err, results){
if (err) throw err;
else {
callback(null, results);
}
});
//GET ROUTE IDS
function getRoutes(agencyKey, callback){
var cursor = routes.find({agency_key:agencyKey});
cursor.toArray(function(err, docs){
if(err) throw err;
for(i in docs){
routeIds.push(docs[i].route_id);
var routeObj = {
route_id:docs[i].route_id,
route_type:docs[i].route_type,
trips:[]
};
routesArr.push(routeObj);
/* I TRIED 3 DIFFERENT WAYS TO PUSH DATA
//1->
collection.update({agency_key:agencyKey}, {$push:{"routes":{
'route_id':docs[i].route_id,
'route_type':docs[i].route_type,
'trips':[]
}}});
//2->
collection.update({agency_key:agencyKey}, {$push:{"routes":routeObj}});
*/
}
// 3->
collection.update({agency_key:agencyKey}, {$push:{routes:{$each:routesArr}}});
callback(null, routeIds);
});
};
},
...
var collection = newCollection(db, 'tm_routes',[]);
function newCollection(db, name, options){
var collection = db.collection(name);
if (collection){
collection.drop();
}
db.createCollection(name, options);
return db.collection(name);
}
Note: I am not using Mongoose and don't want to use if possible.
Melis,
I see what you are asking for, and what you need is help understanding how things are stored in mongodb. Things to understand:
A document is the basic unit of data for MongoDB and can be roughly compared to a row in a relational database.
A collection can be thought of as a table with a dynamic schema
So documents are stored in collections.Every document has a special _id, that is unique within a collection. What you showed us above in the following format is One document.
{
"_id" : ObjectId("564b3300953d9d51429163c3"),
"agency_key" : "DDOT",
"routes" : [
{
"route_id" : "6165",
"route_type" : "3",
"trips" : [ ]
},
{
"route_id" : "6170",
"route_type" : "3",
"trips" : [ ]
},
...
]}
If you run a query in your tm_routes collection. The find() will return each document in the collection that matches that query. Therefore when you run the query db.tm_routes.find({'routes.route_id':'6165'}).pretty(); it is returning the entire document that matches the query. Therefore this statement is wrong:
The following query returns every document in the array
If you need to find a specific route in that document, and only return that route, depending on your use, because its an array, you may have to use the $-Positional Operator or the aggregation framework.
For Node and Mongodb users using Mongoose, this is one of the ways to write the query to the above problem:
db.tm_routes.updateOne(
{
routes: {
$elemMatch: {
route_id: 6165 (or if its in a route path then **6165** could be replaced by **req.params.routeid**
}
}
},
{
$push: {
"routes.$.trips":{
//the content you want to push into the trips array goes here
}
}
}
)
What is the root property value if I get a Json like that:
{
"status": {
"status": 0,
"msg": "Ok",
"protocolversion": "extjs.json"
},
"value": {
"table": [
[
"admin",
"Administrator",
""
],
[
"test",
"Test",
""
]
],
"total": 2
}
}
The data will be displayed in a gridpanel, 1 row is admin, 1 row is test, etc.
Tried:
value, value.table
How to get this to work?
value.table is correct for the root property, but you are using a json format that I don't think Ext is set up to handle by default. It has a reader for json that is used for an array of objects, not for an nested arrays of field values with no object mapping information.
If you have to use that format, you will either need to create your own readers/writers or just use Ext.Ajax.request(), and in the callback, parse the nested array into objects. Something like:
Ext.Ajax.request({
url: 'path.com/to/content',
success: function (response, operation) {
var data = Ext.JSON.decode(response.responseText);
var fields = data.value.table;
var records = [];
Ext.Array.each(fields, function (fieldArray, fieldIndex) {
Ext.Array.each(fieldArray, function(fieldValue, valueIndex) {
//Create record object if it doesn't exist
var record = records[valueIndex] || {};
//Create a switch statement based on the array index to set fields
switch(fieldIndex) {
case 0:
record.User_id = fieldValue;
break;
case 1:
record.Username = fieldValue;
break;
}
});
});
//Add the objects to the empty store
store.add(records);
}
});
That's not a production solution by any means since it doesn't handle that empty string at the end of your list or the case that you get a jagged array of arrays per field which I can't imagine what to do with. If it's within your control or influence, I would suggest using a format more like what Ext suggests so you can use the built in json proxy/reader/writer or go crazy with it and implement ext's remote procedure call format:
{
"success": true,
"message": "Success",
"data": [
{
"User_id": "admin",
"Username": "Administrator"
}, {
"User_id": "admin",
"Username": "Administrator"
}
]
}
In above example "value" is root property. But for JSON reader it's a property name (or a dot-separated list of property names if the root is nested).
so you can assign into your field you need following.
fields:['table.admin','table.test']