count number of rows in cloudant in response - cloudant

I have below response from my map reduce .
Now i want to count the number of rows in the response can any one help me how i can do it in cloudant? I need something in response like to get the total count of distinct correlationid in a period.
{
rows: [
{
key: [
"201705",
"aws-60826346-"
],
value: null
},
{
key: [
"201705",
"aws-60826348802-"
],
value: null
},
{
key: [
"201705",
"aws-las97628elb"
],
value: null
},
{
key: [
"201705",
"aws-ve-test"
],
value: null
},
{
key: [
"201705",
"aws-6032dcbce"
],
value: null
},
{
key: [
"201705",
"aws-60826348831d"
],
value: null
},
{
key: [
"201705",
"aws-608263488833926e"
],
value: null
},
{
key: [
"201705",
"aws-608263488a74f"
],
value: null
}
]
}

You need to implement a slightly obscure concept called "chained map-reduce" to accomplish this. You can't do this in the Cloudant administrative GUI, so you'll have to write your design document by hand.
Have your map/reduce emit an array as the key. The 1st array element will be month and the second will be your correlationid. The value should be 1. Then specify the built-in _count as the reduce function.
Now you need to add the chaining part. Chaining basically involves automatically copying the result of a map/reduce into a new database. You can then do another map/reduce on that database. Thereby creating a chain of map/reduces...
Here's a tiny sample database using your example:
https://rajsingh.cloudant.com/so44106569/_all_docs?include_docs=true&limit=200
Here's the design document containing the map/reduce, along with the dbcopy command that updates a new database (in this case called sob44106569) with the results of the view called view:
{
"_id": "_design/ddoc",
"_rev": "11-88ff7d977dfff81a05c50b13d854a78f",
"options": {
"epi": {
"dbcopy":
{
"view": "sob44106569"
}
}
},
"language": "javascript",
"views": {
"view": {
"reduce": "_count",
"map": "function (doc) {\n emit([doc.month, doc.machine], 1);\n}"
}
}
}
Here's the result of the map function (no reduce) showing 10 rows. Notice that there are two documents with month 201705 and machine aws-6032dcbce:
https://rajsingh.cloudant.com/so44106569/_design/ddoc/_view/view?limit=200&reduce=false
If you just do the built-in _count reduce on this view at group_level=1, you'll get a value of 9 for 201705, which is wrong for your purposes because you want to count that aws-6032dcbce only once, even though it shows up in the data twice:
https://rajsingh.cloudant.com/so44106569/_design/ddoc/_view/view?limit=200&reduce=true&group=true&group_level=1
So let's take a quick look at the map/reduce at group_level=2. This is what gets copied to the new database:
https://rajsingh.cloudant.com/so44106569/_design/ddoc/_view/view?limit=200&reduce=true&group=true&group_level=2
Here you see that aws-6032dcbce only shows up once (but with value=2), so this is a useful view. The dbcopy part of our map/reduce creates the database sob44106569 based on this view. Let's look at that:
https://rajsingh.cloudant.com/sob44106569/_all_docs?include_docs=true
Now we can run a very simple map/reduce on that database, emitting the month and machine again (now they are in an array so have different names), but this time the repeated values for machine have already been "reduced" away.
function (doc) {
if (doc.key && doc.key.length == 2 )
emit(doc.key[0], doc.key[1]);
}
And finally here's the count of distinct "machines". Now we can finally see the desired value of 8 for 201705.
https://rajsingh.cloudant.com/sob44106569/_design/views/_view/counted?limit=200&reduce=true&group=true&group_level=1
response:
{
"rows": [
{
"key": "201705",
"value": 8
},
{
"key": "201706",
"value": 1
}
]
}

Emit 1 instead of null and use the built-in reducer _count.

Related

Merging nested object fields into a single nested array field in MongoDB Aggregation

As part of my aggregation pipeline I have the following scenario. This is the result of grouping previously unwound fields from each document (so in this case there are two documents with the same _id but with a different value for UniqueFieldName)
TopLevelField: [
{
UniqueFieldName: "Values go here!"
},
{
UniqueFieldName: "More values go here too!"
}
]
All I want to do is merge the nested object fields into one field and push all the values into that field as an array, like so.
TopLevelField: {
UniqueFieldName: [
"Values go here!",
"More values go here too!",
],
}
The idea is that I could have multiple fields with multiple values under each field grouped together for easier iteration.
TopLevelField: {
UniqueFieldName: [
"Values go here!",
"More values go here too!",
],
SecondFieldName: [
"This is text",
],
AnotherOne: [
"TEXT",
"Here too!",
"More values",
],
}
The problem I run into is that trying to use dot notation in the $group stage throws an error. It seems that mongo doesn't like to group with nested objects like this?
The easy solution is to just change the TopLevelField to some concatenation of the nested fields like this,
TopLevelField-UniqueFieldName: [
"Values go here!",
"More values go here too!",
],
TopLevelField-SecondFieldName: [
"This is text",
],
TopLevelField-AnotherOne: [
"TEXT",
"Here too!",
"More values",
],
But this is suboptimal for my use case. Is there a solution to this or do I need to rethink the entire pipeline?
You can try this :
db.collection.aggregate([
{ $unwind: '$TopLevelField' },
{
$group: {
_id: '', 'UniqueFieldName': { $push: '$TopLevelField.UniqueFieldName' },
'UniqueFieldName2': { $push: '$TopLevelField.UniqueFieldName2' },
'UniqueFieldName3': { $push: '$TopLevelField.UniqueFieldName3' },
'UniqueFieldName4': { $push: '$TopLevelField.UniqueFieldName4' }
}
}, { $project: { _id: 0 } }, { $project: { 'TopLevelField': '$$ROOT' } }])
Test : MongoDB-Playground

Mongodb - Take only one element in nested array

I'm using mongodb to store my data. My collection consists in a list of objects identified by a type a list of other objects for each of them.
An example of my collection is:
[
{
"type": "a",
"properties": [
{
"value": "value_a",
"date": "my_date_a"
},
{
"value": "value_b",
"date": "my_date_b"
},
...
]
},
...
]
Based on the above data structure, I want to retrieve all collections by a given type, taking for each of them only one element in the nested array (reducing the nested list to a list of only one element).
So, given a type "a", an example of the result may be:
[
{
"type": "a",
"properties": [
{
"value": "value_a",
"date": "my_date_a"
}
]
},
...
]
I'm started trying this query { "type": "a" } to filter the collections. But, how can I do to take only one "properties" element? I cannot use the "slice" operator.
Thanks a lot.
I'm assuming from your reference to slice, that you're not interested in matching a particular nested element, and rather just getting a value at a fixed index (eg, 0).
If you're willing to use the aggregation pipeline, you can use arrayElementAt within a projection:
db.collection.aggregate([
// matches documents with type 'a'
{ $match: { type: 'a' } },
// creates a new document for each
{ $project: {
// that contains the original value for type
type: 1,
// and the first element from the original properties for properties
properties: { $arrayElemAt: [ "$properties", 0 ] }
} }
])

Restheart query for an nested array subdocument

I m working with mongodb and restheart.
In my nosql db i have a unique document with this structure:
{
"_id": "docID",
"users": [
{
"userID": "12",
"elements": [
{
"elementID": "1492446877599",
"events": [
{
"event1": "one"
},
{
"event2": "two",
}
]
}
},
{
"userID": "11",
"elements": [
{
"elementID": "14924",
"events": [
{
"event1": "one"
},
{
"event2": "two",
}
]
}
}
]
}
how can i build an url-query in order to get the user with id 11?
Using mongo shell it should be something like this one:
db.getCollection('collection').find({},{'users':{'$elemMatch':{'userID':'12'}}}).pretty()
I cannot find anything similar on restheart.
Could someone help me?
Using this
http://myHost:port/documents/docID?filter={%27users%27:{%27$elemMatch%27:{%27userID%27:%2712%27}}}
restheart returns me all the documents: userID 11 and 12.
Your request is against a document resource, i.e. the URL is http://myHost:port/documents/docID
The filter query parameter applies for collection requests, i.e. URLs such as http://myHost:port/documents
In any case you need to projection (the keys query parameter) to limit the returned properties.
You should achieve it with the following request (I haven't tried it) using the $elementMatch projection operator:
http://myHost:port/documents?keys={"users":{"$elemMatch":{"userID":"12"}}}

Generate query results based on tags in PouchDB

I'm new to NoSQL but have decided to use PouchDB for an Angular Application I am creating.
There are going to be a series of questions (about 1000 in total) which each have their own tags. Each object shouldn't have more that 6 or 7 tags. Example data is:
{
"text": "Question?",
"answers": [
{ "text": "Yes", "correct": true },
{ "text": "No", "correct": false }
],
"tags": ["tag1", "tag3"]
},
{
"text": "Question?",
"answers": [
{ "text": "Yes","correct": true },
{ "text": "No", "correct": false }
],
"tags": ["tag2", "tag3"]
}
I'm at a total loss on how I can query the db in order to retrieve only questions that have "tag2" or questions that have "tag1" and "tag3".
I came across the question found at How to query PouchDB with SQL-like operators but can't seem to wrap my head around how it works. I tried to modify it based on my data and I always get 0 results when querying the database.
I guess my biggest struggle is comparing it to SQL when it isn't. Does anyone know how I can go about creating a query based on specific tags?
Yup, you create a map/reduce query like this:
// document that tells PouchDB/CouchDB
// to build up an index on tags
var ddoc = {
_id: '_design/my_index',
views: {
my_index: {
map: function (doc) {
doc.tags.forEach(function (tag) {
emit(tag);
});
}.toString()
}
}
};
// save it
pouch.put(ddoc).then(function () {
// success!
}).catch(console.log.bind(console));
Then you query it:
pouch.query('my_index', {key: myTag, include_docs: true}).then(function (res) {
// got a result
}).catch(console.log.bind(console));
If you want to find multiple tags, you can just keys instead of key.
BTW this will be easier in the future when I add $elemMatch and $in to pouchdb-find.

Multiple search filtering is not working in cloudant, why?

Here i quoted my code for multiple search filtering. I could not find the mistakes in that. please give a right code to make it work well.
Employee document:
{
"_id": "527c8d9327c6f27f17df0d2e17000530",
"_rev": "24-276a8dc913559901897fd601d2f9654f",
"proj_role": "TeamMember",
"work_total_experience": "3",
"personal": {
"languages_known": [
"English","Telugu"
]},
"skills": [
{
"skill_set": "Webservices Framework",
"skill_exp": 1,
"skill_certified": "yes",
"skill_rating": 3,
},
{
"skill_set": "Microsoft",
"skill_exp": 1,
"skill_certified": "yes",
"skill_rating": 3,
}
]
"framework_competency": "Nasscom",
"type": "employee-docs"
}
Design Document:
{
"_id": "_design/sample",
"_rev": "86-1250f792e6e84f6f33447a00cf64d61d",
"views": {},
"language": "javascript",
"indexes": {
"search": {
"index": "function(doc){\n index(\"default\", doc._id);if(doc.type=='employee-docs'){\nif (doc.proj_role){index(\"project_role\", doc.proj_role);}if(doc.work_total_experience){\nindex(\"work_experience\", doc.work_total_experience);}\nif(doc.personal.languages_known){for(c in doc.personal.languages_known){ \n index(\"languages_known\",doc.personal.languages_known[c]);}} if(doc.skills){for (var i=0;i<doc.skills.length;i++){\nindex('skill_set',doc.skills[i].skill_set);}}}}"
}
}
}
Run using below URL : https://ideyeah4.cloudant.com/opteamize_new/_design/sample/_search/search?q=project_role:TeamMember%20AND%20work_experience:%223%22%20AND%20languages_known:Telugu%20AND%20skill_set:Microsoft&include_docs=true
A simple way to debug this is to query the top 100 results in your index:
https://ideyeah4.cloudant.com/opteamize_new/_design/sample/_search/search?q=*:*&limit=100
This will at least tell you whether there are any documents in your index at all.
Your current query (without URL encoding) looks like:
project_role:TeamMember AND work_experience:"3" AND languages_known:Telugu AND skill_set:Microsoft
I'd suggest that some of these search values require quotes - always true when you are searching string values. Next, you could try:
project_role:"TeamMember"
see if you get any results and refine from there.
Debugging this might also be easier if you store the values as well as index them (so you can see exactly what is indexed). To do this, add an object to each index call { "store": true }. For example,
index("languages_known", doc.personal.languages_known[c], { "store": true });
Now, when you query the index it will return a list of fields which were stored with each match.

Resources