I have several objects:
{
"product":"bike",
"ExpectedDeliveryDate": "2018-11-05T00:00:00"
},
{
"product":"flowers",
"ExpectedDeliveryDate": "2018-11-05T00:00:00"
},
{
"product":"balls",
"ExpectedDeliveryDate": "2018-11-10T00:00:00"
},
{
"product":"phones",
"ExpectedDeliveryDate": "2018-11-10T00:00:00"
},
{
"product":"microphones",
"ExpectedDeliveryDate": "2018-11-08T00:00:00"
},
{
"product":"monitors",
"ExpectedDeliveryDate": "2018-11-08T00:00:00"
},
I want a list of distinct dates, removing duplicated. How can I extract the three unique dates here
2018-11-05T00:00:00
2018-11-10T00:00:00
2018-11-08T00:00:00
Using Workflow expressions?
Using Workflow expressions?
It seems that it's hard to use workflow expression to achieve it. You want to get the ExpectedDeliveryDate property and deduplicate it if it is duplicate.
However, there is no attribute like distinct in workflow expression, you could refer to this article.
So, I suggest that you could use Azure function to achieve your logic and call azure function in logic app. Here is a similar article you could refer to.
Related
So I have this scenario where I want to save a condition whether if a user "has joined the website onboarding tour". So the question I want to ask is not how to save the field to MongoDB, but rather what field type should I use for it?
I want it to be reusable for another certain condition such as "has joined a campaign A", or "has visited special page B". Now I have 3 cases that you can look:
Case 1: Just a single boolean field for every case: One field for one condition, such as hasJoinedNewOnboarding: true or hasJoinedCampaignA: false. All I need to do is search by key: true.
Case 2: Use an array: One array for many conditions, such as ['hasJoinedNewOnboarding','hasJoinedCampaignA']. Let's say the field name is meta_data. All I need to do is search using $elemMatch like { meta_data: { $elemMatch: { 'hasJoinedNewOnboarding' } } }
Case 3: Use an object: One object for many conditions, such as Case 2: Use an array: One array for many conditions, such as { hasJoinedNewOnboarding: true, hasJoinedCampaignA: false }. Let's say the field name is meta_data. All I need to do is search it like { 'meta_data.hasJoinedNewOnboarding': true }
With that said, which one do you think is the best way to store the conditions in the database? Or do you have something in mind that is better than these 3 cases?
Thanks
There's not much difference when you want to query your data, it will always be either:
db.col.find({hasJoinedNewOnboarding: true})
or for the second approach:
db.col.find({arrayName: "hasJoinedNewOnboarding"})
Both ways are easy however I would recommend storing such events in an array because it's easier to aggregate the data when you don't need to refer to multiple key names in MongoDB,
For example, if you have a document like:
{
events: [
"hasJoinedNewOnboarding",
"hasJoinedCampaignA"
]
}
You can dynamically count how many users have done something by running following query:
db.collection.aggregate([
{
$unwind: "$events"
},
{
$group: {
_id: "$events",
count: { $sum: 1 }
}
}
])
Mongo Playground
Alternatively if you decide to use first or third approach the name of the event is represented by the name of the key in MongoDB's document so you can still easily count single event occurances but if you want to group all events dynamically you need to use $objectToArray operator which becomes more cumbersome.
So the recommended approach would be to keep them as an array of strings or an array of objects like:
{ events: [ { eventType: "NewOnboarding", date: ... } ] }
Follwing statement find all profiles that has Facebook or twitter and this works:
$filter=SocialAccounts/any(x: search.in(x, 'Facebook,Twitter'))
But I cant find any samples for finding all that has both Facebook and twitter. I tried:
$filter=SocialAccounts/all(x: search.in(x, 'Facebook,Twitter'))
But this is not valid query.
Azure Search does not support the type of ‘all’ filter that you’re looking for. Using search.in with ‘all’ would be equivalent to using OR, but Azure Search can only handle AND in the body of an ‘all’ lambda (which is equivalent to OR in the body of an ‘any’ lambda).
You might try a workaround like this:
$filter=tags/any(t: t eq 'Facebook') and tags/any(t: t eq 'Twitter')
However, this isn't actually equivalent to using all with search.in. The query as expressed using all is matching documents where every social account is strictly either Facebook or Twitter. If any other social account is present, the document won’t match. The workaround doesn’t have this property. A document must have at least Facebook and Twitter in order to match, but not exclusively those. This is certainly a valid scenario; it just isn't the same as using all with search.in, which was the original question.
No matter how you try to rewrite the query, you won’t be able to express an equivalent to the all query. This is a limitation due to the way Azure Search stores collections of strings and other primitive types in the inverted index.
Please vote on user voice to help prioritize:
https://feedback.azure.com/forums/263029-azure-search/suggestions/37166749-efficient-way-to-express-a-true-all
A possible workaround is to use the new Complex Types feature, which does allow more expressive filters inside lambda expressions. For example, if you model tags as objects with a single value property instead of as a collection of strings, you should be able to execute a filter like this:
$filter=tags/all(t: search.in(t/value, 'Facebook,Twitter'))
In the REST API, you'd define tags like this:
{
"name": "myindex",
"fields": [
...
{
"name": "tags",
"type": "Collection(Edm.ComplexType)",
"fields": [
{ "name": "value", "type": "Edm.String", "filterable": true }
]
}
]
}
Note that this feature is in preview at the time of this writing, but will be generally available (and publicly documented) soon.
I am using Cloudant database and I would like to retrieve all the documents within the db that match specific fields.
I mean - I'd want to get only those documents which have some fields with specific values that I put.
Could you please help me with an example of code I can test?
Thanks in advance
A good, but quite general question.
You can achieve this in several ways. The most canonical CouchDB way would be to create a map-reduce view (secondary index) keyed on the field you wish to be able to query on, for example:
function (doc) {
if (doc && doc.surname) {
emit(doc.surname, 1);
}
}
You can create such views using the Cloudant dashboard:
You can now query this for all documents with a particular surname:
curl 'https://ACCOUNT.cloudant.com/examples/_design/example/_view/by_surname?limit=100&reduce=false&include_docs=true&startkey="kruger"&endkey="kruger0"'
If you want to be able to query on a combination of fields, you can create a vector-valued key:
function (doc) {
if (doc && doc.surname && doc.firstname) {
emit([doc.surname, doc.firstname], 1);
}
}
which is queried as this:
curl 'https://ACCOUNT.cloudant.com/examples/_design/example/_view/by_name?limit=100&reduce=false&include_docs=true&startkey=\["kruger", "stefan"\]&endkey=\["kruger","stefan0"\]'
If you're new to Cloudant, another way to query is by using the aptly named Cloudant Query (a.k.a. Mango), which is a json-based declarative query language.
It is well documented (https://console.bluemix.net/docs/services/Cloudant/api/cloudant_query.html), but the gist of it is queries of the type:
{
"selector": {
"year": {
"$gt": 2010
}
},
"fields": ["_id", "_rev", "year", "title"],
"sort": [{"year": "asc"}],
"limit": 10,
"skip": 0
}
So I'm using this Rest API with ngResource to do get, query, post and update requests. What I'm looking for, is a way to define the structure for each entity.
For example, assuming we have:
module.factory('app.entity.item', function($resource) {
return $resource('http://xmpl.io/items/:itemId', { itemId: '#id' });
});
I want to instantiate it in a controller like:
module.controller('AddItemCtrl', ['app.entity.item', function(Item) {
$scope.item = new Item();
});
and bind it to the respective form in my template.
The actual problem that I have run into, is that I have to deal with 1:m tables.
An example of the entity structure would be:
{
"name": "",
"categories": [],
"list": [
{
"value": "",
"list": [
{
"value": "",
"list": [
{
"value": ""
}
]
}
]
}
]
}
(A more thorough example in the fiddle below)
Now the first two fields are obviously not the problem. It is the third one. The list. Each one of these lists can have a variable number of items.
I am currently using ngRepeat and an add(type, context) method, which adds a new set of fields to the scope (value field in this example and child lists for the first two levels), which will appear in UI by ngRepeat so the user can fill it up and submit it to the service.
First off, I have to define the structure, so the UI would not be empty when the page loads.
module.controller('AddItemCtrl', ['app.entity.item', function(Item) {
$scope.item = new Item({
"name": "",
"categories": [],
"list": [
{
"value": "",
"list": [
{
"value": "",
"list": [
{
"value": ""
}
]
}
]
}
]
});
});
But that is redundant. I have to do it everywhere!
Another issue is that when the item.$save is called, the model is emptied (perhaps re-instantiated?) and the fields inside the list property (managed by the ngRepeat directive) are gone.
So I'm wondering, what would you do under such circumstances.
Is there a way to define the entity (resource) structure?
SAMPLE: http://jsfiddle.net/g15sqd5s/3/
trying to give simple answer - for simple structures I would use something like
module.factory('Item', function($resource) {
var resource = $resource('http://xmpl.io/items/:itemId', { itemId: '#id' },
// you can also define transformRequest here:
{ transformRequest: function(data) {
// data can be transformed here
return angular.toJson(data);
}});
return angular.extend(resource.prototype,
{
name: null,
categories: []
});
});
but then be aware of need to 'flatten' the object.
and for the more complex model I would check restangular
similar topic is also discussed here:
How can I extend the constructor of an AngularJS resource ($resource)?
I would go ahead and revise my model structure in the backend in the first place - the models on the client side should merely follow the ones already defined, rather than being re-defined in a transform block. So, to answer your question, the "default" model structure comes from the server. What you get in your $resource objects has the structure of what your server returns.
To start off, is it really ok to invoke $save on the Item model when the user has populated some values? What we want to save are obviously the lists associated with an item, not the item itself. A separate resource defined in the backend, say items/<item_id>/list, may be a cleaner solution. It may not scale very well, as you'll have to make a separate GET request for each item to fetch its list, but that's the proper RESTful way to do it.
Extending this approach to the example in your fiddle, I imagine a routing scheme like buildings/<building_id>/floors/<floor_id>/units/<unit_id> would be a proper solution. Making a GET request to buildings/ should yield you a list of buildings; each building in the array returned should be an instance of a Building model, which has the proper URL set so the user can perform a single POST and update only the building name, instead of sending back the whole structure back to the server. Applying this recursively to the nested resources should give you a clean and concise way to deal with model changes.
Regarding the UI part - I would go ahead and define three directives for buildings, floors and units, and let each one manage an array with the respective resources, also taking care for the UI bindings to the model values.
So how could a Building model look like?
var BuildingResource = $resource('/buildings/:id', { id: '#id' });
Invoking BuildingResource.query() should yield an array of existing buildings. Adding a new building could look like this:
var newBuilding = new BuildingResource();
newBuilding.$save().then(function(building) {
$scope.buildings.push(building);
}, function(errData) {
//Handle error here...
});
It should be easy to extend this pattern for the rest of the resources - note that what the server needs to return for every building is just the name and the id; knowing the id is sufficient to construct an URL (and a $resource object, respectively) to fetch the needed child resources (in this case, floors).
what seemed a simple task, came to be a challenge for me.
I have the following mongodb structure:
{
(...)
"services": {
"TCP80": {
"data": [{
"status": 1,
"delay": 3.87,
"ts": 1308056460
},{
"status": 1,
"delay": 2.83,
"ts": 1308058080
},{
"status": 1,
"delay": 5.77,
"ts": 1308060720
}]
}
}}
Now, the following query returns whole document:
{ 'services.TCP80.data.ts':{$gt:1308067020} }
I wonder - is it possible for me to receive only those "data" array entries matching $gt criteria (kind of shrinked doc)?
I was considering MapReduce, but could not locate even a single example on how to pass external arguments (timestamp) to Map() function. (This feature was added in 1.1.4 https://jira.mongodb.org/browse/SERVER-401)
Also, there's always an alternative to write storedJs function, but since we speak of large quantities of data, db-locks can't be tolerated here.
Most likely I'll have to redesign the structure to something 1-level deep, like:
{
status:1,delay:3.87,ts:138056460,service:TCP80
},{
status:1,delay:2.83,ts:1308058080,service:TCP80
},{
status:1,delay:5.77,ts:1308060720,service:TCP80
}
but DB will grow dramatically, since "service" is only one of many options which will append each document.
please advice!
thanks in advance
In version 2.1 with the aggregation framework you are now able to do this:
1: db.test.aggregate(
2: {$match : {}},
3: {$unwind: "$services.TCP80.data"},
4: {$match: {"services.TCP80.data.ts": {$gte: 1308060720}}}
5: );
You can use a custom criteria in line 2 to filter the parent documents. If you don't want to filter them, just leave line 2 out.
This is not currently supported. By default you will always receive the whole document/array unless you use field restrictions or the $slice operator. Currently these tools do not allow filtering the array elements based on the search criteria.
You should watch this request for a way to do this: https://jira.mongodb.org/browse/SERVER-828
I'm attempting to do something similar. I tried your suggestion of using the GROUP function, but I couldn't keep the embedded documents separate or was doing something incorrectly.
I needed to pull/get a subset of embedded documents by ID. Here's how I did it using Map/Reduce:
db.parent.mapReduce(
function(parent_id, child_ids){
if(this._id == parent_id)
emit(this._id, {children: this.children, ids: child_ids})
},
function(key, values){
var toReturn = [];
values[0].children.forEach(function(child){
if(values[0].ids.indexOf(product._id.toString()) != -1)
toReturn.push(child);
});
return {children: toReturn};
},
{
mapparams: [
"4d93b112c68c993eae000001", //example parent id
["4d97963ec68c99528d000007", "4debbfd5c68c991bba000014"] //example embedded children ids
]
}
).find()
I've abstracted my collection name to 'parent' and it's embedded documents to 'children'. I pass in two parameters: The parent document ID and an array of the embedded document IDs that I want to retrieve from the parent. Those parameters are passed in as the third parameter to the mapReduce function.
In the map function I find the parent document in the collection (which I'm pretty sure uses the _id index) and emit its id and children to the reduce function.
In the reduce function, I take the passed in document and loop through each of the children, collecting the ones with the desired ID. Looping through all the children is not ideal, but I don't know of another way to find by ID on an embedded document.
I also assume in the reduce function that there is only one document emitted since I'm searching by ID. If you expect more than one parent_id to match, than you will have to loop through the values array in the reduce function.
I hope this helps someone out there, as I googled everywhere with no results. Hopefully we'll see a built in feature soon from MongoDB, but until then I have to use this.
Fadi, as for "keeping embedded documents separate" - group should handle this with no issues
function getServiceData(collection, criteria) {
var res=db[collection].group({
cond: criteria,
initial: {vals:[],globalVar:0},
reduce: function(doc, out) {
if (out.globalVar%2==0)
out.vals.push({doc.whatever.kind.and.depth);
out.globalVar++;
},
finalize: function(out) {
if (vals.length==0)
out.vals='sorry, no data';
return out.vals;
}
});
return res[0];
};