HTTP Rest API Pagination Power Automate Logic Apps - azure-logic-apps

I guess this question applies to both Power Automate and Logic Apps. I'm querying an API that returns a number of pages.
The JSON output looks like this:
{
"_results": [
{ ... },
{ ... },
{ ... },
{ ... }
],
"pageNumber": 1,
"totalPageCount": 5,
"pageSize": 100,
"pageCount": 100,
"totalCount": 445,
}
I've omitted the results data to save space.
In this example, the API result shows the data for Page 1 and we also see that there are 5 pages in total.
I tried using the pagination setting in Logic Apps but I think that only applies to continuation tokens.
I would like to know how we can loop through the pages based on the logic of pageNumber <= totalPageCount ?
Thanks

I think I've managed to resolve this now. Here are the steps that I've created in the logic app:
Parse JSON response from API call
Initialize an integer variable (TotalPages) and assign it the totalPageCount item from the response
Initialize an array variable (Pages) and assign it the expression: range(1, variables('TotalPages')).
Use a For Each function to loop through each item in the Pages array variable.
Within the For Each there's another API call that uses the Current Item, from the For Each, for the pageNumber parameter.

Related

Cannot return documents based off a sorted index using Fauna DB

I'm bumbling my way through adding a back-end to my site and have decided to get acquainted with graphQL. I may be structuring things totally the wrong way, however from following some tutorials I have a React front-end (hosted on Vercel), so I have created an api folder in my app to make use of Vercel's serverless functions. I'm using Apollo server and I decided to go with Fauna as my database.
I've successfully been able to return an entire collection via my API. Now I wish to be able to return the collection sorted by my id field.
To do this I created an index which looks like this:
{
name: "sort_by_id",
unique: false,
serialized: true,
source: "my_first_collection",
values: [
{
field: ["data", "id"]
},
{
field: ["ref"]
}
]
}
I then was able to call this via my api and get back and array, which simply contained the ID + ref, rather than the associated documents. I also could only console log it, I assume because the resolver was expecting to be passed an array of objects with the same fields as my typedefs. I understand I need to use the ref in order to look up the documents, and here is where I'm stuck. An index record looks as follows:
[1, Ref(Collection("my_first_collection"), "352434683448919125")]
In my resolvers.js script, I am attempting to receive the documents of my sorted index list. I've tried this:
async users() {
const response = await client.query(
q.Map(
q.Paginate(
q.Match(
q.Index('sort_by_id')
)
),
q.Lambda((ref) => q.Get(ref))
)
)
const res = response.data.map(item => item.data);
return [... res]
}
I'm unsure if the problem is with how I've structured my index, or if it is with my code, I'd appreciate any advice.
It looks like you also asked this question on the Fauna discourse forums and got an answer there: https://forums.fauna.com/t/unable-to-return-a-list-of-documents-via-an-index/3511/2
Your index returns a tuple (just an array in Javascript) of the data.id field and the ref. You confirmed that with your example result
[
/* data.id */ 1,
/* ref */ Ref(Collection("my_first_collection"), "352434683448919125")
]
When you map over those results, you need to Get the Ref. Your query uses q.Lambda((ref) => q.Get(ref)) which passes the whole tuple to Get
Instead, use:
q.Lambda(["id", "ref"], q.Get(q.Var("ref")))
// or with JS arrow function
q.Lambda((id, ref) => q.Get(ref))
or this will work, too
q.Lambda("index_entry", q.Get(q.Select(1, q.Var("index_entry"))))
// or with JS arrow function
q.Lambda((index_entry) => q.Get(q.Select(1, index_entry)))
The point is, only pass the Ref to the Get function.

How to traverse the following JSON in angularJS

So i am recieving a JSON string from a rest service which has the following structure-
{
"map": {
"Project [projectId=1, projectName=Project1]": [
{
"employeeId": 4,
"employeeName": "fourthEmployee"
}, {
"employeeId": 3,
"employeeName": "thirdEmployee"
}
],
"Project [projectId=2, projectName=Project2]": [
{
"employeeId": 4,
"employeeName": "fourthEmployee"
}
]
}
}
( for reference, this is the object type being made available by rest - Map<Project,List<Employee>>)
My question is, how do i display the data from this particular JSON onto the HTML page in the form of a table, using angularJS?
I do realise ng-repeat is to be used multiple times. Its just that using ng-repeat, i'm not able to get the values from JSON. Could someone reply with code, that would be really helpful thanks.
EDIT: As pointed out by Robert, the problem is i am unable to get values from the compound project data. Help is appreciated. Thanks.
Based on insufficient data let me provide a solution to the problem that I speculate OP is actually having.
Compound project data problem
I suspect that the main problem you're having is not about using nested ngRepeat but parsing the compound data of your projects.
This means that before you try displaying your data, you have to parse it or re-map it to something that you can actually use in your view.
This example uses regular expression to parse individual project property values.
var rx = /^Project \[projectId=(\d+), projectName=(.+)\]$/i;
var result = [];
for(var p in data.map)
{
if (data.map.hasOwnProperty(p) && rx.test(p))
{
var m = rx.exec(p);
result.push({
id: m[1],
name: m[2],
employees: data.map[p]
});
}
}
This is a working Fiddle with your provided data that displays projects including their data.

Extracting Arrays as values in a JSON file using AngularJS

This is sort of a three part question. I have a JSON file. A few of the values in the JSON file are arrays. Keeping that in mind:
1) On any given page, I'd only want one set of values coming out of the JSON file. For example (as you'll see in code below) my JSON file is a list of attorneys. On any given bio page, I'd obviously only want one attorney's information. I'm currently, successfully, doing this by pulling back the entire JSON and then using ng-show. But this is causing some other issues that I'll explain in later points, so I'm wondering if there's something to put in the app.factory itself to only bring back the one set in the first place.
2) As mentioned, some of the values are arrays. This comes into play two ways in this situation. One of the ways is that there is an array of quotes about the attorney that I'll need to drop into a JS array so that my JS function can loop through them. Currently, I'm hardcoding the quotes for the one test attorney but I'm really trying to figure out how to make this dynamic. This is one reason I'm trying to figure out how to bring back only one attorney's information so I can then, somehow, say his quotes go into this array.
3) Another array value is a list of his specialty areas. I have another, hardcoded, JS object, associating the short terms with the display names. I realized though, that this has two issues.
a) The JS renders after the Angular, so I can't reference that JS in the Angular code
b) I have no way , anyway, to display the JS dynamically inside the Angular code.
My solution to that aspect was to create a second JSON file holding the area hash but besides being a little cumbersome, I'm also not sure how to dynamically display just the ones I want. e.g: If my attorney only specializes in securities and litigation, how would I tell the code to only display {{areas.securities}} and {{areas.litigation}}? So,I'm open to thoughts there as well.
Here is the current, relevant code. If you need more, just ask.
Thanks.
attorneys.json (irrelevant lines removed)
{"attorneys":
[
{
"id":1,
"name":"Bob Smith",
"quotes":
[
{
"id": 1,
"quote": "Wonderful guy!",
"person": "Dovie"
},
{
"id": 2,
"quote": "If ye be wanting a haggis like no other, Bob be yer man!",
"person": "Angus McLoed"
},
{
"id": 3,
"quote": "Wotta Hottie!",
"person": "Bob's wife"
}
],
"areas": ["altdispute", "litigation", "securities"],
}
]
}
...and the relevant current JS object that I'm not sure what to do with:
var practiceareas = {
altdispute: "Alternative Dispute Resolution",
businesscorp: "Businesses & Corporations",
estateplanning: "Estate Planning",
futures: "Futures & Derivatives",
litigation: "Litigation",
productliability: "Product Liability",
realestate: "Real Estate",
securities: "Securities"
}
script.js (relevant function)
var idno = 0;
/* This is what I want replaced by the Angular pull */
var quotelist = ["\"Wonderful guy!\"<br/>-Dovie", "\"If ye be wanting a haggis like no other, Bob be yer man!\"<br/>-Angus McLoed", "\"Hubba, Hubba! What a hottie!\"<br/>-Bob's wife"];
$("#bio_quotes").html(quotelist[0]);
function quoteflip(id, total){
var src1 = quotelist[idno];
$("#bio_quotes").fadeOut(500, function(){
$("#bio_quotes").html(src1).fadeIn(500);
});
idno = (id + 1) % total;
window.setTimeout(function(){quoteflip(idno, quotelist.length);}, 5000);
}
window.setTimeout(function(){quoteflip(idno, quotelist.length);}, 500);
By the way, as far as the quotes, I'm even happy to turn the JSON into a more condensed version by removing the id and consolidating the quote and author - making it an array of strings instead of mini-objects - if that makes it easier. In fact, it might be easier as far as the function anyway.
Can definitely filter things out at the service / factory using Array.filter. If you want to filter it server side, you have to have the code at server side that will do that.
Not sure what your backend store is but definitely doable.
Again, you can do this pretty easily with Array.map which let you pull specific values into a new Array. If you just want the name and quotes' quote and person name, you can definitely do this using Array .filter and .map and bind the new array to your viewmodel / scope.
Hmm.. again, I'd disagree, this look like the same issue with JavaScript array manipulation. You can definitely as part of the transformation in point 1 and 2, include this so it will transfer area to the long practice area names. The easiest way to show the relevant practice area is to map it to the long name during the transformation in the service layer.
//get matching attorney from the store by id
var matches = data.attorneys.filter(function(a) {
return a.id === id;
});
//If match found,
if (matches.length === 1) {
var result = matches[0];
//map the long name for practicing area
//use the matching attorney's area name as key
//and overwrite the result areas with the map
result.areas = result.areas.map(function(a) {
return practiceareas[a];
});
return result;
}
See this solution: http://embed.plnkr.co/xBPju7/preview
As for the fade in and fade out, I'll let you figure it out...

Angular Resource - Default Model Structure

So I'm using this Rest API with ngResource to do get, query, post and update requests. What I'm looking for, is a way to define the structure for each entity.
For example, assuming we have:
module.factory('app.entity.item', function($resource) {
return $resource('http://xmpl.io/items/:itemId', { itemId: '#id' });
});
I want to instantiate it in a controller like:
module.controller('AddItemCtrl', ['app.entity.item', function(Item) {
$scope.item = new Item();
});
and bind it to the respective form in my template.
The actual problem that I have run into, is that I have to deal with 1:m tables.
An example of the entity structure would be:
{
"name": "",
"categories": [],
"list": [
{
"value": "",
"list": [
{
"value": "",
"list": [
{
"value": ""
}
]
}
]
}
]
}
(A more thorough example in the fiddle below)
Now the first two fields are obviously not the problem. It is the third one. The list. Each one of these lists can have a variable number of items.
I am currently using ngRepeat and an add(type, context) method, which adds a new set of fields to the scope (value field in this example and child lists for the first two levels), which will appear in UI by ngRepeat so the user can fill it up and submit it to the service.
First off, I have to define the structure, so the UI would not be empty when the page loads.
module.controller('AddItemCtrl', ['app.entity.item', function(Item) {
$scope.item = new Item({
"name": "",
"categories": [],
"list": [
{
"value": "",
"list": [
{
"value": "",
"list": [
{
"value": ""
}
]
}
]
}
]
});
});
But that is redundant. I have to do it everywhere!
Another issue is that when the item.$save is called, the model is emptied (perhaps re-instantiated?) and the fields inside the list property (managed by the ngRepeat directive) are gone.
So I'm wondering, what would you do under such circumstances.
Is there a way to define the entity (resource) structure?
SAMPLE: http://jsfiddle.net/g15sqd5s/3/
trying to give simple answer - for simple structures I would use something like
module.factory('Item', function($resource) {
var resource = $resource('http://xmpl.io/items/:itemId', { itemId: '#id' },
// you can also define transformRequest here:
{ transformRequest: function(data) {
// data can be transformed here
return angular.toJson(data);
}});
return angular.extend(resource.prototype,
{
name: null,
categories: []
});
});
but then be aware of need to 'flatten' the object.
and for the more complex model I would check restangular
similar topic is also discussed here:
How can I extend the constructor of an AngularJS resource ($resource)?
I would go ahead and revise my model structure in the backend in the first place - the models on the client side should merely follow the ones already defined, rather than being re-defined in a transform block. So, to answer your question, the "default" model structure comes from the server. What you get in your $resource objects has the structure of what your server returns.
To start off, is it really ok to invoke $save on the Item model when the user has populated some values? What we want to save are obviously the lists associated with an item, not the item itself. A separate resource defined in the backend, say items/<item_id>/list, may be a cleaner solution. It may not scale very well, as you'll have to make a separate GET request for each item to fetch its list, but that's the proper RESTful way to do it.
Extending this approach to the example in your fiddle, I imagine a routing scheme like buildings/<building_id>/floors/<floor_id>/units/<unit_id> would be a proper solution. Making a GET request to buildings/ should yield you a list of buildings; each building in the array returned should be an instance of a Building model, which has the proper URL set so the user can perform a single POST and update only the building name, instead of sending back the whole structure back to the server. Applying this recursively to the nested resources should give you a clean and concise way to deal with model changes.
Regarding the UI part - I would go ahead and define three directives for buildings, floors and units, and let each one manage an array with the respective resources, also taking care for the UI bindings to the model values.
So how could a Building model look like?
var BuildingResource = $resource('/buildings/:id', { id: '#id' });
Invoking BuildingResource.query() should yield an array of existing buildings. Adding a new building could look like this:
var newBuilding = new BuildingResource();
newBuilding.$save().then(function(building) {
$scope.buildings.push(building);
}, function(errData) {
//Handle error here...
});
It should be easy to extend this pattern for the rest of the resources - note that what the server needs to return for every building is just the name and the id; knowing the id is sufficient to construct an URL (and a $resource object, respectively) to fetch the needed child resources (in this case, floors).

mongodb - retrieve array subset

what seemed a simple task, came to be a challenge for me.
I have the following mongodb structure:
{
(...)
"services": {
"TCP80": {
"data": [{
"status": 1,
"delay": 3.87,
"ts": 1308056460
},{
"status": 1,
"delay": 2.83,
"ts": 1308058080
},{
"status": 1,
"delay": 5.77,
"ts": 1308060720
}]
}
}}
Now, the following query returns whole document:
{ 'services.TCP80.data.ts':{$gt:1308067020} }
I wonder - is it possible for me to receive only those "data" array entries matching $gt criteria (kind of shrinked doc)?
I was considering MapReduce, but could not locate even a single example on how to pass external arguments (timestamp) to Map() function. (This feature was added in 1.1.4 https://jira.mongodb.org/browse/SERVER-401)
Also, there's always an alternative to write storedJs function, but since we speak of large quantities of data, db-locks can't be tolerated here.
Most likely I'll have to redesign the structure to something 1-level deep, like:
{
status:1,delay:3.87,ts:138056460,service:TCP80
},{
status:1,delay:2.83,ts:1308058080,service:TCP80
},{
status:1,delay:5.77,ts:1308060720,service:TCP80
}
but DB will grow dramatically, since "service" is only one of many options which will append each document.
please advice!
thanks in advance
In version 2.1 with the aggregation framework you are now able to do this:
1: db.test.aggregate(
2: {$match : {}},
3: {$unwind: "$services.TCP80.data"},
4: {$match: {"services.TCP80.data.ts": {$gte: 1308060720}}}
5: );
You can use a custom criteria in line 2 to filter the parent documents. If you don't want to filter them, just leave line 2 out.
This is not currently supported. By default you will always receive the whole document/array unless you use field restrictions or the $slice operator. Currently these tools do not allow filtering the array elements based on the search criteria.
You should watch this request for a way to do this: https://jira.mongodb.org/browse/SERVER-828
I'm attempting to do something similar. I tried your suggestion of using the GROUP function, but I couldn't keep the embedded documents separate or was doing something incorrectly.
I needed to pull/get a subset of embedded documents by ID. Here's how I did it using Map/Reduce:
db.parent.mapReduce(
function(parent_id, child_ids){
if(this._id == parent_id)
emit(this._id, {children: this.children, ids: child_ids})
},
function(key, values){
var toReturn = [];
values[0].children.forEach(function(child){
if(values[0].ids.indexOf(product._id.toString()) != -1)
toReturn.push(child);
});
return {children: toReturn};
},
{
mapparams: [
"4d93b112c68c993eae000001", //example parent id
["4d97963ec68c99528d000007", "4debbfd5c68c991bba000014"] //example embedded children ids
]
}
).find()
I've abstracted my collection name to 'parent' and it's embedded documents to 'children'. I pass in two parameters: The parent document ID and an array of the embedded document IDs that I want to retrieve from the parent. Those parameters are passed in as the third parameter to the mapReduce function.
In the map function I find the parent document in the collection (which I'm pretty sure uses the _id index) and emit its id and children to the reduce function.
In the reduce function, I take the passed in document and loop through each of the children, collecting the ones with the desired ID. Looping through all the children is not ideal, but I don't know of another way to find by ID on an embedded document.
I also assume in the reduce function that there is only one document emitted since I'm searching by ID. If you expect more than one parent_id to match, than you will have to loop through the values array in the reduce function.
I hope this helps someone out there, as I googled everywhere with no results. Hopefully we'll see a built in feature soon from MongoDB, but until then I have to use this.
Fadi, as for "keeping embedded documents separate" - group should handle this with no issues
function getServiceData(collection, criteria) {
var res=db[collection].group({
cond: criteria,
initial: {vals:[],globalVar:0},
reduce: function(doc, out) {
if (out.globalVar%2==0)
out.vals.push({doc.whatever.kind.and.depth);
out.globalVar++;
},
finalize: function(out) {
if (vals.length==0)
out.vals='sorry, no data';
return out.vals;
}
});
return res[0];
};

Resources