I have a blob with over 5000 files. Checking the documentation and limited by pagination this amount.
I'm trying to do a paging in the APP LOGIC but I'm not able to do this loop. When I get the PAGE MARKER it returns strange information and I don't understand how to put it in the loop
How do I adjust my code for looping and paging the data inside the BLOB?
I set up this stream to get the paging
I created a string variable to view the PAGINATION content
When I run app logic and parse the variable, it returns this weird content instead of pages 1, 2...
When I try to configure the loop to go through all the pages I don't know what to inform the counter and the variable paging marker. How should I configure this part?
Instead of #equals(body('Lists_blobs_(V2)')?['nextPageMarker'], '') use #equals(variables('pagination'), '').
Before the end of the Until loop, set the pagination variable to Next page marker from Lists blobs (V2) 2 (the action inside the loop).
Related
For a weblog I am trying to get the top 10 popular posts from for example the last month. I figured I'd get the data out of Matomo, as that's already tracking visits and has an API. I've never used this API before though, so I've been reading the documentation and trying out some things. I am able to get data from the API using the Actions.getPageUrls method. However, when I try to filter using segment=^http://example.org/post I still get data from other URL's. It looks like it filters on session and gives back all data from the sessions that have at least 1 page that conforms to the filter.
The full URL I'm using is: http://example.org/matomo/index.php?&module=API&token_auth=12345&method=Actions.getPageUrls&format=json&idSite=1&period=month&date=today&expanded=1&segment=pageUrl%3D%5Ehttp%253A%252F%252Fexample.org%252Fpost. I've also tried with less and no URL encoding for the segment, but that doesn't seem to make a difference. If I use a URL that doesn't exist I get an empty array returned.
Am I doing something wrong? Is there a different way to only get the top pages with a URL starting with http://example.org/post? Or do I have to sift through the data myself to only get the pages I want?
I am using Matomo version 3.13.5.
I figured it out. There is no need to use segment. This can be achieved using the flat, filter_column and filter_pattern parameters.
Setting flat=1 will make it so all pages are returned in a single array, instead of hierarchically.
With filter_column and filter_pattern I can filter the results.
The URL I use now is: http://example.org/matomo/index.php?&module=API&token_auth=12345&method=Actions.getPageUrls&format=json&idSite=1&period=month&date=today&flat=1&filter_column=label&filter_pattern=%5E%2Fpost%2F. This does exactly what I want.
The unencoded pattern is ^/post/, so this will filter out any page that does not start with /post/.
I have a big data portion that I would like to post in a table. However, the data should be sorted and paginated. I know I am able to pass the whole data to the client at once and then paginate it using angular, but this will be too slow. I prefer to pass the data page-by-page, so one the client want to open a page from a table to load the data for it.
Up until now I have created an API that returns me the data that I need, based on the page number and the number of rows on the page. However, I don't know how to use it with AngularJS.
Can you please help me?
It looks like a backend problem. If you are using a standard restful backend, use the limit/skip parameters, you can encapsulate into a paginate.
Example:
localhost:1337/dataTable?skip=0&limit=100
localhost:1337/dataTable?skip=100&limit=100
localhost:1337/dataTable?skip=200&limit=100
...
On the frontend use a table object like ng-Table, and use the pages to keep track of the offset, the page number and the total items available.
skip = (pagNum - 1 * pageSize)
limit = pageSize
Make your backend return you the page you want plus the available dataNumber so you can build the pages controller.
Documentation for skip/limit on sails
http://sailsjs.org/documentation/reference/waterline-orm/queries/limit
http://sailsjs.org/documentation/reference/waterline-orm/queries/skip
Best approach is to keep track of the limit and offset in your controller. Then when user selects new page (offset) or changes items per page (limit), update the corresponding values and use them to make a new http request.
You could call a function on ng-change of a dropdown and that drop down would contain values of page number and number of records to fetch. Or you can provide two text boxes one for page number other for number of records and keep a button and on its ng-click event that will take value of those text boxes and post to your server and bring back data based on new values in text boxes
I have a view with several megabytes of data and I expect it to grow radically.
Controller index function is implemented in a default way, with pagination.
I would like to export this view to a csv without making much changes to the controller ( I'm fine to define header rows and rows to be included in a CSV but not to remove pagination as it brakes the html representation ).
The idea is simple: render csv view template, change the page, render another one. But how can I change current pagination settings in a custom view?
PS: I did take a look at the csv plugin. It doesn't work with pagination so I get out of the memory limits, it also creates a tmp file, I prefer to stream content on the fly.
I wouldn't use the paginator here, just get the total amount of records then do a while() loop and fetch the data in batches to avoid memory limitations. And send it as it comes from the DB directly to the client. Use the HTTP Client that comes with CakePHP and set the proper header properties.
See these two answers how to send it as stream:
send a file to client
PHP: stream remote pdf to client browser
Streaming a large file using PHP
I want to make a search into google patent using the following URL which is obsolete
https://ajax.googleapis.com/ajax/services/search/patent?v=1.0&q=thumb%20wrestling%20apparatus&userip=192.168.1.102
It gives me limited number of records per page.
But at the end of the JSON it also returns the cursor which has start and label keys. So my question is that how can I use that cursor to show all the records in my search. Like if there are 8 pages and each page contains 4 records so I want to show all 32 records on my UI.
How can I achieve that?
And second question that is there REST APi for google patent search? If yes then how can I search the patent using REST API and how can I get all the records on one page?
It looks like the API is restricted to a maximum of 8 results per request (you can increase your current 4 results to 8 by using the query param rsz=8.
So I guess the only way to get all results is by performing multiple requests. So if the current page info data is...
"pages":[
{"start":"0","label":1},
{"start":"8","label":2},
{"start":"16","label":3},
{"start":"24","label":4},
{"start":"32","label":5}
]
You would make 5 requests chaining the start param start=0, start=8 ... and so on, extracting the results and pushing to an array store. If you're not already I recommend using something like Restangular, as it would make this process much easier.
Depending on how your UI is set out, it would be nice maybe to do this with some lazy loading as the user is scrolling through the list?
I'm using Google AppEngine as backend and AngularJS as front end for web application I'm making. I'm presenting data in pages to the user.
AppEngine has the ability to select data and return 3 pieces of information: the items selected, indication if there are more items and cursor for the next page.
I need to return all 3 pieces to the client app so it can present the fetched items and allow the user to go to the next page.
I also would like to use ngResource to interact with the server.
The problem is that ngResource expect the list of items to be a list and here it is an object with the 3 pieces.
Is there a way to modify ngResource a bit so that after fetching the data it will use the items to build the array of items?
Not necessarily, ngResource can deal with arrays as well as single item or json object. The standard get operation returns a object whereas query returns array. Bottom line as long it is a valid json data ngResource would work.
You can always call get on the resource, get the data into a json object and then it can have sub-properties which can be of array type.
You can share your specific structure and the community can help you with understand how to access it using ngResource