I want to make a search into google patent using the following URL which is obsolete
https://ajax.googleapis.com/ajax/services/search/patent?v=1.0&q=thumb%20wrestling%20apparatus&userip=192.168.1.102
It gives me limited number of records per page.
But at the end of the JSON it also returns the cursor which has start and label keys. So my question is that how can I use that cursor to show all the records in my search. Like if there are 8 pages and each page contains 4 records so I want to show all 32 records on my UI.
How can I achieve that?
And second question that is there REST APi for google patent search? If yes then how can I search the patent using REST API and how can I get all the records on one page?
It looks like the API is restricted to a maximum of 8 results per request (you can increase your current 4 results to 8 by using the query param rsz=8.
So I guess the only way to get all results is by performing multiple requests. So if the current page info data is...
"pages":[
{"start":"0","label":1},
{"start":"8","label":2},
{"start":"16","label":3},
{"start":"24","label":4},
{"start":"32","label":5}
]
You would make 5 requests chaining the start param start=0, start=8 ... and so on, extracting the results and pushing to an array store. If you're not already I recommend using something like Restangular, as it would make this process much easier.
Depending on how your UI is set out, it would be nice maybe to do this with some lazy loading as the user is scrolling through the list?
Related
I have a web form from which I want to save the data from the very first page to an index by using this sample code below.
Startup.Init<SomeModel>("http://localhost:8983/solr/somemodels");
var solr = ServiceLocator.Current.GetInstance<ISolrOperations<Quote>>();
solr.Add(new SomeModel() {Id=1001; Content="Some Content"});
solr.Commit();
On the very last page, the user is given a chance to change/update his entries on the form. Should I also use this line of code?
solr.Add(new SomeModel() {Id=1001; Content="New Content"});
Also, is this a good practice - having indexes updated in this fashion?
You can consider the below things.
Is there any chance that user does something in the first page and drops off in before reaching the last page?
If this is valid scenario and data loss is not acceptable, you should
save the data.
If you only want to save the data when use reaches at the last page, then you should do the below.
You should save the data which you want to be saved between pages. At the last page, you save it too SOLR.
Hope you are also using some backend data source to save the data and solr for search use cases. In this case, it is advisable to update one doc once per operation.
In case of Cassandra as backend. there are chances for tombstone when you update a document multiple times.
For a weblog I am trying to get the top 10 popular posts from for example the last month. I figured I'd get the data out of Matomo, as that's already tracking visits and has an API. I've never used this API before though, so I've been reading the documentation and trying out some things. I am able to get data from the API using the Actions.getPageUrls method. However, when I try to filter using segment=^http://example.org/post I still get data from other URL's. It looks like it filters on session and gives back all data from the sessions that have at least 1 page that conforms to the filter.
The full URL I'm using is: http://example.org/matomo/index.php?&module=API&token_auth=12345&method=Actions.getPageUrls&format=json&idSite=1&period=month&date=today&expanded=1&segment=pageUrl%3D%5Ehttp%253A%252F%252Fexample.org%252Fpost. I've also tried with less and no URL encoding for the segment, but that doesn't seem to make a difference. If I use a URL that doesn't exist I get an empty array returned.
Am I doing something wrong? Is there a different way to only get the top pages with a URL starting with http://example.org/post? Or do I have to sift through the data myself to only get the pages I want?
I am using Matomo version 3.13.5.
I figured it out. There is no need to use segment. This can be achieved using the flat, filter_column and filter_pattern parameters.
Setting flat=1 will make it so all pages are returned in a single array, instead of hierarchically.
With filter_column and filter_pattern I can filter the results.
The URL I use now is: http://example.org/matomo/index.php?&module=API&token_auth=12345&method=Actions.getPageUrls&format=json&idSite=1&period=month&date=today&flat=1&filter_column=label&filter_pattern=%5E%2Fpost%2F. This does exactly what I want.
The unencoded pattern is ^/post/, so this will filter out any page that does not start with /post/.
I have a big data portion that I would like to post in a table. However, the data should be sorted and paginated. I know I am able to pass the whole data to the client at once and then paginate it using angular, but this will be too slow. I prefer to pass the data page-by-page, so one the client want to open a page from a table to load the data for it.
Up until now I have created an API that returns me the data that I need, based on the page number and the number of rows on the page. However, I don't know how to use it with AngularJS.
Can you please help me?
It looks like a backend problem. If you are using a standard restful backend, use the limit/skip parameters, you can encapsulate into a paginate.
Example:
localhost:1337/dataTable?skip=0&limit=100
localhost:1337/dataTable?skip=100&limit=100
localhost:1337/dataTable?skip=200&limit=100
...
On the frontend use a table object like ng-Table, and use the pages to keep track of the offset, the page number and the total items available.
skip = (pagNum - 1 * pageSize)
limit = pageSize
Make your backend return you the page you want plus the available dataNumber so you can build the pages controller.
Documentation for skip/limit on sails
http://sailsjs.org/documentation/reference/waterline-orm/queries/limit
http://sailsjs.org/documentation/reference/waterline-orm/queries/skip
Best approach is to keep track of the limit and offset in your controller. Then when user selects new page (offset) or changes items per page (limit), update the corresponding values and use them to make a new http request.
You could call a function on ng-change of a dropdown and that drop down would contain values of page number and number of records to fetch. Or you can provide two text boxes one for page number other for number of records and keep a button and on its ng-click event that will take value of those text boxes and post to your server and bring back data based on new values in text boxes
I read the couple threads on SO about paging using app engine and objectify (via cursors), and this link here: https://developers.google.com/appengine/articles/paging
So far all the discussions with cursors have been about just providing a previous/next page. What I want is to generate a list of links for the next page, the 'next+1' page, the 'next+2' page, etc.
The bottom of any google search query has a list of page links https://www.google.co.kr/?gfe_rd=cr&ei=wZ0MVKKLCIrH8geM74DABQ&gws_rd=ssl#newwindow=1&q=paging
I can generate those using limit and offset.. But we know that is inefficient and offset is limited to a value of 1000 max.
So,
1) Am I stuck with just using limit,offset and only providing page links for 1000 entities? So, at 20 results per page, 50 pages, and that's it? Just use 'Next page' thereafter?
2) Should I forget about page links altogether and just use Next/Prev links via cursors?
Appreciate any discussion!
I believe there is no other option than to page through the complete set and save the cursors at each of the page offsets. This is a one time effort and users clicking the link can then use the cursor to navigate directly to the correct position without incurring the offset cost.
This is only a viable solution if your data set is not changing too often, so your cursors stay valid.
I'm doing some searching using the Instagram REST API. Specifically, the endpoint I'm using is /tags/{tag-name}/media/recent. I'm first getting the number of media items with the tag in question using /tags/{tag-name}, which is ~445K. I'm going through all the media items using the 'next_max_tag_id' that comes along with the response. I don't know how long this takes, but at some point, the max IDs start to loop. Not back to back, but as of ~250K, it's just constantly looping over the same 5 next_max_tag_id values.
Am I doing it wrong by counting the number of items I've processed and comparing it to the number of media items I'd previously gotten?
How do you know when you've reached the end of the list?
Look at the answer here:
Instagram realtime get post from callback
I think this scenario is the same as yours.