Firebase - Lazy Loading List of Data - angularjs

I have a requirement where I want to display say the first 10 entries in a list and once the user scrolls down I would like to append the next 10 entries. I am currently using Angularfire and all the documentation specifies that I should not do array operations on a $FirebaseArray:
This array should not be directly manipulated. Methods like splice(), push(), pop(), and unshift() will cause the data to become out of sync with server.
So my options are to load the next 10 entries and:
Use $add(), which would write them to the server again (think this could cause some nasty recursion)
Use concat, in which case my data will get out of sync with the server
Get the list again but adjust the limit to be 20, which I think would cause all the data to be reloaded defeating the purpose of lazy loading.
Here is the code that initially loads the list (based on the Angularfire seed app):
var lastKey = null;
var firstKey = null;
$scope.messages = fbutil.syncArray(userMessages,{'limit':10});
$scope.messages.$loaded(function(data){
lastKey = data.$keyAt(data.length-1);
firstKey = data.$keyAt(0);
});
And here is the code that is triggered when the user scrolls down:
$document.on('scroll', function() {
if($(window).scrollTop() + $(window).height() > $(document).height() - 100) {
var newMessages = fbutil.syncArray(messagePath,{'limit':10,'startAt':lastKey});
newMessages.$loaded(function(data){
lastKey = data.$keyAt(data.length-1);
firstKey = data.$keyAt(0);
$scope.messages.concat(newMessages);// this is probably a bad idea
});
}
});

Based on Kato's comment the following is the best solution given the current API.
var limit= 10;
$scope.messages = fbutil.syncArray(messagePath,{'limit':limit});
And the scroll trigger
$document.on('scroll', function() {
if($(window).scrollTop() + $(window).height() > $(document).height() - 100) {
limit += 10;
$scope.messages = fbutil.syncArray(messagePath,{'limit':limit});
}
});

Related

in angularjs, With time, $interval delay is reducing automatically. would it be an issue by having multiple $intervals in single controller?

I have an AngularJs app. in a single controller, i have to pull data from 3 different source and display the data in 3 different sections. (in top , mid and bottom section). data in each section would vary in a set delay. for top section, i have used Set interval and clear interval. for mid and bottom section , i have used $interval.For mid section, the delay is set to 7 and for bottom section, it is set to 10 sec.
when i launch the application, everything works fine. but with time, the mid and bottom section varies in every 2 or 3 seconds instead of 7 and 10 respectively. i dont know where i am doing the mistake. i have not added $interval.cancel as i want the mid and bottom section to bring the data in 7 and 10 sec delay continuously without stopping. i am also using socket to bring the data for all the sections. I have not used $scope.on'Destory' . i don't if that has any impact here. below is the code. Any help would be greatly appreciated.
socket.on('midsection', function (data) {
tableJson = [];
SplicedJson = [];
jsoninput = [];
jsoninput.push(data);
// splitting single array in to two array's so that we can display them one after the other after some delay
SplicedJson.push(jsoninput[0].splice(0, 6));
tableJson.push(jsoninput[0]);
$scope.tableData = tableJson;
//Creating interval and setting counter just to swap the json source for display
var Counter = 1;
$interval(function () {
Counter++;
if (Counter % 2 === 0) {
$scope.tableData = SplicedJson;
}
else {
$scope.tableData = tableJson;
}
}, 7000);
});
socket.on('lastSection', function (data) {
if (data.length > 3) {
array1.push(data[0]["publisher"]);
array1.push(data[1]["publisher"]);
array1.push(data[2]["publisher"]);
array2.push(data[0]["title"]);
array2.push(data[1]["title"]);
array2.push(data[2]["title"]);
$scope.msg = array1;
$scope.msg2 = array2;
$interval(function () {
data = shuffle(data); // caling custom built shuffle fn to shuffle the data to display random data everytime
console.log('Shuffled', data);
array1= [];
array2= [];
array1.push(data[0]["publisher"]);
array1.push(data[1]["publisher"]);
array1.push(data[2]["publisher"]);
array2.push(data[0]["title"]);
array2.push(data[1]["title"]);
array2.push(data[2]["title"]);
$scope.msg = array1;
$scope.msg2 = array2;
},10000);
}
});
Code # connection launch is below
io.on('connection', function (socket) {
socket.join(socket.handshake.query.room)
var entExFilepath = './midsectionData.json';
var parsedJSON = JSON.parse(fs.readFileSync(entExFilepath, 'utf8'));
fs.watchFile(entExFilepath, function () {
parsedJSON = JSON.parse(fs.readFileSync(entExFilepath, 'utf8'));
io.sockets.emit('midsection', parsedJSON);
});
io.sockets.emit('midsection', parsedJSON);
if (feedresults.length > 3) {
io.sockets.emit('lastsection', feedresults);
}
I found the cause for this issue. the problem was when i send the data for mid and last section, i was doing socket.emit which was equivalent to broadcasting the answer to all the clients.So Whenever a new request comes to the server, socket.emit was being called and sent the data to the client. (which means it also calls the internal block when ever a new request comes. so the interval is called again and again).I should have done like below. This resolved the issue.
//io.sockets.emit('lastsection', feedresults);
io.sockets.in(socket.handshake.query.room).emit('lastsection', feedresults);// This should send the data to the current client
Hope this helps others

ui-grid infinite scroll with row processor filtering

I have an angularjs app using ui.grid with the infinite scrolling module. I am using whole row filtering as described in the documentation like so:
function MyController($scope){
var that = this;
var getData = function(){
//logic left out for brevity
};
var onRegisterApi = function(gridApi){
gridApi.grid.registerRowsProcessor(function (rows) {
return that.filterRowProcessor.apply(that, [rows]);
}, 200);
gridApi.infiniteScroll.on.needLoadMoreData($scope, getData);
};
this.options["onRegisterApi"] = onRegisterApi;
}
//...code excluded for brevity...
MyController.prototype.filterRowProcessor = function(renderableRows){
renderableRows.forEach(function(row) {
if (this.selectedMap[row.entity["Id"]]) {
row.visible = false;
}
});
return renderableRows;
}
The idea is to filter out rows which have an Id belonging to a specific collection; which works as designed. My problem is that when I get my first page of data the filter row processor removes enough rows from visibility that my scroll bar disappears. This in turn causes the infinite scroll api to never raise the "needLoadMoreData" event.
Is this uncharted territory, or is there a way around this? I am also open to not filtering by that mechanism if its easier to do another way.
UPDATE (01/08/2016)
I have found a work around that I don't like very much. Essentially I have a known page size and if the data coming in to the grid is less than that page size and my callback returns false for "end of query", I automatically issue a next page query. I would rather find a solution via the grid api, but for now this will work.
if(this.itemsSource.data.length < constants.PAGE_SIZE && !continuation.endOfQuery){
//call get data again
}
After thinking about it for a while I decided on the below method as my solution. I am still open to suggestions if it makes more sense to do it a different way. Rather than relying on a length of data (which only loosely translates to having a scroll bar) I decided to calculate the height of the total rows visible, compared to the viewport of the grid.
//this method get called from the callback from needsMoreData
//hasMoreData is the same boolean flag sent in to dataLoaded
var shouldRetrieveMore = function (gridApi, hasMoreData){
if (!hasMoreData) {
return false;
}
var totalCountOfRows = gridApi.grid.getVisibleRowCount();
if (totalCountOfRows === 0) {
return true;
}
var height = gridApi.grid.getViewportHeight();
var heightOfRow = gridApi.grid.getVisibleRows()[0].$$height;
return ((heightOfRow * totalCountOfRows) <= height);
}
One additional addendum to the solution could be to sum the $$heights of all the rows, but I decided against it since in my uses they are always the same height.

AngularJS - wait for data from server and add them correctly to list

I am currently working on an AngularJS project in which I have to get project information, for a specific month and year, from the server and show them to the user.
First of all I'm getting a list of Project Id's (projectList), which can be variable, and then I need to get the info on those projects for a specific year and month. With this code I'm trying to get the data and to refresh the data when the last projects is successful. After the data is fetched, I use a ng-repeat to show it to the user.
$scope.getData = function(){
$scope.projectInfoList = [];
for(var index=0; index < $scope.projectList.length; index++){
projectService.getProject($scope.model.year, $scope.model.month, parseInt($scope.projectList[index]) ).success(function(data){
var listInput = { projectID : $scope.projectList[index], data : data};
$scope.projectInfoList.push(listInput);
if(index == $scope.projectList.length - 1){
$scope.$apply();
}
});
};
}
This has 2 mistakes.
It adds only data to the last index.
It doesn't refresh the data immediately when I request data for another month or year
I have looked for solutions with $q.all but I'm not sure how I would use it together with variable amount of functions of 'projectService.getProject(..)'
The anonymous callback that you give to the function success use a closure to the variable index.
But, your anonymous callback will be called asynchronously (when the call will be done). So when it will be called when index will be the last index of the array (so here $scope.projectList.length - 1).
To avoid that, you can use the following pattern:
for(var index=0; index < $scope.projectList.length; index++){
(function (index) {
projectService.getProject($scope.model.year, $scope.model.month, parseInt($scope.projectList[index]) ).success(function(data){
var listInput = { projectID : $scope.projectList[index], data : data};
$scope.projectInfoList.push(listInput);
if(index == $scope.projectList.length - 1){
$scope.$apply();
}
});
})(index)
}
Your second mistake is probably because you change the reference of the array projectInfoList in your function with $scope.projectInfoList = [];.
Take a look at this post for more details on this last problem: ng-repeat not updating on update of array

AngularJS : Why the data is not displayed in view may I use $scope.apply?

I am getting data from service and display on view using ng-repeat .Actually I am getting event when user scroll to bottom mean when user reached to bottom I will do something.When It reached to bottom I am changing the contend of my array .I am getting the correct contend in ng-repeat array (display array) but it is not reflect on view why ? May I use $scope.apply() or $scope.digest()
Here is my code
http://plnkr.co/edit/XgOxJnPXZk4DneJonlKV?p=preview
Here I am changing the contend of my display array which is not reflect on view
if (container[0].offsetHeight + container[0].scrollTop >= container[0].scrollHeight) {
if(scope.var<scope.arrays.length)
scope.display=[];
var nextvar =++counter;
var increment=counter+1
console.log("nextvar:"+nextvar+"increment:"+increment)
scope.display=scope.arrays[nextvar].concat(scope.arrays[increment]);
console.log(scope.display)
}
As #Claies mentioned you should use apply(). Though the digest() would probably have worked as well.apply() calls digest() internally. He also mentioned that your variable that seems to be storing the page number gets reset to 0 each time you scroll. You should store that in a scope variable outside that handler.
I tried to fix with minimum change
http://plnkr.co/edit/izV3Dd7raviCt4j7C8wu?p=preview
.directive("scrollable", function() {
return function(scope, element, attrs) {
var container = angular.element(element);
container.bind("scroll", function(evt) {
console.log('scroll called'+container[0].scrollTop);
var counter = scope.page;
if (container[0].scrollTop <= 0) {
if (scope.var > 0)
scope.display = scope.arrays[--scope.var].concat(scope.arrays[scope.var+1]);
}
if (container[0].offsetHeight + container[0].scrollTop >= container[0].scrollHeight) {
if (scope.var < scope.arrays.length)
scope.display = [];
var nextvar = ++counter;
var increment = counter + 1
console.log("nextvar:" + nextvar + "increment:" + increment)
scope.display = scope.arrays[nextvar].concat(scope.arrays[increment]);
console.log(scope.display)
scope.page = counter;
}
scope.$apply();
});
};
})
generally I would have implemented this differently. For example by having a spinning wheel on the bottom of the list that when displayed you get the rest of data.
It is difficult to give you a full working plunker. Probably you should have multiple JSON files in the plunker, each containing the data for one page so that we can add the data to the bottom of the display list.
After you modify display array you just have to call scope.$apply() so that it runs the $digest cycle and updates the view. Also you need the initialize scope.var either in your controller or the directive and modify it conditionally.
I dont if this is what you want. I have modified the plunker take a look.
http://plnkr.co/edit/J89VDMQGIXvFnK86JUxx?p=preview

eXTjS 3.3.1 LiveGrid store.each gives only buffered rows

I use ExtJs 3.3.1 because many extensions don't work under 4.xx
One of these extensions is LiveGrid.
I can't try but i suppose a simmilar thing happens with a 4.x buffered grid.
When i do a report of the lines visible in the grid only the buffered lines are returned, i reposition the current record but the loading of the rest of the records only happens after the reporting finishes. How can i get all the records ?
In an button handler i call toReport(grid).
toReport = function(grid){
var store = grid.getStore();
var view = grid.getView();
store.each(function(record) {
Ext.defer(function(){
index = readRow(store, record);
if (index % 10 == 0){
view.focusRow(index);
}
}, 500, this);
});
console.log(output)
}
readRow = function(store, record){
output = "";
for (var xlCol=1;xlCol<record.fields.length+1;xlCol++){
var veld = store.fields.itemAt(xlCol-1).name;
waarde = record.get(veld);
if (realTypeOf(waarde)==="date"){
output += waarde.format("d-m-Y");
}else{
output += record.get(veld);
}
}
console.log(store.indexOf(record)+ " " + output);
return store.indexOf(record);
}
The grid needs to manipulate its store filters, sorters, paging, etc., in order to obtain the records it want to display. The store itself only keeps in memory the subset of records that matches its filters, etc. That is the way stores are designed in Ext: they are intended to be bounded to one and only one view.
I think in your case, the simplest solution is to create another store with a similar configuration, and use its load method with params such that you get all the records.
If you're reticent to fire multiple requests for retrieving essentially the same data, have a look at Ext.data.Proxy. Unlike stores, proxies are not bound to a specific view or task and can be shared between multiple store instances. So in theory, you can create a proxy that requests all the records from the server at once, and then feeds a subset of them to multiple stores. For that you'll have to implement the doRequest method of the proxy (or most probably overrides the one of the proxy you're already using).
I did find a solution by using recursion.
Like this the view kan keep up with the enumeration.
Ext.toExcel = function(grid){
var store = grid.getStore();
var view = grid.getView();
readRow(store, view, 0);
}
readRow = function(store, view, index){
output = "";
record = store.getAt(index);
for (var xlCol=1;xlCol<record.fields.length+1;xlCol++){
var veld = store.fields.itemAt(xlCol-1).name;
waarde = record.get(veld);
if (realTypeOf(waarde)==="date"){
output += waarde.format("d-m-Y");
}else{
output += record.get(veld);
}
}
//console.log(index+ " " + output);
if (index % 50 == 0){view.focusRow(index);}
Ext.defer(function(){
if (index < store.totalLength){
readRow(store, view, index+1);
}
}, 100, this);
}

Resources