I'm currently using angularJS and phonegap to build a test application for Android / iOS.
The app use only text data stored in a Firebase database. I want the app to have its own local database (used when the device is offline) and sometime (when the device is online)
sync with a Firebase database.
The offline mode uses the storage API of phonegap/cordova. Could I just check the device's online state and backup the online database periodically ?
Any clues on how I can achieve this ? Last time a similar question was asked, the answer was "not yet"... (here)... because it focused on a hypothetical Firebase feature.
If Firebase is online at the start and loses its connection temporarily, then reconnects later, it will sync the local data then. So in many cases, once Firebase is online, you can simply keep pushing to Firebase during an outage.
For true offline usage, you will probably want to monitor the device's state, and also watch .info/connected to know when Firebase connects.
new Firebase('URL/.info/connected').on('value', function(ss) {
if( ss.val() === null ) /* firebase disconnected */
else /* firebase reconnected */
});
The way to achieve this with the current Firebase toolset, until it supports true offline storage, would
keep the local data simple and small
when the device comes online, convert the locally stored data to JSON
use set() to save the data into Firebase at the appropriate path
Additionally, if the app loads while the device is offline, for some reason, you can "prime" Firebase by calling set() to "initialize" the data. Then you can use Firebase as normal (just as if it were online) until it comes online at some point in the future (you would also want to store your local copy to handle the case where it never does).
Obviously, the simpler the better. Concurrent modifications, limits of local storage size, and many other factors will quickly accumulate to make any offline storage solution complex and time consuming.
After some time, I would like to add $0.03 to #Kato's answer:
Opt to call snapshot.exists() instead of calling snapshot.val() === null. As the documentation points out, exists() is slightly more efficient than comparing snapshot.val() to null.
And if you want to update data prefer to use the update() method rather then set(), as the last will overwrite your Firebase data. You can read more here.
Related
I'm trying to figure out how to create an offline / online approch to use within a huge application.
Right now, each part of the application has its own model and datalayer, who directly read / write data from / to SQL. My boss is asking me to create a kind of buffer that, in case of connectivity failure, might be used to store data until the connection to SQL return active.
What I'm trying to create is something like this: move all datalayers into a servicestack service. Each "GET" method should query the database and store the result into a cache to be reused once the connection to SQL is not available. Each "POST" and "PUT" method must execute their actions or store the request into a cache if the connection fail. this cache must be cleared once the connection to SQL is restored.
How can I achieve this? Mine is a WPF application running on Windows 10.
Best regards
Enrico
Maintaining caches on the server is not going to help create an offline Application given the client wouldn't have access to the server in order to retrieve those caches. What you'd need instead is to maintain state on the client so in the event that network access is lost the client is loading from its own local caches.
Architecturally this is easiest achieved with a Web App using a Single Page App framework like Vue (+ Vuex) or React (+ Redux or MobX). The ServiceStack TechStacks and Gistlyn Apps are good (well documented) examples of this where they store client state in a Vuex store (for TechStacks created in Vue) or Redux Store (for Gistlyn created in React), or the Old TechStacks (created with AngularJS).
For good examples of this checkout Gistlyn's snapshots feature where the entire client state can be restored from a single serialized JSON object or approach used the Real Time Network Traveler example where an initial client state and delta's can be serialized across the network to enable real-time remote control of multiple connected clients.
They weren't developed with offline in mind, but their architecture naturally leads to being offline capable, courtesy of each page being first loaded from its local store then it fires off a Request to update its local cache which thanks to the reactivity of JS SPA fx's, the page is automatically updated with the latest version of the server.
Messaging APIs
HTTP has synchronous tight coupling which isn't ideal for offline communication, what you want instead is to design your write APIs so they're One Way/Asynchronous so you can implement a message queue on the client which queues up Request DTOs and sends them reliably to the server by resending them (using an exponential backoff) until the succeed without error. Then for cases where the client needs to be notified that their request has been processed they can either be done via Server Events or via the client long-polling the server checking to see if their request has been processed.
My question is about how to store data which once was received online and still can be processed after the mobile device got offline and/or was restarted.
I'm using AngularJS with Ionic (PhoneGap) for building apps. But my question is not explicitly adressing these technologies.
Best practices, patterns or algorythms would be very helpful to me or even some useful articles or key words.
1) The most simple challenge is to make my app more user-friendly by making its functionality usable not only if the device is online but also in offline mode. In my case this implies that I have to make the last fetched online data available for later use (while device is offline and also after restarting the device!).
2) A bit more difficult is to reduce the communication costs by only synchronizing the server side changed data when the device reconnects to the internet.
3) Entities can also be produced on client side while the device is offline and they must get synchronized to the server too. There are no potential risks of conflicts because the users don't share Entities with write access.
4) I use Googles and Apples push services to inform the devices about newer entity versions, which should get updated on client side. So polling isn't needed.
Client side technologies: Javascript, AngularJS Framework, Ionic Framework, SQLite (WebSQL) or IndexedDB, PhoneGap (Cordova)
Server side technologies: Java EE, JPA, MySQL
Data Format and communication: JSON over REST / http, Googles and Apples push services for server-to-client messaging
1) Store the needed data inside a local SQLite database, and pull it out when the app starts/resumes.
2)In the MySQL database you need a table that creates new entries when you update/change/create content. You would need to store an id and a timestamp (maybe a boolean value if the content was deleted).
On the device you would make a request to the server to send the data from that table and compare it with the locally stored data. If there is a new id or the timestamp has change make a new request to pull the updated data.
3)Store the created data locally with a flag that it isn't synced with the server. When the device then goes online again check for not synced flags and send the data to the server with an identifier to know which device it comes from and where to save it.
4)See 2)
You could make a Java script which checks every x minutes for updated entries and send an automatic push Notification with it. What you would need is 2 tables, one with the newest updates and one with the updates that got pulled by the device(just id's and timestamps, not all the data).
I hope this was helpful, if something new comes to my mind I will update this answer.
I'm making a mobile HTML5 webapp and I'm wondering if I can use local storage to enable users to still use the app when they lose internet access.
The basic idea would be that when they have wi-fi / 3G they download the HTML and data, but when they lose internet access they can at least access the last version with old cached data (with a warning that data may not be up to date until they get internet access again).
Is this possible with local storage ?
Certainly. One of the purposes with localStorage is to enable offline applications.
you can check (see here for details):
window.navigator.onLine
to see if you are online or offline, or simply:
window.addEventListener("offline", offlineFunc, false)
window.addEventListener("online", onlineFunc, false)
and if offline you serve the stored content from localStorage by updating the page partially.
Another way of doing this is to use a cache manifest.
Here you can define which files shall be available if browser become offline, and which require network and so forth.
See here for details on that:
https://en.wikipedia.org/wiki/Cache_manifest_in_HTML5
http://diveintohtml5.info/offline.html
Besides from localStorage you can also use IndexedDB which also allow you to store Blobs (or files) (File API is coming, currently only for Chrome).
User makes HTTP-request to the server. This request is processed with an object of some class, let's call it "Processor". Then the same user in two minutes makes another HTTP request. And I want it to be processed with the same instance of Processor as the first one. So basically I want to keep the state of some object among several requests.
I know that I can save it each time to the datastore and then load back, but this approach seems to be very slow. Is there a way to store objects in some RAM place?
How about using memcache?
You can't ensure that consecutive requests to your app will go to the same instance, but memcache can help reduce or eliminate the overhead of accessing the datastore for each request.
It sounds like you are describing is a session.
I am not sure which language runtime and web framework you are using, but it is sure to include support for a sessions. (If you are using Java you will need to enable it.)
The standard session mechanism puts a small ID in a cookie that is stored in the user's browser. On every request, each of which could be go to a different application server, this ID is used as a key to read and write persistent information from the data store.
If the datastore accesses are too slow for you I would suggest not using memcache for this session storage, because memcache is by design unreliable, so the user's session information could disappear at any time, which would be a bad experience for them.
If the amount of data you want to store is less than about a few kilobytes, then I recommend doing what Play Framework does, which is to encrypt your session data and store it directly in a cookie stored in the user's browser. This is fast and truly stateless.
If you have more data than can be stored in a cookie, and you don't want to use the data store, you could could use JavaScript local storage on the browser, and use AJAX calls to communicate with the server. (If you want to support older browsers you may need to use the jStorage wrapper library.)
If memcache isn't enough, you could use backends to maintain state. Use a resident backend (or a set of them) and route incoming requests from the frontend to the backend machine that has the state.
Docs: Python Java
I'm developing a spring based web application that is deployed on Google app Engine.
I have a manager that store data in application scope. I'm using a spring bean (singleton) that hold a map and simply perform get and remove from the map,
however GAE is a distributed environment and this design has a problem since each application instance will have its own manager and request are not guaranteed to be made to the same application instance.
So I've looked around and found 2 possible solution:
use data store
use memcache
storing the data will cause a lot of read and writes and eventually I do not need the data to be saved.
second looked promising but Google mentioned that:
In general, an application should not expect a cached value to always be available.
and I need a guarantee that when I ask my manger to get a value it will return it.
is there any other solution?
did I miss anything?
A common solution is storing the values in a memcache, backed by the datastore.
First fetch the application-scope value from the memcache, and if the memcache returns zero-result (a cache-miss event), fetch the value from the datastore and put the fetched value on the memcache.
In this way, the next fetch to the memcache would return your application-scope data, reducing the need for a (relatively) costly read to the datastore.
App Engine memcache is a least-recently used cache, so values that are frequently read would suffer from few cache-miss event.
A more complex solution to have a application-scope value is to store the values in-memory at a resident backend, and have all your other instances request/update the value from/to this particular backend via a servlet handler.