I'm trying to figure out how to create an offline / online approch to use within a huge application.
Right now, each part of the application has its own model and datalayer, who directly read / write data from / to SQL. My boss is asking me to create a kind of buffer that, in case of connectivity failure, might be used to store data until the connection to SQL return active.
What I'm trying to create is something like this: move all datalayers into a servicestack service. Each "GET" method should query the database and store the result into a cache to be reused once the connection to SQL is not available. Each "POST" and "PUT" method must execute their actions or store the request into a cache if the connection fail. this cache must be cleared once the connection to SQL is restored.
How can I achieve this? Mine is a WPF application running on Windows 10.
Best regards
Enrico
Maintaining caches on the server is not going to help create an offline Application given the client wouldn't have access to the server in order to retrieve those caches. What you'd need instead is to maintain state on the client so in the event that network access is lost the client is loading from its own local caches.
Architecturally this is easiest achieved with a Web App using a Single Page App framework like Vue (+ Vuex) or React (+ Redux or MobX). The ServiceStack TechStacks and Gistlyn Apps are good (well documented) examples of this where they store client state in a Vuex store (for TechStacks created in Vue) or Redux Store (for Gistlyn created in React), or the Old TechStacks (created with AngularJS).
For good examples of this checkout Gistlyn's snapshots feature where the entire client state can be restored from a single serialized JSON object or approach used the Real Time Network Traveler example where an initial client state and delta's can be serialized across the network to enable real-time remote control of multiple connected clients.
They weren't developed with offline in mind, but their architecture naturally leads to being offline capable, courtesy of each page being first loaded from its local store then it fires off a Request to update its local cache which thanks to the reactivity of JS SPA fx's, the page is automatically updated with the latest version of the server.
Messaging APIs
HTTP has synchronous tight coupling which isn't ideal for offline communication, what you want instead is to design your write APIs so they're One Way/Asynchronous so you can implement a message queue on the client which queues up Request DTOs and sends them reliably to the server by resending them (using an exponential backoff) until the succeed without error. Then for cases where the client needs to be notified that their request has been processed they can either be done via Server Events or via the client long-polling the server checking to see if their request has been processed.
Related
I'm sending data from my backend every 10 seconds and I wanted to display that data in reactjs. I've searched on the net to use socket.io to display real-time data. Is there a better way to use it?
If you're dead set on updating your data every 10 seconds, it would make more sense to make a request from the client to the server, as HTTP requests can only be opened from client to server. By using HTTP requests, you won't need to use socket.io, but socket.io is an easy alternative if you need much faster requests.
Depending on how you are generating the data being sent from your backend, specifically if you are using a database, there is most likely a way to subscribe to changes in the database. This would actually update the data in realtime, without a 10 second delay.
If you want a more detailed answer, you'll have to provide more detail regarding your question: what data are you sending? where is it coming from or how are you generating it?
I'm working on an autodialer feature, in which an agent will get a call when I trigger the button from the frontend (using react js language), and then automatically all the leads in the agent assigned portal will get back-to-back calls from agent number. However, because this process is automatic, the agent won't know who the agent has called, so I want to establish a real-time connection so that I can show a popup on the frontend that contains information about the lead who was called.
I'm working with React-Redux and I've been looking at libraries to manage to sync my data with the server. I've seen libraries like react-offline and react-persist that handle parts of my problem, but not all of it. I have these requirements:
I have control of the API, so I can change it in any way necessary
I don't want to send the entire data set to the client. Users will only have access to a subset of the data in the database
I want the application to be able to be disconnected from the internet and still run normally (the area it functions in might are spotty, or totally disconnected)
On login, a client will request all the data it will have access to
At intervals, it will request a list of changes to the data it has access to so it can reconcile the client data locally (and invisibly)
At intervals, or when changes occur (configurable), it will send a batch of all changes the client has made to the server
Please let me know if there are redux libraries that handle all these requirements. Also, feel free to comment if some of the requirements feel like an overengineering or not best practice.
Thanks in advance!
My question is about how to store data which once was received online and still can be processed after the mobile device got offline and/or was restarted.
I'm using AngularJS with Ionic (PhoneGap) for building apps. But my question is not explicitly adressing these technologies.
Best practices, patterns or algorythms would be very helpful to me or even some useful articles or key words.
1) The most simple challenge is to make my app more user-friendly by making its functionality usable not only if the device is online but also in offline mode. In my case this implies that I have to make the last fetched online data available for later use (while device is offline and also after restarting the device!).
2) A bit more difficult is to reduce the communication costs by only synchronizing the server side changed data when the device reconnects to the internet.
3) Entities can also be produced on client side while the device is offline and they must get synchronized to the server too. There are no potential risks of conflicts because the users don't share Entities with write access.
4) I use Googles and Apples push services to inform the devices about newer entity versions, which should get updated on client side. So polling isn't needed.
Client side technologies: Javascript, AngularJS Framework, Ionic Framework, SQLite (WebSQL) or IndexedDB, PhoneGap (Cordova)
Server side technologies: Java EE, JPA, MySQL
Data Format and communication: JSON over REST / http, Googles and Apples push services for server-to-client messaging
1) Store the needed data inside a local SQLite database, and pull it out when the app starts/resumes.
2)In the MySQL database you need a table that creates new entries when you update/change/create content. You would need to store an id and a timestamp (maybe a boolean value if the content was deleted).
On the device you would make a request to the server to send the data from that table and compare it with the locally stored data. If there is a new id or the timestamp has change make a new request to pull the updated data.
3)Store the created data locally with a flag that it isn't synced with the server. When the device then goes online again check for not synced flags and send the data to the server with an identifier to know which device it comes from and where to save it.
4)See 2)
You could make a Java script which checks every x minutes for updated entries and send an automatic push Notification with it. What you would need is 2 tables, one with the newest updates and one with the updates that got pulled by the device(just id's and timestamps, not all the data).
I hope this was helpful, if something new comes to my mind I will update this answer.
I am starting to work with angular and am fascinated by the bi-directional data-binding capabilities and by its $http method, which lets me save changes in to my mysql database, without refreshing the page.
Another thing I am currently fascinated by is the real time capability across multiple clients using firebase. Here all clients are updated in REAL TIME, when the database receives any changes. I'd probably like to use firebase, but I would have to drop Laravel and MySql as a persistence layer entirely, which I would like to keep for the moment, since my application is already working in Laravel, just not in real time.
How would I go about having a Real Time application, which updates every client, without refreshing the view, in Laravel using MySQL and Angular?
If I am not mistaken, Pusher and PubNub, are providing this necessary open connection with the server using websockets, so when the server has something to share, angular will now and render it.
Since I would like to use Laravel and MySQL as a persistence layer, I am not sure, what the best way would be. I am not even sure, if I understood everything correctly, which I wrote above, since I am new to angular and real-time applications.
What would be the next necessary steps, to get some Real-Time capability into a PHP/MySQL application?
The solution for your problem is:
1º - open websocket connection with the websocket-server and subscribe a channel, after this send the data to your serve using ajax
tutorial angular pusher
2º - In server side, you get the data, saves to your database and send a 'PUBLISH' to the respective channel into websocket server
lib useful for this
3º - Through the subscribe gets the data in real time
Pusher.subscribe('channel', 'event', function (item) {
// code
});
I had a similar problem recently and I finally ended up using Redis publish/subscribe Redis. You can store data in the channel and then subscribe to any changes. When something changes you can send it to Pusher which will send it then to the clients.
I also recommend considering Node.js and Socket.io since you can achieve very good performance without third party service, and even if you don't have experience with node you can find very good examples on Socket.IO how to write an application.
For Redis there is a good library for PHP called Predis and there is Redis Node client as well, so you can mix it all together.
Have an Angular web app that track status of many objects. Now, I have a directive for each object to poll a nodejs server for status. This works but it is quite chatty and I am a bit worried about scalability. However, the backend does not support subscription so it could not use a pusher. One option is to let the nodejs server do the polling and push status back. Then nodejs need to keep a list of objects to poll and client probably need to check the message to determine whether a status needs update. Really appreciate your suggestion for a better strategy.
You can use firebase and load the data from a third party source into the firebase database then use angular-fire to get that data into the app... alternatively you could use a socket connection to DIY. In either scenario, I assume like you said the NodeJS portion on the server will need to handle polling the third party system for data and populating the firebase instance which would be a read-only layer as far as the system is concerned... changes would be sent directly to the third party and NodeJS would just be responsible for updating the firebase DB.