I am using Delphi with Firebird DB in a software that display many calculations and according to requirement I need to store and display the total whenever other dependent tables gets updated.
I am currently using the refresh method for several TIBTable components to display the result of these calculations to the user. considering this a multi user app and the fact that refresh make posting slow, is there any alternative to refresh or more optimized way ?
For a multi-user application, where clients need to receive notifications, one option is to use Firebird events to send a 'broadcast' message for every data change (SQL INSERT, UPDATE or DELETE).
Clients can 'register' (listen) for a specific message type, and whenever the Firebird server sends a message with this type, they will receive it, and run client application code, which in your case would refresh a user interface object to display the total.
While this can be a sufficient solution in many simple use cases, there are also some restrictions. I recently blogged about this topic here:
Firebird Database Events and Message-oriented Middleware
(I am author of middleware libraries for Delphi and Free Pascal)
There's really no way around refreshing to get new data. But if you were using IBObjects for your database access, you could use the TIBOTable.InvalidateBookmark method to have it only refresh specific rows. This is a lot faster than calling Refresh, which refreshes the entire dataset.
Looking at the documentation, it doesn't appear that TIBTable has anything comparable, unfortunately.
Related
I need to display on a datagrid the content of a table from my db (postgresql).
I'm using WPF, but I'm considering to migrating to Avalonia.
I can easy connect the db in vb.net or c# and populate datagrid.
Iusse is that anothe application can write into this db, so data can change, and I need to show ever the actual values.
So there are any way to "bind" the table with the datagrid in a way that the db change is notified? or should I simply put a timer and query every sec.?
Your simplest option is to poll every minute or whatever.
Postgresql has listen and notify but I would not recommend these.
https://www.postgresql.org/docs/9.0/sql-notify.html
Other (more) practical approaches I have seen are quite involved.
EG all updates to the database go via a web layer.
These changes then also store time and change data for every change. Table, id, change type and time.
Clients subscribe to a process uses the above to write changes ready for them to pick up.
The client then can pull just those changes which are applied to an on client cache.
Alternatively there is a push service which sends changes to the client.
Often using web sockets.
The fully refined implementation seems likely to be massive overkill but can support huge numbers of clients. EG for online betting ( where the client is in a browser ) each player needs to see changing odds and new events appear.
I'm new to Firebase, and I was researching about it to see if it fits our needs. It has everything we need except an offline database. Well, I know that it has the capability to cache changes when the user is offline and then sync them when the user becomes online, but this is not what I'm talking about.
As firebase is costly, we want our free users to be able to use the app only offline and the data should not sync to the cloud no matter the user is online or not, and only use sync for subscribed users.
One solution which we have not yet put much thought into is to use an offline DB like SQLite and:
a) when the user subscribes move the data to firebase
b) if the user cancels the subscription move the data to SQLite
but this solution needs 2 completely different codings the same thing. Extra code for migrating from SQLite to firebase and from firebase to SQLite. Is there a better solution to use the Firestore database and also have a complete offline database functionality?
In my opinion, your solution can work. But there are some situations that you should take into consideration.
a) when the user subscribes move the data to Firebase.
As I understand, it's one or the other. In this case, when a user gets subscribed, you should always consider locking the local SQLite database for writes, until all the data is written on the Firebase servers. When the operation completes, only then you should allow the user to write data in the cloud.
Why is this needed? To always have consistent data.
b) if the user cancels the subscription move the data to SQLite.
If the user cancels the subscription, you might consider using almost the same mechanism as above. But first, you have to clear the local database, as it will contain outdated data, and then copy all data from Firestore right into the SQLite database.
Edit:
You can also take into consideration copying the data from the local cache rather than getting it from the Firebase servers. This will imply no costs.
but this solution needs 2 completely different codings the same thing.
That's correct, but since both, the offline and online databases share the same fields, the operation might not be so complicated as think. Simply attach a real-time listener on a property within the user object, most likely called "subscribed" which can hold the value of true/false, and take actions accordingly when is changed.
I am building a mobile app using AngularJS and PhoneGap. The app allows the user to access a large amount of data-items. These data-items come with the app in form of a number of .json files.
One use-case is that a user can favorite any of those data-items.
Currently, I store the (ids of) the items that have been favorited in localStorage. It works and it's great and very simple.
But now I would like create an online-backend for the app. By this I mean that the (ids of) the items that have been favorited should also be stored on a server somewhere in some form of database.
Now my question is:
How do I best do this?
How do I keep the localStorage data and online-backend data in synch?
In particular, the user might not have an internet connection at the time were he favorites a data-item. Additionally, if the user favorites x data-items in a row, I would need to make x update calls to the server db, which clearly isn't great.
So, how do people do it?
Does Angular have anything build-in for this?
Is there any plugin?
Any other framework?
This very much seems like a common problem that must have a well-known solution?
I think you've almost got the entire solution. All you need to do is periodically (on app start load the data from the service if available, otherwise use current local storage, then maybe with a timer and on app close update the data if connected) send the JSON out to a service (I generally prefer PHP, but Python, Java, Ruby, Perl, whatever floats your boat) that puts it in a database. If you're concerned with merging synchronization changes you'll need to use timestamps in the data in local storage and in the database to make the right call on what should be inserted vs what should be updated.
I don't think there's a one size fits all solution to the problem, though I imagine someone may have crafted a library that handles the different potential scenarios the configuration may be as complicated as just writing the logic yourself.
I've been developing a website using JSF & PrimeFaces. At the time of development, I noticed that there're 2 bottlenecks for any web applications. Correct me if I'm wrong.
The bottlenecks are:
I've used Hibernate framework for persistence layer. Now if a change
occurs in database, then there's no way to reflect that in scoped
beans. Hibernate has dynamic-update attribute which helps to update
only the affected records of the table [at the time of persisting]. But I've not found similar
kind of mechanism by which I can always get updated DAO. Here,
developer has to take responsibility of updating them by using
session.refresh() method, which just reloads the entire object
tree from database table. So,for each small database changes, I
think the caching functionality of DAO [in Hibernate] is missed
since every time they're evicted from session cache. In a word, database updates doesn't trigger DAO updates.
After updating DAO, if I want to reflect the changes in view level, then I had to take help of Primeface sockets [PrimePush] since refreshing the pages every time isn't a good implementation & primeface socket allows updating of specific ids. So, that means for each DAO fields, I've to take help of many different Primeface sockets, each one having unique channel. Also sending messages to those different sockets has to be done by developer in bean codes.
So, the question is how these can be handled in a efficient way? Is there any other technologies/framework which handles these issues so that developer doesn't have to worry about?
Ideally you shoule be doing like :
Hibernate Persistence Layer (have DAO performing CRUD operations)
Managed Beans which access your DAO
View (Primefaces) using BackBean updating the View.
You don't need PrimePush or something. It should be refreshed by actions in your Views
These days I am trying to design architecture of a new MMORPG mobile game for my company. This game is similar to Mafia Wars, iMobsters, or RISK. Basic idea is to prepare an army to battle your opponents (online users).
Although I have previously worked on multiple mobile apps but this is something new to me. After a lot of struggle, I have come up with an architecture which is illustrated with the help of a high-level flow diagram:
We have decided to go with client-server model. There will be a centralized database on server. Each client will have its own local database which will remain in sync with server. This database acts as a cache for storing things that do not change frequently e.g. maps, products, inventory etc.
With this model in place, I am not sure how to tackle following issues:
What would be the best way of synchronizing server and client databases?
Should an event get saved to local DB before updating it to server? What if app terminates for some reason before saving changes to centralized DB?
Will simple HTTP requests serve the purpose of synchronization?
How to know which users are currently logged in? (One way could be to have client keep on sending a request to server after every x minutes to notify that it is active. Otherwise consider a client inactive).
Are client side validations enough? If not, how to revert an action if server does not validate something?
I am not sure if this is an efficient solution and how it will scale. I would really appreciate if people who have already worked on such apps can share their experiences which might help me to come up with something better. Thanks in advance.
Additional Info:
Client-side is implemented in C++ game engine called marmalade. This is a cross platform game engine which means you can run your app on all major mobile OS. We certainly can achieve threading and which is also illustrated in my flow diagram. I am planning to use MySQL for server and SQLite for client.
This is not a turn based game so there is not much interaction with other players. Server will provide a list of online players and you can battle them by clicking battle button and after some animation, result will be announced.
For database synchronization I have two solutions in mind:
Store timestamp for each record. Also keep track of when local DB
was last updated. When synchronizing, only select those rows that
have a greater timestamp and send to local DB. Keep a isDeleted flag
for deleted rows so every deletion simply behaves as an update. But
I have serious doubts about performance as for every sync request we
would have to scan the complete DB and look for updated rows.
Another technique might be to keep a log of each insertion or update
that takes place against a user. When the client app asks for sync,
go to this table and find out which rows of which table have been
updated or inserted. Once these rows are successfully transferred to
client remove this log. But then I think of what happens if a user
uses another device. According to logs table all updates have been
transferred for that user but actually that was done on another
device. So we might have to keep track of device also. Implementing
this technique is more time consuming but not sure if it out
performs the first one.
I've actually worked on some of the titles you mentioned.
I do not recommend using mysql, it doesn't scale up correctly, even if you shard. If you do you are loosing any benefits you might have in using a relational database.
You are probably better off using a no-sql database. Its is faster to develop, easy to scale and it is simple to change the document structure which is a given for a game.
If your game data is simple you might want to try couchDB, if you need advanced querying you are probably better of with MongoDB.
Take care of security at the start. They will try to hack the game for sure and if you have a number of clients released it is hard to make security changes backward compatible. SSL won't do much as the end user is the problem not an eavesdropper. Signing or encrypting your data will make it harder for a user to add items and gold to their accounts.
You should also define your architecture to support multiple clients without having a bunch of ifs and case statements. Read the client version and dispatch that client to the appropriate codebase.
Have a maintenance mode with flags for upgrading, maintenance, etc. It will cut you some slack if you need to re-shard your DB or any other change that might require downtime.
Client side validations are not enough, specially if using in app purchases. I agree with the above post. Server should control game logic.
As for DB sync, its best to memcache read only data. Typical examples are buyable items, maps, news, etc. User data is harder as you might not be able to afford loosing any modified data. The easiest setup is to cache user data for a couple of hours and write directly to the DB every time. If you are using no-sql it will probably withstand a high load without the need of using a persistence queue.
I see two potential problem hidden in the fact that you store all the state on the client, and then update the state on the server using a background thread.
How can the server validate the data being posted? If someone hacked your application, they could modify the code so whenever they swing their sword (or whatever they do in your game), it is always a hit. Doing that in a single player game is not that big a deal, but doing that in an MMORPG can ruin the experience for everyone else. So the server should validate every update of data - or even better, the server should be in charge of every business rule. So when you swing your sword against an opponent, that should be a server call, and the server returns whether or not it is a hit, and how many hit points the opponent lost.
What about interaction with other players (since you say it is an MMORP, there will be interaction with other players)? Since you say that you update the server, and get updates in a background thread, interaction will be sluggish. When you communicate with another character you have first wait for you background thread to sync data, but you also have to wait on the background thread of the other player to sync data.
Looks nice. But what is the client-side made of ? Web ? Can you use threading to synchronize both DB ? I should make the game in that way that it interacts immediately with the local DB, and let some background mechanism do the sync (something like a snapshot). This leads me to think about mysql replication. I think it is worth to be tried, but I never did. It also brings you answers to other questions. But what about the charge (how many customers are connected together) ?
http://dev.mysql.com/doc/refman/5.0/en/replication.html
Make your client issue commands to the server ("hit player"), and server send (relevant) events to client ("player was killed"). I wouldn't advice going with data synchronization. Server should be responsible for all important game decisions.