we want to build an application (c#/.Net) for the following Scenario:
internal "alert System". Users should be informed about it-system outage, planned downtime for Services and so on.
only one-way : central Service will push Messages to user
we also Need the possibility to enable/disable a message, for example:
The message "there a Problems with mail System" should be removed from every Computer after the Problem is solved
we want to shedule Messages for planned maintanance
about 1000 windows Clients, we also want to "group" this Clients, so we can control which Client will get a message
First thought was writing small application which will query every X seconds a central database for new and existing Messages.
Maybe somebody has already worked on similar Project?
Is a Client with database query a way to go? Better to use other Technology, like WCF Service?
Thanks for your help
Marc
Sounds like you need an enhanced version of push notifications.
I'd suggest using push for all the messaging, it's delivered faster and I find it more reliable. Simply make the client connect to a message server and maintain the connection open. Whenever a message is supposed to be displayed to the client, have the server push it trough the connection (that's where the name comes from).
To group and manage the clients you could use a database, it's probably the best way to go, but the server needs to handle all the open connections, and databases can only store DATA, not virtual objects representing a connection, so the server software need to manage them in a different way.
My suggestion: Whenever the server receives an incoming client connection, it will accept and query the client computer for a ID number that will also be used to find that client's information in the database.
Then it will create a dictionary using that ID as key, and the connection as the value.
This way at the time of sending a message to a determined group, you can do in two ways:
1) You can load from the database the IDs that belong to that group, and then send the messages to them. You will have to check whether that ID exists in the dictionary's KEYS array, because it is possible that a determined client is not yet connected.
2) You can iterate of the KEYS array of dictionary, check to which group that ID is part of, and if it is the desires group, send it.
If you're dealing with a big number of clients, I suggest you use method 1.
To disable/remove a message from the client's computer, simply have the server send a special Command message that the client software interprets as "remove that message". To make this possible every non-command message must have unique IDs, so that command messages can tell the client software which message that command applies to.
Your project sounds very interesting.
I would be glad to help you by writing a library you could use, or just help you figure it out on your own if you prefer. (Free of charge, just for the experience).
Related
I would like to implement a chat system as part of a game I am developing on App Engine. To implement this, I would like to use WebSockets, and have clients connect to each other though a hub, in this case an instance of GCE. Assuming this game needed to scale to multiple instances on GCE, how would this work? If I had a client 1, and the load balancer directed that request of client 1 to instance A, and another client (2) came in and was directed to instance B, but those clients wanted to chat with each other, they would each be connected to different hubs, and would be unable to reach each other. How would this be set up to work with scale? Would I implement it using queues, where each instance listens on that queue, and if so, how would I do that?
Google Play Game Services offers exactly the functionality that you want but in regard to Android and ios clients. So this option may not be compatible with your game tech design.
In general you're reasoning correctly. Messages from client who want to talk to each other will most of the time hit different server instances. What you want to do is to make instances handle the communication between users. Pub/sub (publish-subscribe pattern) is very suitable pattern in this scenario. Roughly:
whenever there's a message directed to client X a message is published on the channel X,
whenever client X creates a session, instance handling it subscribes to channel X.
You can use one of many existing solutions for starters. It's very easy to set this up using redis. If you need something more low-level and more flexible check out zeromq.
You can expect single instance of either solution to be able to handle thousands of QPS.
Unfortunately I don't have any experience with scaling neither of these solutions so can't offer you any practical advice as to the limits of their scalability.
PS. There are also other topics you may want to explore such as: message persistence and failure recovery I didn't address here at all.
I didn't try to implement this yet but I'll probably have to soon, I think it should be fairly simple to handle it yourself.
You have: server 1 with list of clients and you have server 2 with another list of clients,
so if client wants to send data to another client which might be on server 2, you have to:
Lookup if the receiver is on current server - if it is, you just send it (standard)
Otherwise you send the same data to all other servers you have, so they would check their lists for particular client (or clients) and send data to them.
I use PushSharp to send notifications for a few Apps.
PushSharp is great it really simplifies the work with push services, and I wonder what is the right way to work with it?
I haven't found examples/ explanations about that.
Now, when I have a message to send , I ...
create a PushSharp object
do a PushService.QueueNotification() for all devices
do a PushService.StopAllServices to send all queued messages
exits the method (and kill the PushService object).
Should I work this way, or keep this PushService object alive and call its methods when needed?
How should I use a PushService object to get the unregistered device ids? with a dedicated instance?
Any suggestion would be appreciated.
This is a question which frequently comes up.
The answer isn't necessarily one way or the other, but it depends on your situation. In most cases it would be absolutely fine to just create a PushBroker instance whenever you need it, since most platforms use HTTP based protocols for sending notifications. In the case of Apple, they state in their documentation that you should keep your connection to APNS open in order to minimize overhead of opening and closing secure connections.
However, in practice I think this means that they don't want you connecting and disconnecting VERY frequently (eg: they don't want you creating a new connection for every message you send). In reality, if you're sending batches of notifications every so often (let's say every 15 minutes or every hour) they probably won't have a problem with you opening a new connection for each batch and then closing it when done.
I've never heard of anyone being blocked from Apple's APNS servers for doing this. In fact in the very early days of working with push notifications, I had a bug that caused a new apns connection to be created for each notification. I sent thousands of notifications a day like this and never heard anything about it from Apple (eventually I identified it as a bug and fixed it of course).
As for collecting feedback, by default the ApplePushService will poll the feedback servers after 10 seconds of starting, and then every 10 minutes thereafter. If you want to disable this from happening you can simply set the ApplePushChannelSettings.FeedbackIntervalMinutes to <= 0. You can then use the FeedbackService class to poll for feedback whenever you need to, manually.
I'm trying to figure out if PollingDuplex is the right way to go for my problem.
Here is my scenario:
1. 3rd party application sends a UDP packet with a client's IP address to a server app.
2. The server app needs to the notify the specified client and send along some data.
The client is a Silverlight application.
I've been looking at some guides and sample code (http://petermcg.wordpress.com/2008/09/03/silverlight-polling-duplex-part-1-architecture/) but I don't understand how clients are identified on the server using PollingDuplex. I understand that the clients register with the server and continually poll for messages. How would I make sure that only the right clients get the message designated for that client? In other words, the messages on the server should not be broadcasted to all polling clients but only sent to one specific client.
Any help is much appreciated.
Whether you're using Net.TCP or HttpDuplexBinding, clients can be identified using OperationContext.Current.Channel.SessionId. And more specifically, you can grab the actual channel that WCF uses to talk to them using OperationContext.Current.GetCallbackChannel<IMyCustomServiceInterface>(). You can store those in memory, perhaps associated with some other identifier passed up from the client, and when you need to communicate with the client in question (e.g., to pass them the data from the UDP packet), you call the appropriate method on that specific stored channel; and the client will get notified.
I should note that while I don't particularly recommend HttpDuplexBinding, apart from its quirks and stability and performance issues, it should work for what you're doing, and in exactly the same way as Net.TCP. Although the clients technically do "poll" the server, that's hidden from you. All you know on the server is that you're calling a method on a particular channel. The underlying binding code takes care of making sure that the right client gets notified.
Polling duplex is actually an entirely client side implementation that exists only for Silverlight (there's no regular .NET framework version of it, except a project on Codeplex Microsoft's own internal consulting services developed for a high profile client of theirs). There's nothing at all special about it on the server side.
It's not really meant to be used in production by Microsoft's own admission (we have a Microsoft contact at our company who admitted this to us candidly). It's not very robust or well implemented and can/will DoS your server under any kind of volume:
http://forums.silverlight.net/p/89970/239380.aspx
You're better off rolling your own client side polling mechanism - or (better and more scalable) using TCP with session in Silverlight 4, which provides true duplex support (because the connection is not stateless and thus supports true push notifications):
http://www.silverlightshow.net/items/WCF-NET.TCP-Protocol-in-Silverlight-4.aspx.
What's the best strategy to keep all the clients of a database server synchronized?
The scenario involves a database server and a dynamic number of clients that connect to it, viewing and modifying the data.
I need real-time synchronization of the data across all the clients - if data is added, deleted, or updated, I want all the clients to see the changes in real-time without putting too much strain on the database engine by continuous polling for changes in tables with a couple of million rows.
Now I am using a Firebird database server, but I'm willing to adopt the best technology for the job, so I want to know if there is any kind of already existing framework for this kind of scenario, what database engine does it use and what does it involve?
Firebird has a feature called EVENT that you may be able to use to notify clients of changes to the database. The idea is that when data in a table is changed, a trigger posts an event. Firebird takes care of notifying all clients who have registered an interest in the event by name. Once notified, each client is responsible for refreshing its own data by querying the database.
The client can't get info from the event about the new or old values. This is by design, because there's no way to resolve this with transaction isolation. Nor can your client register for events using wildcards. So you have to design your server-to-client notification pretty broadly, and let the client update to see what exactly changed.
See http://www.firebirdsql.org/doc/whitepapers/events_paper.pdf
You don't mention what client platform or language you're using, so I can't advise on the specific API you would use. I suggest you google for instance "firebird event java" or "firebird event php" or similar, based on the language you're using.
Since you say in a comment that you're using WPF, here's a link to a code sample of some .NET application code registering for notification of an event:
http://www.firebirdsql.org/index.php?op=devel&sub=netprovider&id=examples#3
Re your comment: Yes, the Firebird event mechanism is limited in its ability to carry information. This is necessary because any information it might carry could be canceled or rolled back. For instance if a trigger posts an event but then the operation that spawned the trigger violates a constraint, canceling the operation but not the event. So events can only be a kind of "hint" that something of interest may have happened. The other clients need to refresh their data at that time, but they aren't told what to look for. This is at least better than polling.
So you're basically describing a publish/subscribe mechanism -- a message queue. I'm not sure I'd use an RDBMS to implement a message queue. It can be done, but you're basically reinventing the wheel.
Here are a few message queue products that are well-regarded:
Microsoft MSMQ (seems to be part of Windows Professional and Server editions)
RabbitMQ (free open-source)
Apache ActiveMQ (free open-source)
IBM WebSphere MQ (probably overkill in your case)
This means that when one client modifies data in a way that others may need to know about, that client also has to post a message to the message queue. When consumer clients see the message they're interested in, they know to refresh their copy of some data.
SQL Server 2005 and higher support notification based data source caching expiry.
I'm a beginning web developer sitting on an ambitious web application project.
So after having done some research, I found out about SQL Service Broker. It seems like something I could use, but I'm not sure. Since learning it requires someone to put in lots of time, I wanted to be sure that it would fit my needs.
I need to implement a system where website users can submit text to the website. This stream of messages has to be redundant and dealt with in a FIFO way, with on the other end of the stream another group of users dealing with the messages.
Now, a message that is being read by one of this last group of users, should be locked so that no-one else can read it at the same time. The user can then decide to handle the message or not. Only if he decides to deal with the message can it be deleted from the queue. If he decides that he doesn't want to deal with the message, the message should be put back in the queue (at the end of the queue, or at least with the highest priority), so that another user can read it and decide.
Is this something I would be able to implement with SQL Service Broker? Am I on the wrong track?
Thank you!
IMO, the best use of Service Broker is for connecting to independent Application in a loosely coupled way. What I mean by that is that systems tied in this way can communicate through a set of mutually agreed message types. This in contrast to one application manipulating directly the other's database, for example.
From what you've said, I would implement it as a simple table, for example: Create a message table with an identity PK, an Allocation flag and your custom columns. Whenever an operator wants to fetch the last message, get the lowest PK value for which Allocation = 'N' and update Allocation to 'Y'. This in a single transaction.
When/if the operation decides to return the message to queue, just set its AllocationFlag to 'N' and its back.
This is just an example. The database in this case is providing you consistency, heavy load performance, etc.
Behind the screens all data you submit to SSB is stored and manipulated as tables, so there is no reason for it to be necessarily faster than a database solution .