I have an application where users can collaborate on photo albums. I currently use polling on the client to check for new content every 30 seconds. There can be any number of people uploading and viewing an album at any given time.
On the server side, I cache the data to return (so the query for new content is cheap). I assume that polling every 30 seconds from multiple clients will cause more instances to stay active (and thus increase costs).
Would it be overkill to use the channel api for the above use case instead of polling?
Does the channel api keep instances alive too?
Are there any use cases where polling is preferable instead of using the channel api?
I'm using channels but I'm finding they're not great. If a channel times out from a network disconnect, it somehow screws up the history on my browser. I've filed a bug a bit over a week ago, but it hasn't been acknowledged. There's another bug filed over a month ago that hasn't been acknowledged either - so don't expect quick support on channel issues.
Channels are nice to have - you can notify users in less than a second if status of some sort changes, but they're not reliable. Sometimes the disconnect event doesn't occur, but the channel just stops working. My current system uses channels, but also polls every 5-10 seconds. Because of the unreliability, I wouldn't use channels as a replacement for polling, just a way to give faster response.
Even then you'll have to work out whether it'll save you money. If you're expecting users to leave your app open for 15 minutes without hitting the server, then maybe you'll save some instance time. However, if your users are hitting the server anyways, your instances probably wouldn't get time to shut down. And keeping your instances up actually helps reduce cold starts a bit too.
Related
I am creating an Instant Messaging application for our department. The features of this application are:
The messages will be stored in a database
The messages may be sent to one, multiple, or all users/locations
The logged in user will be able to see a history of the messages they are included in.
My question: is it appropriate to constantly query the database from each client - there should be less than 20 clients running - say every 15 - 30 secs or so? I have seen examples of a server/client messaging app using tcipclient but am not familiar with that subject. So I thought querying the database might be the approach I could go with. What are the ramifications of performing these queries so often? I'm also looking at sqldependencies??? Should I really go back to and try and learn tcip technology?
Thanks
If you know that you will always have of the order of tens of clients but not of the order of thousands of clients, then polling will work just fine, and you do not have to poll every 15 seconds, (it would be unusable if you did so,) you can poll every 100 or 200 milliseconds, so chatting will appear instantaneous.
Just make sure that each polling operation is as simple as possible. The simplest operation you can do is this:
SELECT * FROM chat_log WHERE chat_log.id > ? where id is your IDENTITY primary key, and ? is the last id that your client has seen so far from the server. Therefore, if there are no new chat messages, no rows are retrieved. With every row retrieved by a client, update the largest id that the client has seen so far, and you are good to go.
I have done it and it works like a charm.
From a technical point of view polling is a very ignoble technique, but in many situations it can be a practical compromise which may yield good enough results with very little development. (The alternative would be to create a proper chat server which sends push notifications to the clients, good luck with that.)
If its less that 20 clients (20 select queries every 20 seconds + some writes), SQL Server will have no issues to process these messages.
Selection of tools and technology depends on your actual requirements. (size of messages, allow file transfers, delete/edit messages...)
I can suggest few options to improve performance,
Reading Messages - You can use Caching (e.g. Azure Redis Cache) for recent messages (last 30days). You can come up with background cache update strategy to make sure it's continuously updated with new messages. Read messages will call the cache first, it will hit the database only if there is a cache miss.
Also you can create a local message cache (client side) which will dramatically improve performance for end user. You can create a SQLite for this (like Skype does. Win + R -> %appdata%\skype -> folder -> main.db)
Or else you can simply have an Archive table in your db where a scheduled (every 24 hours) background process archives messages older than 14/30 days. So you will have recent messages
Writing - Writing messages will be chatty, rather than directly updating the database you can use a Message queue (Azure Message Queue, Rabbit MQ.. etc). Then you can have another process to write messages to the database.
Each technology selection will have it's own cost, pros and cons and learning time. Therefore start simple and leave room to scale later.
I use PushSharp to send notifications for a few Apps.
PushSharp is great it really simplifies the work with push services, and I wonder what is the right way to work with it?
I haven't found examples/ explanations about that.
Now, when I have a message to send , I ...
create a PushSharp object
do a PushService.QueueNotification() for all devices
do a PushService.StopAllServices to send all queued messages
exits the method (and kill the PushService object).
Should I work this way, or keep this PushService object alive and call its methods when needed?
How should I use a PushService object to get the unregistered device ids? with a dedicated instance?
Any suggestion would be appreciated.
This is a question which frequently comes up.
The answer isn't necessarily one way or the other, but it depends on your situation. In most cases it would be absolutely fine to just create a PushBroker instance whenever you need it, since most platforms use HTTP based protocols for sending notifications. In the case of Apple, they state in their documentation that you should keep your connection to APNS open in order to minimize overhead of opening and closing secure connections.
However, in practice I think this means that they don't want you connecting and disconnecting VERY frequently (eg: they don't want you creating a new connection for every message you send). In reality, if you're sending batches of notifications every so often (let's say every 15 minutes or every hour) they probably won't have a problem with you opening a new connection for each batch and then closing it when done.
I've never heard of anyone being blocked from Apple's APNS servers for doing this. In fact in the very early days of working with push notifications, I had a bug that caused a new apns connection to be created for each notification. I sent thousands of notifications a day like this and never heard anything about it from Apple (eventually I identified it as a bug and fixed it of course).
As for collecting feedback, by default the ApplePushService will poll the feedback servers after 10 seconds of starting, and then every 10 minutes thereafter. If you want to disable this from happening you can simply set the ApplePushChannelSettings.FeedbackIntervalMinutes to <= 0. You can then use the FeedbackService class to poll for feedback whenever you need to, manually.
I'm building a multiuser realtime application with Google App Engine (Python) that would look like the Facebook livestream plugin: https://developers.facebook.com/docs/reference/plugins/live-stream/
Which means: 1 to 1 000 000 users on the same webpage can perform actions that are instantly notified to everyone else. It's like a group chat but with a lot of people...
My questions:
- Is App Engine able to scale to that kind of number?
- If yes, how would you design it?
- If no, what would be your suggestions?
Right now, this is my design:
- I'm using the App Engine Channel API
- I store every user connected in the memcache
- Everytime an action is performed, a notification task is added to a taskqueue
- The task consist in retrieving all users from memcache and send them a notification.
I know my bottleneck is in the task. Everybody is notified through the same task/ request. Right now, for 30 users connected, it lasts about 1 sec so for 100 000 users, you can imagine how long it could take.
How would you correct this?
Thanks a lot
How many updates per user do you expect per second? If each user updates just once every hour, you'll be sending 10^12 messages per hour -- every sent message results in 1,000,000 more sends. This is 277 million messages per second. Put another way, if every user sends a message an hour, that works out to 277 incoming messages per second, or 277 million outgoing messages.
So I think your basic design is flawed. But the underlying question: "how do I broadcast the same message to lots of users" is still valid, and I'll address it.
As you have discovered, the Channel API isn't great at broadcast because each call takes about 50ms. You could work around this with multiple tasks executing in parallel.
For cases like this -- lots of clients who need the exact same stateless data, I would encourage you to use polling, rather than the Channel API, since every client is going to receive the exact same information -- no need to send individualized messages to each client. Decide on an acceptable average latency (eg. 1 second) and poll at twice that rate (eg. 2 seconds). Write a very lightweight, memcache-backed servlet to just get the most recent block of data and let the clients de-dupe.
Google app engine seems to have recently made a huge decrease in free quotas for channel creation from 8640 to 100 per day. I would appreciate some suggestions for optimizing channel creation, for a hobby project where I am unwilling to use the paid plans.
It is specifically mentioned in the docs that there can be only one client per channel ID. It would help if there were a way around this, even if it were only for multiple clients on one computer (such as multiple tabs)
It occurred to me I might be able to simulate channel functionality by repeatedly sending XHR requests to the server to check for new messages, therefore bypassing limits. However, I fear this method might be too slow. Are there any existing libraries that work on this principle?
One Client per Channel
There's not an easy way around the one client per channel ID limitation, unfortunately. We actually allow two, but this is to handle the case where a user refreshes his page, not for actual fan-out.
That said, you could certainly implement your own workaround for this. One trick I've seen is to use cookies to communicate between browser tabs. Then you can elect one tab the "owner" of the channel and fan out data via cookies. See this question for info on how to implement the inter-tab communication: Javascript communication between browser tabs/windows
Polling vs. Channel
You could poll instead of using the Channel API if you're willing to accept some performance trade-offs. Channel API deliver speed is on the order of 100-200ms; if you could accept 500ms average then you could poll every second. Depending on the type of data you're sending, and how much you can fit in memcache, this might be a workable solution. My guess is your biggest problem is going to be instance-hours.
For example, if you have, say, 100 clients you'll be looking at 100qps. You should experiment and see if you can serve 100 requests in a second for the data you need to serve without spinning up a second instance. If not, keep increasing your latency (ie., decreasing your polling frequency) until you get to 1 instance able to serve your requests.
Hope that helps.
Working on a GAE project and one requirement we have is that we want to in a timely manner be able to determine if a user has left the application. Currently we have this working, but is unreliable so I am researching alternatives.
The way we do this now is we have a function setup to run in JS on an interval that sends a heartbeat signal to the GAE app using an AJAX call. This works relatively well, but is generating a lot of traffic and CPU usage. If we don't hear a heartbeat from a client for several minutes, we determine they have left the application. We also have the unload function wired up to send a part message, again through an AJAX call. This works less then well, but most of the time not at all.
We are also making use of the Channels API. One thing I have noticed is that our app when using an open channel, the client seems to also be sending a heartbeat signal in the form of a call to http://talkgadget.google.com/talkgadget/dch/bind. I believe this is happening from the iFrame and/or JS that gets loaded when opening channel in the client.
My question is, can my app on the server side some how hook in to these calls to http://talkgadget.google.com/talkgadget/dch/bind and use this as the heartbeat signal? Is there a better way to detect if a client is still connected even if they aren't actively doing anything in the client?
Google have added this feature:
See https://developers.google.com/appengine/docs/java/channel/overview
Tracking Client Connections and Disconnections
Applications may register to be notified when a client connects to or
disconnects from a channel.
You can enable this inbound service in appengine-web.xml:
Currently the channel API bills you up-front for all the CPU time the channel will consume for two hours, so it's probably cheaper to send messages to a dead channel than to send a bunch of heartbeat messages to the server.
https://groups.google.com/d/msg/google-appengine/sfPTgfbLR0M/yctHe4uU824J
What I would try is attach a "please acknowledge" parameter to every Nth message (staggered to avoid every client acknowledging a single message). If 2 of these are ignored mute the channel until you hear from that client.
You can't currently use the Channel API to determine if a user is still online or not. Your best option for now depends on how important it is to know as soon as a user goes offline.
If you simply want to know they're offline so you can stop sending messages, or it's otherwise not vital you know immediately, you can simply piggyback pings on regular interactions. Whenever you send the client an update and you haven't heard anything from them in a while, tag the message with a 'ping request', and have the client send an HTTP ping whenever it gets such a tagged message. This way, you'll know they're gone shortly after you send them a message. You're also not imposing a lot of extra overhead, as they only need to send explicit pings if you're not hearing anything else from them.
If you expect long periods of inactivity and it's important to know promptly when they go offline, you'll have to have them send pings on a schedule, as you suggested. You can still use the trick of piggybacking pings on other requests to minimize them, and you should set the interval between pings as long as you can manage, to reduce load.
I do not have a good solution to your core problem of "hooking" the client to server. But I do have an interesting thought on your current problem of "traffic and CPU usage" for periodic pings.
I assume you have a predefined heart-beat interval time, say 1 min. So, if there are 120 clients, your server would process heart beats at an average rate of 2 per second. Not good if half of them are "idle clients".
Lets assume a client is idle for 15 minutes already. Does this client browser still need to send heart-beats at the constant pre-defined interval of 1 min?? Why not make it variable?
My proposal is simple: Vary the heart-beats depending on activity levels of client.
When the client is "active", heart-beats work at 1 per minute. When the client is "inactive" for more than 5 minutes, heart-beat rate slows down to 50% (one after every 2 minutes). Another 10 minutes, and heart-beat rate goes down another 50% (1 after every 4 minutes)... At some threshold point, consider the client as "unhooked".
In this method, "idle clients" would not be troubling the server with frequent heartbeats, allowing your app server to focus on "active clients".
Its a lot of javascript to do, but probably worth if you are having trouble with traffic and CPU usage :-)