Working on a GAE project and one requirement we have is that we want to in a timely manner be able to determine if a user has left the application. Currently we have this working, but is unreliable so I am researching alternatives.
The way we do this now is we have a function setup to run in JS on an interval that sends a heartbeat signal to the GAE app using an AJAX call. This works relatively well, but is generating a lot of traffic and CPU usage. If we don't hear a heartbeat from a client for several minutes, we determine they have left the application. We also have the unload function wired up to send a part message, again through an AJAX call. This works less then well, but most of the time not at all.
We are also making use of the Channels API. One thing I have noticed is that our app when using an open channel, the client seems to also be sending a heartbeat signal in the form of a call to http://talkgadget.google.com/talkgadget/dch/bind. I believe this is happening from the iFrame and/or JS that gets loaded when opening channel in the client.
My question is, can my app on the server side some how hook in to these calls to http://talkgadget.google.com/talkgadget/dch/bind and use this as the heartbeat signal? Is there a better way to detect if a client is still connected even if they aren't actively doing anything in the client?
Google have added this feature:
See https://developers.google.com/appengine/docs/java/channel/overview
Tracking Client Connections and Disconnections
Applications may register to be notified when a client connects to or
disconnects from a channel.
You can enable this inbound service in appengine-web.xml:
Currently the channel API bills you up-front for all the CPU time the channel will consume for two hours, so it's probably cheaper to send messages to a dead channel than to send a bunch of heartbeat messages to the server.
https://groups.google.com/d/msg/google-appengine/sfPTgfbLR0M/yctHe4uU824J
What I would try is attach a "please acknowledge" parameter to every Nth message (staggered to avoid every client acknowledging a single message). If 2 of these are ignored mute the channel until you hear from that client.
You can't currently use the Channel API to determine if a user is still online or not. Your best option for now depends on how important it is to know as soon as a user goes offline.
If you simply want to know they're offline so you can stop sending messages, or it's otherwise not vital you know immediately, you can simply piggyback pings on regular interactions. Whenever you send the client an update and you haven't heard anything from them in a while, tag the message with a 'ping request', and have the client send an HTTP ping whenever it gets such a tagged message. This way, you'll know they're gone shortly after you send them a message. You're also not imposing a lot of extra overhead, as they only need to send explicit pings if you're not hearing anything else from them.
If you expect long periods of inactivity and it's important to know promptly when they go offline, you'll have to have them send pings on a schedule, as you suggested. You can still use the trick of piggybacking pings on other requests to minimize them, and you should set the interval between pings as long as you can manage, to reduce load.
I do not have a good solution to your core problem of "hooking" the client to server. But I do have an interesting thought on your current problem of "traffic and CPU usage" for periodic pings.
I assume you have a predefined heart-beat interval time, say 1 min. So, if there are 120 clients, your server would process heart beats at an average rate of 2 per second. Not good if half of them are "idle clients".
Lets assume a client is idle for 15 minutes already. Does this client browser still need to send heart-beats at the constant pre-defined interval of 1 min?? Why not make it variable?
My proposal is simple: Vary the heart-beats depending on activity levels of client.
When the client is "active", heart-beats work at 1 per minute. When the client is "inactive" for more than 5 minutes, heart-beat rate slows down to 50% (one after every 2 minutes). Another 10 minutes, and heart-beat rate goes down another 50% (1 after every 4 minutes)... At some threshold point, consider the client as "unhooked".
In this method, "idle clients" would not be troubling the server with frequent heartbeats, allowing your app server to focus on "active clients".
Its a lot of javascript to do, but probably worth if you are having trouble with traffic and CPU usage :-)
Related
I'm building a network application in RedHat/C with a protocol called SMPP that is being used in telecom to send SMS.
I'm at a point where I send messages (~70 SMS/second) to the server and I have to wait to a few seconds and for a successful response and delete the messages, but if the message timed-out then I have to resend the message to the server.
The question is how to design something to retry the expired messages?
There is an id in the SMPP spec called sequence_number - this should be monotonically incrementing for every request you make and the response coming back from the server will have the sequence_number of the request it is responding to.
If you wait for a bit (maybe 10 seconds, maybe longer) and you don't get your response back you can re-send the request with the same sequence_number and the server should spot it as a duplicate if it did receive it first time; if it did not receive it first time then it will treat it as a new request.
The server may also make requests to your client; e.g. here is a delivery receipt or here is a mobile-originate message - it will also have it's own sequence_number counter and you should acknowledge it's requests with responses having the same sequence number. You should track the sequence numbers you have seen so you can tell if you hit a duplicate request.
This property is called http://en.wikipedia.org/wiki/Idempotence and is something you should become familiar with if you are implementing telecoms protocols.
In order to get your 70 msgs/sec you will likely need to build on top of Idempotence using a sliding window http://en.wikipedia.org/wiki/Flow_control_(data)#Sliding_Window so you can have a maximum of N (maybe 10) requests outstanding you are still waiting for the response acknowledgements to - unless you are very close to the SMPP server with very low latency.
Doing SMPP right is not trivial I would recommend you read SMPP v3.4 spec front to back before you get too far into an implementation.
It is not very clear what you are asking for so the answer will be also probably not very precise.
I would suggest to see how this is implemented in some existing solutions. I have worked a bit with kannel and mbuni (this is rather for MMS) and I suggest take a look at kannel especially.
Kannel is basically open source SMS gateway and have working SMPP support.
Take also a look at this stackoverflow thread which may also help to understand some ideas.
I have an application where users can collaborate on photo albums. I currently use polling on the client to check for new content every 30 seconds. There can be any number of people uploading and viewing an album at any given time.
On the server side, I cache the data to return (so the query for new content is cheap). I assume that polling every 30 seconds from multiple clients will cause more instances to stay active (and thus increase costs).
Would it be overkill to use the channel api for the above use case instead of polling?
Does the channel api keep instances alive too?
Are there any use cases where polling is preferable instead of using the channel api?
I'm using channels but I'm finding they're not great. If a channel times out from a network disconnect, it somehow screws up the history on my browser. I've filed a bug a bit over a week ago, but it hasn't been acknowledged. There's another bug filed over a month ago that hasn't been acknowledged either - so don't expect quick support on channel issues.
Channels are nice to have - you can notify users in less than a second if status of some sort changes, but they're not reliable. Sometimes the disconnect event doesn't occur, but the channel just stops working. My current system uses channels, but also polls every 5-10 seconds. Because of the unreliability, I wouldn't use channels as a replacement for polling, just a way to give faster response.
Even then you'll have to work out whether it'll save you money. If you're expecting users to leave your app open for 15 minutes without hitting the server, then maybe you'll save some instance time. However, if your users are hitting the server anyways, your instances probably wouldn't get time to shut down. And keeping your instances up actually helps reduce cold starts a bit too.
I use PushSharp to send notifications for a few Apps.
PushSharp is great it really simplifies the work with push services, and I wonder what is the right way to work with it?
I haven't found examples/ explanations about that.
Now, when I have a message to send , I ...
create a PushSharp object
do a PushService.QueueNotification() for all devices
do a PushService.StopAllServices to send all queued messages
exits the method (and kill the PushService object).
Should I work this way, or keep this PushService object alive and call its methods when needed?
How should I use a PushService object to get the unregistered device ids? with a dedicated instance?
Any suggestion would be appreciated.
This is a question which frequently comes up.
The answer isn't necessarily one way or the other, but it depends on your situation. In most cases it would be absolutely fine to just create a PushBroker instance whenever you need it, since most platforms use HTTP based protocols for sending notifications. In the case of Apple, they state in their documentation that you should keep your connection to APNS open in order to minimize overhead of opening and closing secure connections.
However, in practice I think this means that they don't want you connecting and disconnecting VERY frequently (eg: they don't want you creating a new connection for every message you send). In reality, if you're sending batches of notifications every so often (let's say every 15 minutes or every hour) they probably won't have a problem with you opening a new connection for each batch and then closing it when done.
I've never heard of anyone being blocked from Apple's APNS servers for doing this. In fact in the very early days of working with push notifications, I had a bug that caused a new apns connection to be created for each notification. I sent thousands of notifications a day like this and never heard anything about it from Apple (eventually I identified it as a bug and fixed it of course).
As for collecting feedback, by default the ApplePushService will poll the feedback servers after 10 seconds of starting, and then every 10 minutes thereafter. If you want to disable this from happening you can simply set the ApplePushChannelSettings.FeedbackIntervalMinutes to <= 0. You can then use the FeedbackService class to poll for feedback whenever you need to, manually.
Google app engine seems to have recently made a huge decrease in free quotas for channel creation from 8640 to 100 per day. I would appreciate some suggestions for optimizing channel creation, for a hobby project where I am unwilling to use the paid plans.
It is specifically mentioned in the docs that there can be only one client per channel ID. It would help if there were a way around this, even if it were only for multiple clients on one computer (such as multiple tabs)
It occurred to me I might be able to simulate channel functionality by repeatedly sending XHR requests to the server to check for new messages, therefore bypassing limits. However, I fear this method might be too slow. Are there any existing libraries that work on this principle?
One Client per Channel
There's not an easy way around the one client per channel ID limitation, unfortunately. We actually allow two, but this is to handle the case where a user refreshes his page, not for actual fan-out.
That said, you could certainly implement your own workaround for this. One trick I've seen is to use cookies to communicate between browser tabs. Then you can elect one tab the "owner" of the channel and fan out data via cookies. See this question for info on how to implement the inter-tab communication: Javascript communication between browser tabs/windows
Polling vs. Channel
You could poll instead of using the Channel API if you're willing to accept some performance trade-offs. Channel API deliver speed is on the order of 100-200ms; if you could accept 500ms average then you could poll every second. Depending on the type of data you're sending, and how much you can fit in memcache, this might be a workable solution. My guess is your biggest problem is going to be instance-hours.
For example, if you have, say, 100 clients you'll be looking at 100qps. You should experiment and see if you can serve 100 requests in a second for the data you need to serve without spinning up a second instance. If not, keep increasing your latency (ie., decreasing your polling frequency) until you get to 1 instance able to serve your requests.
Hope that helps.
I'm trying to enable a streamed response using Silverlight 4 and polling duplex, but I'm getting strange behaviour when the rate at which updates are sent to the client is greater than the maxOutputDelay, which results in no updates being sent.
For example, with a maxOutputDelay of 7 seconds, and 1 update sent every 10 seconds, everything works fine. But if I have a maxOutputDelay of 1 second, and an update sent every 500 milliseconds, the updates just sit on the server side and don't get sent to the client.
It's my understanding that setting transferMode="StreamedResponse" should send the updates immediately to the client, but this doesn't seem to be working.
Here's the binding in my Web.config for the web service:
This config is based on the information from this article: http://blogs.msdn.com/b/silverlightws/archive/2010/06/25/http-duplex-improvements-silverlight-4.aspx
Thanks.
If you are not totally focussed on using Duplex Channels (which are a pain to configure in anything but a single host scenario) it might be worth checking out alternative solutions for implementing Server Callbacks - even if that means that you have to maintain two different types of connection to your backend.
Duplex Channel Alternatives:
PokeIn
Kaazing WebSocket Gateway
I think this article answers the question:
http://blogs.msdn.com/b/silverlightws/archive/2010/07/16/pollingduplex-multiple-mode-timeouts-demystified.aspx
The maxOutputDelay is more like an intra-message timer. So if your message rate exceeds this delay you will never trigger a flush until the buffer fills. It gets reset on each new message added to the queue. So I guess we have to tune the queue size as well as this timer to achieve a maximum actual latency.
I'm not sure why the streamed response still buffers but I see it too. Does anyone know how to tune that buffer size?
[Edited]
Ok, this article says that we cannot control the buffering int he streamed response (it is 16k in self-hosted and 32k in IIS). So, given that, it seems like small messages coming in at a rate greater than your maxoutputdelay are a pathological case. Maybe I have to pad them with data...
http://blogs.msdn.com/b/silverlightws/archive/2010/06/25/http-duplex-improvements-silverlight-4.aspx