I'm trying to enable a streamed response using Silverlight 4 and polling duplex, but I'm getting strange behaviour when the rate at which updates are sent to the client is greater than the maxOutputDelay, which results in no updates being sent.
For example, with a maxOutputDelay of 7 seconds, and 1 update sent every 10 seconds, everything works fine. But if I have a maxOutputDelay of 1 second, and an update sent every 500 milliseconds, the updates just sit on the server side and don't get sent to the client.
It's my understanding that setting transferMode="StreamedResponse" should send the updates immediately to the client, but this doesn't seem to be working.
Here's the binding in my Web.config for the web service:
This config is based on the information from this article: http://blogs.msdn.com/b/silverlightws/archive/2010/06/25/http-duplex-improvements-silverlight-4.aspx
Thanks.
If you are not totally focussed on using Duplex Channels (which are a pain to configure in anything but a single host scenario) it might be worth checking out alternative solutions for implementing Server Callbacks - even if that means that you have to maintain two different types of connection to your backend.
Duplex Channel Alternatives:
PokeIn
Kaazing WebSocket Gateway
I think this article answers the question:
http://blogs.msdn.com/b/silverlightws/archive/2010/07/16/pollingduplex-multiple-mode-timeouts-demystified.aspx
The maxOutputDelay is more like an intra-message timer. So if your message rate exceeds this delay you will never trigger a flush until the buffer fills. It gets reset on each new message added to the queue. So I guess we have to tune the queue size as well as this timer to achieve a maximum actual latency.
I'm not sure why the streamed response still buffers but I see it too. Does anyone know how to tune that buffer size?
[Edited]
Ok, this article says that we cannot control the buffering int he streamed response (it is 16k in self-hosted and 32k in IIS). So, given that, it seems like small messages coming in at a rate greater than your maxoutputdelay are a pathological case. Maybe I have to pad them with data...
http://blogs.msdn.com/b/silverlightws/archive/2010/06/25/http-duplex-improvements-silverlight-4.aspx
Related
We are using google-cloud-pubsub (0.24.0-beta) pull client for reading messages from subscriber and seeing high rate of duplicates in that. Google documentation says that little duplication is expected but in our case, we are seeing 80% of messages are getting duplicated even after acknowledgement.
The most weird part is, even if we acknowledge the message immediately in receiver using consumer.ack(), duplicates are still occurring.
Does anybody know how to handle this.
A large number of message duplicates could be the result of flow control settings being set too high or too low. If your flow control settings are too high, where you are allowing too many messages to be outstanding to your client at the same time, then it is possible that the acks are being set too late. If this is the cause, you would probably see the CPU of your machine at or near 100%. In this case, try setting the max number of outstanding messages or bytes to a lower number.
It could also be the case that the flow control settings are set too low. Some messages get buffered in the client before they are delivered to your MessageReceiver, particularly if you are flow controlled. In this case, messages may spend too much time buffered in the client before they are delivered. There is an issue with messages in this state that is being fixed in an outstanding PR. In this scenario, you could either increase your max outstanding bytes or messages (up to whatever your subscriber can actually handle) or you can try to setAckExpirationPadding to a larger value than the default 500ms.
It is also worth checking your publisher to see if it is unexpectedly publishing messages multiple times. If that is the case, you may see the contents of your messages being the same, but they aren't duplicate messages being generated by Google Cloud Pub/Sub itself.
Edited to mention bug that was in the client library:
If you were using a version of google-cloud-pubsub between v0.22.0 and v0.29.0, you might have been running into an issue where a change in the underlying mechanism for getting messages could result in excessive duplicates. The issue has since been fixed.
I'm building a network application in RedHat/C with a protocol called SMPP that is being used in telecom to send SMS.
I'm at a point where I send messages (~70 SMS/second) to the server and I have to wait to a few seconds and for a successful response and delete the messages, but if the message timed-out then I have to resend the message to the server.
The question is how to design something to retry the expired messages?
There is an id in the SMPP spec called sequence_number - this should be monotonically incrementing for every request you make and the response coming back from the server will have the sequence_number of the request it is responding to.
If you wait for a bit (maybe 10 seconds, maybe longer) and you don't get your response back you can re-send the request with the same sequence_number and the server should spot it as a duplicate if it did receive it first time; if it did not receive it first time then it will treat it as a new request.
The server may also make requests to your client; e.g. here is a delivery receipt or here is a mobile-originate message - it will also have it's own sequence_number counter and you should acknowledge it's requests with responses having the same sequence number. You should track the sequence numbers you have seen so you can tell if you hit a duplicate request.
This property is called http://en.wikipedia.org/wiki/Idempotence and is something you should become familiar with if you are implementing telecoms protocols.
In order to get your 70 msgs/sec you will likely need to build on top of Idempotence using a sliding window http://en.wikipedia.org/wiki/Flow_control_(data)#Sliding_Window so you can have a maximum of N (maybe 10) requests outstanding you are still waiting for the response acknowledgements to - unless you are very close to the SMPP server with very low latency.
Doing SMPP right is not trivial I would recommend you read SMPP v3.4 spec front to back before you get too far into an implementation.
It is not very clear what you are asking for so the answer will be also probably not very precise.
I would suggest to see how this is implemented in some existing solutions. I have worked a bit with kannel and mbuni (this is rather for MMS) and I suggest take a look at kannel especially.
Kannel is basically open source SMS gateway and have working SMPP support.
Take also a look at this stackoverflow thread which may also help to understand some ideas.
I have an application where users can collaborate on photo albums. I currently use polling on the client to check for new content every 30 seconds. There can be any number of people uploading and viewing an album at any given time.
On the server side, I cache the data to return (so the query for new content is cheap). I assume that polling every 30 seconds from multiple clients will cause more instances to stay active (and thus increase costs).
Would it be overkill to use the channel api for the above use case instead of polling?
Does the channel api keep instances alive too?
Are there any use cases where polling is preferable instead of using the channel api?
I'm using channels but I'm finding they're not great. If a channel times out from a network disconnect, it somehow screws up the history on my browser. I've filed a bug a bit over a week ago, but it hasn't been acknowledged. There's another bug filed over a month ago that hasn't been acknowledged either - so don't expect quick support on channel issues.
Channels are nice to have - you can notify users in less than a second if status of some sort changes, but they're not reliable. Sometimes the disconnect event doesn't occur, but the channel just stops working. My current system uses channels, but also polls every 5-10 seconds. Because of the unreliability, I wouldn't use channels as a replacement for polling, just a way to give faster response.
Even then you'll have to work out whether it'll save you money. If you're expecting users to leave your app open for 15 minutes without hitting the server, then maybe you'll save some instance time. However, if your users are hitting the server anyways, your instances probably wouldn't get time to shut down. And keeping your instances up actually helps reduce cold starts a bit too.
Google app engine seems to have recently made a huge decrease in free quotas for channel creation from 8640 to 100 per day. I would appreciate some suggestions for optimizing channel creation, for a hobby project where I am unwilling to use the paid plans.
It is specifically mentioned in the docs that there can be only one client per channel ID. It would help if there were a way around this, even if it were only for multiple clients on one computer (such as multiple tabs)
It occurred to me I might be able to simulate channel functionality by repeatedly sending XHR requests to the server to check for new messages, therefore bypassing limits. However, I fear this method might be too slow. Are there any existing libraries that work on this principle?
One Client per Channel
There's not an easy way around the one client per channel ID limitation, unfortunately. We actually allow two, but this is to handle the case where a user refreshes his page, not for actual fan-out.
That said, you could certainly implement your own workaround for this. One trick I've seen is to use cookies to communicate between browser tabs. Then you can elect one tab the "owner" of the channel and fan out data via cookies. See this question for info on how to implement the inter-tab communication: Javascript communication between browser tabs/windows
Polling vs. Channel
You could poll instead of using the Channel API if you're willing to accept some performance trade-offs. Channel API deliver speed is on the order of 100-200ms; if you could accept 500ms average then you could poll every second. Depending on the type of data you're sending, and how much you can fit in memcache, this might be a workable solution. My guess is your biggest problem is going to be instance-hours.
For example, if you have, say, 100 clients you'll be looking at 100qps. You should experiment and see if you can serve 100 requests in a second for the data you need to serve without spinning up a second instance. If not, keep increasing your latency (ie., decreasing your polling frequency) until you get to 1 instance able to serve your requests.
Hope that helps.
Working on a GAE project and one requirement we have is that we want to in a timely manner be able to determine if a user has left the application. Currently we have this working, but is unreliable so I am researching alternatives.
The way we do this now is we have a function setup to run in JS on an interval that sends a heartbeat signal to the GAE app using an AJAX call. This works relatively well, but is generating a lot of traffic and CPU usage. If we don't hear a heartbeat from a client for several minutes, we determine they have left the application. We also have the unload function wired up to send a part message, again through an AJAX call. This works less then well, but most of the time not at all.
We are also making use of the Channels API. One thing I have noticed is that our app when using an open channel, the client seems to also be sending a heartbeat signal in the form of a call to http://talkgadget.google.com/talkgadget/dch/bind. I believe this is happening from the iFrame and/or JS that gets loaded when opening channel in the client.
My question is, can my app on the server side some how hook in to these calls to http://talkgadget.google.com/talkgadget/dch/bind and use this as the heartbeat signal? Is there a better way to detect if a client is still connected even if they aren't actively doing anything in the client?
Google have added this feature:
See https://developers.google.com/appengine/docs/java/channel/overview
Tracking Client Connections and Disconnections
Applications may register to be notified when a client connects to or
disconnects from a channel.
You can enable this inbound service in appengine-web.xml:
Currently the channel API bills you up-front for all the CPU time the channel will consume for two hours, so it's probably cheaper to send messages to a dead channel than to send a bunch of heartbeat messages to the server.
https://groups.google.com/d/msg/google-appengine/sfPTgfbLR0M/yctHe4uU824J
What I would try is attach a "please acknowledge" parameter to every Nth message (staggered to avoid every client acknowledging a single message). If 2 of these are ignored mute the channel until you hear from that client.
You can't currently use the Channel API to determine if a user is still online or not. Your best option for now depends on how important it is to know as soon as a user goes offline.
If you simply want to know they're offline so you can stop sending messages, or it's otherwise not vital you know immediately, you can simply piggyback pings on regular interactions. Whenever you send the client an update and you haven't heard anything from them in a while, tag the message with a 'ping request', and have the client send an HTTP ping whenever it gets such a tagged message. This way, you'll know they're gone shortly after you send them a message. You're also not imposing a lot of extra overhead, as they only need to send explicit pings if you're not hearing anything else from them.
If you expect long periods of inactivity and it's important to know promptly when they go offline, you'll have to have them send pings on a schedule, as you suggested. You can still use the trick of piggybacking pings on other requests to minimize them, and you should set the interval between pings as long as you can manage, to reduce load.
I do not have a good solution to your core problem of "hooking" the client to server. But I do have an interesting thought on your current problem of "traffic and CPU usage" for periodic pings.
I assume you have a predefined heart-beat interval time, say 1 min. So, if there are 120 clients, your server would process heart beats at an average rate of 2 per second. Not good if half of them are "idle clients".
Lets assume a client is idle for 15 minutes already. Does this client browser still need to send heart-beats at the constant pre-defined interval of 1 min?? Why not make it variable?
My proposal is simple: Vary the heart-beats depending on activity levels of client.
When the client is "active", heart-beats work at 1 per minute. When the client is "inactive" for more than 5 minutes, heart-beat rate slows down to 50% (one after every 2 minutes). Another 10 minutes, and heart-beat rate goes down another 50% (1 after every 4 minutes)... At some threshold point, consider the client as "unhooked".
In this method, "idle clients" would not be troubling the server with frequent heartbeats, allowing your app server to focus on "active clients".
Its a lot of javascript to do, but probably worth if you are having trouble with traffic and CPU usage :-)