I'm building a network application in RedHat/C with a protocol called SMPP that is being used in telecom to send SMS.
I'm at a point where I send messages (~70 SMS/second) to the server and I have to wait to a few seconds and for a successful response and delete the messages, but if the message timed-out then I have to resend the message to the server.
The question is how to design something to retry the expired messages?
There is an id in the SMPP spec called sequence_number - this should be monotonically incrementing for every request you make and the response coming back from the server will have the sequence_number of the request it is responding to.
If you wait for a bit (maybe 10 seconds, maybe longer) and you don't get your response back you can re-send the request with the same sequence_number and the server should spot it as a duplicate if it did receive it first time; if it did not receive it first time then it will treat it as a new request.
The server may also make requests to your client; e.g. here is a delivery receipt or here is a mobile-originate message - it will also have it's own sequence_number counter and you should acknowledge it's requests with responses having the same sequence number. You should track the sequence numbers you have seen so you can tell if you hit a duplicate request.
This property is called http://en.wikipedia.org/wiki/Idempotence and is something you should become familiar with if you are implementing telecoms protocols.
In order to get your 70 msgs/sec you will likely need to build on top of Idempotence using a sliding window http://en.wikipedia.org/wiki/Flow_control_(data)#Sliding_Window so you can have a maximum of N (maybe 10) requests outstanding you are still waiting for the response acknowledgements to - unless you are very close to the SMPP server with very low latency.
Doing SMPP right is not trivial I would recommend you read SMPP v3.4 spec front to back before you get too far into an implementation.
It is not very clear what you are asking for so the answer will be also probably not very precise.
I would suggest to see how this is implemented in some existing solutions. I have worked a bit with kannel and mbuni (this is rather for MMS) and I suggest take a look at kannel especially.
Kannel is basically open source SMS gateway and have working SMPP support.
Take also a look at this stackoverflow thread which may also help to understand some ideas.
Related
I am struggling to work out how to use zmq to implement the architecture I need. I have a classic publish/subscribe situation except that once client x has subscribed to a topic I need the topic data to be sent to it to be cached if the client dies and resent on reconnect. The data order is important and I can't miss messages should the client be offline for a while.
The PUB/SUB pattern doesn't seem to know about individual clients and will just stop sending to client x if it dies. Plus I can't find out this has happened and cache the messages, or know when it reconnects.
To try to get around this I used the REQ/REP pattern so the clients can announce themselves and have some persistence but this is not ideal for a couple of reasons:
1) The clients must constantly ask "got any data for me?" which offends my sensibilities
2) What happens if there's no data to send to client x but there is to client y? Without zmq I'd have had a thread per client and simply block the one with no data but I can't block client x without also blocking client y in a single thread.
Am I trying to shove a round peg in a square hole, here? Is there some way I can get feedback from PUB saying 'failed to send to client x'? so I can cache the messages instead? Or is there some other pattern I should be using?
Otherwise it's back to low level tcp for me...
Many thanks;
Jeremy
This is an area of active research.
I'm currently working on something similar. Our solution is to have a TCP "back channel" on which to receive missed data and have the subscribers know what the last successfully received publication was so that when they reconnect, they can ask for publications since that one.
In some sense you are trying to shove a round peg in a square hole. You have choosen the tool - PUB/SUB - and are trying to solve a problem it are not designed to solve, at least not without some additional design.
The PUB/SUB is an unreliable broadcast. The client can miss messages for several reasons:
Subscribers join late, so they miss messages the server already sent.
Subscribers can fetch messages too slowly, so queues build up and then overflow.
Subscribers can drop off and lose messages while they are away.
Subscribers can crash and restart, and lose whatever data they already received.
etc...
For REQ/RSP the client do not have to constantly ask "got any data for me?", instead the client should probably acknowledge every data so that the server can send correct data next time. If the server has nothing to send, it is just quite.
eg.
client server
Hello ---------------->
(wait until something exist to send)
<-------------------- Msg 1
Ack 1 ---------------->
(wait ...)
<-------------------- Msg 2
...
There are several good ways to do what you want with zmq. First of all you should try to design your protocol. What shall happen when I connect? Should I get any old messages then? If so, how old? If i miss a message when I am connected, should I be able to get it? If the client restarts, should I then get any old messages?
I strongly recommend the very good zmq guide http://zguide.zeromq.org/page:all that have a lot of very good information regardning different ways to get reliability in a protocol. Read the complete guide, including the chapters 4 and 5 whih discuss different techniques on getting a reliable transport. Based on your problem discussion: the Chapter 5 seems like a good start. Try out some of the examples. Then design your protocol.
What about adding an Archiver process. Part of a Client's subscription process would be to also notify the Archiver to start archiving the same subscription(s). The Archiver would keeps all the messages received in an ordered list.
The Clients would record the time or id of the last published message they received. When they started after a crash, they would first contact the Archiver and say "Give me all messages since X". And they would resubscribe with the Publisher. When a client receives the same message from both the Publisher and the Archiver, it tells the Archiver to stop replaying.
The Archiver could purge messages older then the max expected down time for an offline client. Or alternately, Clients could periodically check in to say "I am up to date with message Y", allowing purging of all older items.
I use PushSharp to send notifications for a few Apps.
PushSharp is great it really simplifies the work with push services, and I wonder what is the right way to work with it?
I haven't found examples/ explanations about that.
Now, when I have a message to send , I ...
create a PushSharp object
do a PushService.QueueNotification() for all devices
do a PushService.StopAllServices to send all queued messages
exits the method (and kill the PushService object).
Should I work this way, or keep this PushService object alive and call its methods when needed?
How should I use a PushService object to get the unregistered device ids? with a dedicated instance?
Any suggestion would be appreciated.
This is a question which frequently comes up.
The answer isn't necessarily one way or the other, but it depends on your situation. In most cases it would be absolutely fine to just create a PushBroker instance whenever you need it, since most platforms use HTTP based protocols for sending notifications. In the case of Apple, they state in their documentation that you should keep your connection to APNS open in order to minimize overhead of opening and closing secure connections.
However, in practice I think this means that they don't want you connecting and disconnecting VERY frequently (eg: they don't want you creating a new connection for every message you send). In reality, if you're sending batches of notifications every so often (let's say every 15 minutes or every hour) they probably won't have a problem with you opening a new connection for each batch and then closing it when done.
I've never heard of anyone being blocked from Apple's APNS servers for doing this. In fact in the very early days of working with push notifications, I had a bug that caused a new apns connection to be created for each notification. I sent thousands of notifications a day like this and never heard anything about it from Apple (eventually I identified it as a bug and fixed it of course).
As for collecting feedback, by default the ApplePushService will poll the feedback servers after 10 seconds of starting, and then every 10 minutes thereafter. If you want to disable this from happening you can simply set the ApplePushChannelSettings.FeedbackIntervalMinutes to <= 0. You can then use the FeedbackService class to poll for feedback whenever you need to, manually.
I have a minor question related to the SNMP protocol.
I know that on trap mode, the agent can report to the manager the messages, and this ones can be order using the timestamp field of the SNMP.
But on request/response mode, when the queries are made from the manager to the agent, there is no timestamp field on the request message neither on response message, so how can the manager order this messages?
Imagine a world where many requests are done and many responses are done... ? Does the messages are ordered on the application like wireshark related to the time of the actual capture on wireshark..? or..?
Thanks alot in advance ;)
Does the messages are ordered on the application like wireshark related to the time of the actual capture on wireshark
Yes.
The only "ordering" that the manager can do is to sort by the arrival time of the responses.
Imagine a world where many requests are done and many responses are done
The manager would match requests and replies by response ID.
Working on a GAE project and one requirement we have is that we want to in a timely manner be able to determine if a user has left the application. Currently we have this working, but is unreliable so I am researching alternatives.
The way we do this now is we have a function setup to run in JS on an interval that sends a heartbeat signal to the GAE app using an AJAX call. This works relatively well, but is generating a lot of traffic and CPU usage. If we don't hear a heartbeat from a client for several minutes, we determine they have left the application. We also have the unload function wired up to send a part message, again through an AJAX call. This works less then well, but most of the time not at all.
We are also making use of the Channels API. One thing I have noticed is that our app when using an open channel, the client seems to also be sending a heartbeat signal in the form of a call to http://talkgadget.google.com/talkgadget/dch/bind. I believe this is happening from the iFrame and/or JS that gets loaded when opening channel in the client.
My question is, can my app on the server side some how hook in to these calls to http://talkgadget.google.com/talkgadget/dch/bind and use this as the heartbeat signal? Is there a better way to detect if a client is still connected even if they aren't actively doing anything in the client?
Google have added this feature:
See https://developers.google.com/appengine/docs/java/channel/overview
Tracking Client Connections and Disconnections
Applications may register to be notified when a client connects to or
disconnects from a channel.
You can enable this inbound service in appengine-web.xml:
Currently the channel API bills you up-front for all the CPU time the channel will consume for two hours, so it's probably cheaper to send messages to a dead channel than to send a bunch of heartbeat messages to the server.
https://groups.google.com/d/msg/google-appengine/sfPTgfbLR0M/yctHe4uU824J
What I would try is attach a "please acknowledge" parameter to every Nth message (staggered to avoid every client acknowledging a single message). If 2 of these are ignored mute the channel until you hear from that client.
You can't currently use the Channel API to determine if a user is still online or not. Your best option for now depends on how important it is to know as soon as a user goes offline.
If you simply want to know they're offline so you can stop sending messages, or it's otherwise not vital you know immediately, you can simply piggyback pings on regular interactions. Whenever you send the client an update and you haven't heard anything from them in a while, tag the message with a 'ping request', and have the client send an HTTP ping whenever it gets such a tagged message. This way, you'll know they're gone shortly after you send them a message. You're also not imposing a lot of extra overhead, as they only need to send explicit pings if you're not hearing anything else from them.
If you expect long periods of inactivity and it's important to know promptly when they go offline, you'll have to have them send pings on a schedule, as you suggested. You can still use the trick of piggybacking pings on other requests to minimize them, and you should set the interval between pings as long as you can manage, to reduce load.
I do not have a good solution to your core problem of "hooking" the client to server. But I do have an interesting thought on your current problem of "traffic and CPU usage" for periodic pings.
I assume you have a predefined heart-beat interval time, say 1 min. So, if there are 120 clients, your server would process heart beats at an average rate of 2 per second. Not good if half of them are "idle clients".
Lets assume a client is idle for 15 minutes already. Does this client browser still need to send heart-beats at the constant pre-defined interval of 1 min?? Why not make it variable?
My proposal is simple: Vary the heart-beats depending on activity levels of client.
When the client is "active", heart-beats work at 1 per minute. When the client is "inactive" for more than 5 minutes, heart-beat rate slows down to 50% (one after every 2 minutes). Another 10 minutes, and heart-beat rate goes down another 50% (1 after every 4 minutes)... At some threshold point, consider the client as "unhooked".
In this method, "idle clients" would not be troubling the server with frequent heartbeats, allowing your app server to focus on "active clients".
Its a lot of javascript to do, but probably worth if you are having trouble with traffic and CPU usage :-)
I'm trying to enable a streamed response using Silverlight 4 and polling duplex, but I'm getting strange behaviour when the rate at which updates are sent to the client is greater than the maxOutputDelay, which results in no updates being sent.
For example, with a maxOutputDelay of 7 seconds, and 1 update sent every 10 seconds, everything works fine. But if I have a maxOutputDelay of 1 second, and an update sent every 500 milliseconds, the updates just sit on the server side and don't get sent to the client.
It's my understanding that setting transferMode="StreamedResponse" should send the updates immediately to the client, but this doesn't seem to be working.
Here's the binding in my Web.config for the web service:
This config is based on the information from this article: http://blogs.msdn.com/b/silverlightws/archive/2010/06/25/http-duplex-improvements-silverlight-4.aspx
Thanks.
If you are not totally focussed on using Duplex Channels (which are a pain to configure in anything but a single host scenario) it might be worth checking out alternative solutions for implementing Server Callbacks - even if that means that you have to maintain two different types of connection to your backend.
Duplex Channel Alternatives:
PokeIn
Kaazing WebSocket Gateway
I think this article answers the question:
http://blogs.msdn.com/b/silverlightws/archive/2010/07/16/pollingduplex-multiple-mode-timeouts-demystified.aspx
The maxOutputDelay is more like an intra-message timer. So if your message rate exceeds this delay you will never trigger a flush until the buffer fills. It gets reset on each new message added to the queue. So I guess we have to tune the queue size as well as this timer to achieve a maximum actual latency.
I'm not sure why the streamed response still buffers but I see it too. Does anyone know how to tune that buffer size?
[Edited]
Ok, this article says that we cannot control the buffering int he streamed response (it is 16k in self-hosted and 32k in IIS). So, given that, it seems like small messages coming in at a rate greater than your maxoutputdelay are a pathological case. Maybe I have to pad them with data...
http://blogs.msdn.com/b/silverlightws/archive/2010/06/25/http-duplex-improvements-silverlight-4.aspx