I have a server that sends data as fast as it can produce it and sends the data over a socket. The server uses a queue and has a producer thread and a consumer thread that sends the produced data out a socket to the client.
The problem is reading the data on the client side. How do I design a client to handle the data without it being out of sync?
If I send an acknowledgement from the client to the server I lose the concurrency speed on the server side. How can I write/design a client to handle the incoming data fast enough?
Do I need to implement a queue on the client side?
Unless you have a requirement that you must use something other than TCP, just let TCP do the job of flow control for you. Let the client consume the data as fast as it wants to, and the server will block after it sends more data than the client is prepared to consume and it fills up the TCP window.
TCP will never get out of sync in the sense that data on the socket will always be delivered in order. But the server may certainly have sent out more data than the client has consumed and so it may have moved on to sending the next batch of data while the client is still consuming the previous one. Is this what you mean by out of sync?
You don't want to make the client send an acknowledge before the server starts on the next task because that will cost an RTT (round trip time, i.e. the time for the last of one batch of data to arrive at the client and for the acknowledge to go back), which will slow down your protocol on a high-latency link.
If you don't want this RTT price, you are inevitably going to have to allow either:
for the client to request more than one batch at a time. You can use a tagged protocol like IMAP for this: the client submits several jobs at once on one socket, each with its own tag. The server responds to each request, with the tags in the header of each response so the client knows which response goes with which request. When the client has seen "enough" responses, it submits more requests. The client gets to control how many requests can be ongoing at the same time. If the client allows only one at a time, this degenerates to the simple ACK case with the RTT cost.
for the server to work a little ahead of the client, sending several responses to the client before the client has acknowledged the first one. After the pipe is filled to the maximum number of unacknowledged jobs that the server is willing to allow, it waits to acknowledged and sends one additional job response for each acknowledge it receives from the client. If the server allows only one outstanding job, this degenerates to the simple ACK case as above. If the server allows too many unacknowledged jobs at a time, this degenerates to just filling up TCP's buffers and counting on TCP flow control to block the server until the client is ready to accept more data.
Related
I'm not an advanced user, so please cope with me.
I'm trying to implement a WebSocket client using libcurl, and I'm good until the last step of a connection - termination.
The general logic is as follows:
Client connects and sends an upgrade request.
Websocket server accepts/upgrades and starts sending gibberish.
Client adds up all the gibberish sizes.
Server sends a closing signal after 10 secs.
So far so good. I'm not processing the payloads of incoming messages, and I don't want to. I have very limited resources and I don't want to experience any performance loss in order to check each payload and search for a close signal.
I'm using libcurl's easy interface and receive data with curl_easy_perform(). Is there any way to detect a close signal, or close the websocket connection after 10 secs?
Close signals are part of the WebSocket protocol at the framing layer (see RFC 6455 Sections 1.4, 5, and 5.5.1).
AFAIK, libcurl doesn't natively support WebSockets, just HTTP (which a WebSocket uses for its opening handshake, so you can fake it with libcurl). So, if libcurl doesn't process the WebSocket frames for you, you would have to process them yourself, even if you ignore their payloads.
Otherwise, just set a 10-second timer for yourself and close the underlying TCP connection directly, which you can get from libcurl using curl_easy_getinfo(CURLINFO_ACTIVESOCKET).
But, if the server is sending you a close signal, you SHOULD send one back, per Section 5.5.1, which means parsing the frames properly:
If an endpoint receives a Close frame and did not previously send a Close frame, the endpoint MUST send a Close frame in response. (When sending a Close frame in response, the endpoint typically echos the status code it received.) It SHOULD do so as soon as practical. An endpoint MAY delay sending a Close frame until its current message is sent (for instance, if the majority of a fragmented message is already sent, an endpoint MAY send the remaining fragments before sending a Close frame). However, there is no guarantee that the endpoint that has already sent a Close frame will continue to process data.
After both sending and receiving a Close message, an endpoint considers the WebSocket connection closed and MUST close the underlying TCP connection. The server MUST close the underlying TCP connection immediately; the client SHOULD wait for the server to close the connection but MAY close the connection at any time after sending and receiving a Close message, e.g., if it has not received a TCP Close from the server in a reasonable time period.
If a client and server both send a Close message at the same time, both endpoints will have sent and received a Close message and should consider the WebSocket connection closed and close the underlying TCP connection.
I have a client-server programm. They communicate with some characters.
ex:
client --send A-> server
then
client <-recv A'-- server
But I let server send message back not in order.
ex:
client --A--> --B--> --C--> server
then
client <--A'-- <--C'-- <--B'-- server
what I want:
client <--A'-- <--B'-- <--C'-- server
so I want to handle the situation in client programm.
I only figure out one way is to keep a buffer to record data from server, and client will check the B' received and then check the C' received in order.
Is there anyway to do that in client and avoid using buffer?
Ideally you should send and then wait for reply, send next request when you received earlier reply.
But apparently your client seems to send without waiting for reply and server replies whichever request its done processing.
In such case you need to device a mechanism in your client and server program based on the data being sent and received.
You can decide your client and server will work on data having(Header+data) format something like:
length(2 bytes) - length of actual data
sequence numner(2 bytes) - to be incremented for each request
Actual data(length number of bytes)
When you send the data keep the request data, after sending over socket, in a pending list of requests waiting for reply from server.
The server on receiving request in above format, will update its reply data in the 'Actual data' section , update the length in header, keeping sequence as it is, server will send reply to client.
Client will match the sequence number from reply with the items in its pending list to get the request whose reply it received from server
I am trying to design a Client Server kind of application in which my Server is a daemon that accepts client requests, send client's data over a serial channel to the other side(which is an MCU and its firmware will reply to the Server request over the same serial channel). My client can be a CLI application or any other system program.
My idea of design is -
Use message queues for communication between Client and Server since this is a local application and message queues are bidirectional and fast.
Implement a LIBRARY that acts as an interface between multiple clients and the server. This basically does the stuff of packetizing client data into a message(own defined protocol), create message queues, connect to server, send/receive data and then pass it to the respective client(using call backs). This library also exposes API that can be used by clients. Thus this library gives me the flexibility to add support for any new clients keeping the server program unchanged.
Server gets the data over serial from other side and passes it to the library over message queue. The library uses callbacks to send data to the client.
EDIT:
I am thinking of creating Message queues on the fly when any client requests arrive. If I do this, how does the Server daemon(which has already started at linux boot up) gets information about this message queue? Does the message queue has a name that is persistent across and used by other programs? I want to implement clients that will be blocked until it gets response from the server.
Could you guys please review this design and tell me whether my approach is correct. Please reply if you have any other recommendations.
Thanks in advance.
Let's say we have several clients connected to App Engine using Channel API. Each client sends messages, which should be propagated to other conntected clients according to some rules. The tricky part is that clients may not be to the same App Engine instance.
Is there any way to push data from one instance to the others?
(Yes, I know about Memcache, but this would require some kind of polling.)
You're asking two questions here.
a. Can you push data from one instance to another without the use of polling. The answer is generally no.
b. Can one client send messages to the server that can be propagated to other clients? Yes, and this does not require propagating messages to other server-side instances.
Consider the Channel API as a service. Clients are connected to the Channel API service; they are not connected to any particular instance. Therefore any instance can send messages to any client.
You'll need to store the Channel tokens of your clients in the datastore, in some way that's queryable to match your rules.
Your client makes an HTTP request to send a message to your server.
The handler on the server queries for channel tokens that it needs to propagate the message to (either from memcache or datastore).
The handler on the server sends messages to all the clients.
If the list of destination clients is extremely large, you might want to do steps 3/4 in a task queue where the operation can run longer.
It does not matter what instance a client is connected to, that's hidden from you by the API.
Clients can only "reply" to message via standard HTTP commands, they don't actually have any way to respond via the channel API directly.
So Client A on server A1 wants to sent a message to client B on server B1.
Client A posts to a handler. That might be instance A1 or B1. It does not matter which as the server now passes the message on to client B whatever server client B is connected to via the Channel API.
The real point is that no App Engine instance has any data at all, in general. So it does not matter which instance you connect to, it might be the 99th instance or the very first to start up. So you have to design your application so that it's irrelevant what instance is in use.
Client sends message to server via HTTP.
Server sends message to N clients via the channel API.
Channel API does not make a fixed frontend-instance-to-client connection. Any frontend instance can push message to channel if it knows the channel ID.
What you need to do is pass messages cross-channel.
User one sends message normally to server (e.g. via GET)
Server looks up channel ID of second user and pushes the message
Repeat procedure in other direction: second user to first user.
we wrote in C++ a screen sharing application based on sending screenshots.
It works by establishing a TCP connection btw the server and client, where the server forwards every new screenshot received for a user through the connection, and this is popped-up by the client.
Now, we are trying to host this on google app engine, and therefore need 'servlet'-ize and 'sandbox' the server code, so to implement this forwarding through HTTP requests.
I immagine the following:
1. Post request with the screenshot as multiple-data form (apache uploads ..).
But now the server needs to contact the specified client (who is logged in) to send it/forward the screenshot.
I'm not sure how to 'initiate' such connection from the servlet to the client. The client doesn't run any servlet environment (of course).
I know HTTP 1.1 mantains a TCP connection, but it seems gapps won't let me use it.
1 approaches that comes to mind is to send a CONTINUE 100 to every logged in user at login, and respond with the screenshot once it arrives. Upon receival the client makes another request, and so on.
an alternative (insipired from setting the refresh header for a browser) would be to have the app pool on a regular basis (every 5 secs).
You're not going to be able to do this effectively on GAE.
Problem 1: All output is buffered until your handler returns.
Problem 2: Quotas & Limits:
Some features impose limits unrelated
to quotas to protect the stability of
the system. For example, when an
application is called to serve a web
request, it must issue a response
within 30 seconds. If the application
takes too long, the process is
terminated and the server returns an
error code to the user. The request
timeout is dynamic, and may be
shortened if a request handler reaches
its timeout frequently to conserve
resources.
Comet support is on the product roadmap, but to me your app still seems like a poor fit for a GAE application.
Long Polling is the concept used for such asynchronous communications between server and client.
In Long Polling, servlet keeps a map of client and associated messages. Key of Map being client id and value being list of messages to be sent to the client. When a client opens a connection with server (sends request to a servlet), the servlet checks the Map if there are any messages to be sent to it. If found, it sends the messages to the client exits from the method. On receiving messages, the client opens a new connection to the server. If the servlet does not find any messages for given client, it waits till the Map gets updated with messages for given client.
This is a late reply, I'm aware, but I believe that Google have an answer for this requirement: the Channel API.