I was implementing WebSockets just for practice and I encountered an architectural problem.
It's nice to have WebSockets, but I cannot figure out a simple scalable scenario.
Possible Scenario:
Browser users start some computationally difficult task over the frontend. It goes over the API server, API puts the task to a queue, some other GPU server with celery pulls the task and starts working on it. Somewhere on the way, possibly, there is a database saving a state. So I would say API and celery server writes in the DB under particular task information about what's going on.
Now the important part. There is a WebSocket server connected to the browser client. It would be great that WebSockets are simplex and only sends messages to browser clients about the progress of the task (status, progress bar % and etc). The WebSocket is clever and doesn't need periodical polling, but manages to send data to the browser client based on events that are triggered (by API and celery). Obviously, the WebSocket server needs to listen to this task state (Redis or something, certainly not something at the same place as is WebSocket server). This means that in the WebSocket loop there must be a listener for this state. But this ends up back to WebSocket server polling this redis or something for seeing the state of the task -> this is certainly connection killer in case of a lot of users as there will be a lot of WebSocket connections polling same database.
The question is then: How to solve this in terms of architecture(no polling, WebSockets sends messages only on the state change of some value in some DB)?
I'd propose that celery server also sends a task information to some queue. The WebSocket server would have to have a code responsible for reading from that queue and distributing that task information to its clients (WebSocket connections) that listen for that particular task information.
Related
I have an app that, when activated, uploads location data. Currently it sends the data to the server via REST, however I would like to save on server costs and send the data via iot-core.
Previously, I would queue location updates, and only send them about once every few minutes, this way the phone would only turn on its data broadcasting once every few minutes and not keep it constantly, and save on battery life.
Is there a way to enable similar battery saving when uploading to AWS iot-core? I haven't run tests, but I assume that constantly sending messages via mqtt, websockets, or http is just as battery draining as regular rest messages.
This is somewhat related to Aws IoT Message Delivery.
We are making a web application in Go with a MySQL database. Our users are allowed to only have one active client at a time. Much like Spotify allows you to only listen to music on one device at a time. To do this I made a map with as key the user ids and a reference to their active websocket connection as a value. Based on the websocket id that the client has to send in the header of the request we can identify weather the request comes from their active session.
My question is if it's a good practice to store data (in this case the map with the user ids and websockets) in a global space or is it better to store it in the database.
We don't expect to reach over 10000 simultaneously active clients. Average is probably gonna be around 1000.
If you only run one instance of the websocket server storing it in memory should be sufficient. Because if it for some reason goes down/restarts then all the connections will be lost and all the clients will have to create them again (and hence the list of connection will once again be populated by all the clients who want to use the service).
However, if you plan on scaling it horizontally so you have multiple websocket services behind a load balancer, then the connections may need to be stored in a database of some sort. And not because it necessarily needs to be more persistant but because you need to be able to check the request against all the services connections.
It is also possible to have a separate service which handles the incoming request and asks all the websocket services if any of them have the connection specified in the request. This could be done if you add a pub/sub queue and every websocket service subscribes to channels for all its websocket ids and the service that receives the request then publishes the websocket id, and the websocket services can then send back replies on a separate channel if they have that connection. You must decide how to handle if no one is responding (no websocket service has the websocket id). Either the channel does not exist, or you expect the answer within a specific time. Or you could publish the question on a general topic and expect all the websocket services to reply (yes or no).
And regarding whether you need to scale it I guess depends mostly on the underlying server you're running the service on. If I understand it correctly the websocket service will basically not do anything except from keeping track of its connections (you should add some ping pong to discover if connections are lost). Then your limitation should mainly be on how many file descriptors your system can handle at once. If that limit is much larger than your expected maximum number of users, then running only one server and storing everything in memory might be an OK solution!
Finally, if you're in the business of having a websocket open for all users, why not do all the "other" communication over that websocket connection instead of having them send HTTP requests with their websocket id? Perhaps HTTP fits better for your use case but could be something to think about :)
My application allows a user to enter a URL of an article he/she wishes to analyze. It goes through our API gateway to reach the correct services engaged in this process. The analysis takes between 5 and 30 seconds depending on the article's word count.
For now, my reactjs client sends the request to the API and waits for 5 to 30 seconds to receive the response. Is there a better way to handle this such as enqueuing the job and let the API ping the client (reactjs frontend) once it has been done?
Server-sent Events (SSEs) allow your server to push new information to your browser, and hence look ideal to me for this purpose. They work over HTTP and there is good support for all browsers except for IE.
So the new process could look as follows:
Client send request to server, which initiates the lookup and potentially responds with the topic the browser needs to subscribe to (in case that's unique per lookup)
Server does its thing and sends updates as it processes new content. See how the beauty of this is that you could inform your client about partial updates.
If SSEs is not an option to you, you could leverage good old Websockets for bi-directional communication, but for such a simple endeavor, it might be too much technology to solve the problem.
A third alternative, especially if you are talking amongst services (no web or mobile clients on the other side) is to use web-hooks, so that the interested party would expose and listen on a specific endpoint, that the publisher (the server that does the processing) would write updates to.
Hope this is useful.
we wrote in C++ a screen sharing application based on sending screenshots.
It works by establishing a TCP connection btw the server and client, where the server forwards every new screenshot received for a user through the connection, and this is popped-up by the client.
Now, we are trying to host this on google app engine, and therefore need 'servlet'-ize and 'sandbox' the server code, so to implement this forwarding through HTTP requests.
I immagine the following:
1. Post request with the screenshot as multiple-data form (apache uploads ..).
But now the server needs to contact the specified client (who is logged in) to send it/forward the screenshot.
I'm not sure how to 'initiate' such connection from the servlet to the client. The client doesn't run any servlet environment (of course).
I know HTTP 1.1 mantains a TCP connection, but it seems gapps won't let me use it.
1 approaches that comes to mind is to send a CONTINUE 100 to every logged in user at login, and respond with the screenshot once it arrives. Upon receival the client makes another request, and so on.
an alternative (insipired from setting the refresh header for a browser) would be to have the app pool on a regular basis (every 5 secs).
You're not going to be able to do this effectively on GAE.
Problem 1: All output is buffered until your handler returns.
Problem 2: Quotas & Limits:
Some features impose limits unrelated
to quotas to protect the stability of
the system. For example, when an
application is called to serve a web
request, it must issue a response
within 30 seconds. If the application
takes too long, the process is
terminated and the server returns an
error code to the user. The request
timeout is dynamic, and may be
shortened if a request handler reaches
its timeout frequently to conserve
resources.
Comet support is on the product roadmap, but to me your app still seems like a poor fit for a GAE application.
Long Polling is the concept used for such asynchronous communications between server and client.
In Long Polling, servlet keeps a map of client and associated messages. Key of Map being client id and value being list of messages to be sent to the client. When a client opens a connection with server (sends request to a servlet), the servlet checks the Map if there are any messages to be sent to it. If found, it sends the messages to the client exits from the method. On receiving messages, the client opens a new connection to the server. If the servlet does not find any messages for given client, it waits till the Map gets updated with messages for given client.
This is a late reply, I'm aware, but I believe that Google have an answer for this requirement: the Channel API.
Can anyone think of a good way to allow the server to notify the client based upon server processing? For example, consider the following events:
A user requests a deletion of data, however, due to it's long-running time, we kick it off to a queue.
The client receives a "Yes we completed your transaction successfully".
The server deletes the item and now wants to update any local structures any clients may be using (I'd also like to notify the user).
I know this can be done by client-side polling. Is there a event bus type way to do this? Any suggestions are welcome, but please keep in mind I am using GWT with App Engine.
The standard AJAX interaction is that the client sends requests to the server and expects some sort of response back fairly quickly.
In order for the server to initiate a request to the client, you will need to use WebSockets, and experimental HTML5 feature currently only supported by Chrome.
Or, to simulate this kind of interaction, you can use Comet (long-polling), made available in GWT by the rocket-gwt project.
You want server events for GWT? Have a look at GwtEventService (they couldn't have chosen a better name): http://code.google.com/p/gwteventservice/wiki/StartPage
Of course, it uses a Comet implementation, but you can't do any different when using HTTP, the client always initiates the communication. Request, response.