I am aware that a WebClient instance can only support a single asynchronous request at any one time, therefore in order to perform concurrent requests you need to create multiple WebClient instances. However, is there a limit to the number of WebClients I could, or should create?
For example, if I need to download 100 files, can I just create 100 WebClients? will Silverlight manage this with some sensible concurrency limits? or is that my job? Or do I need to create my own queue mechanism for requests?
Also, what about Windows Phone 7?
If you are connection to a single host (server) the connection limit imposed by the browser is 2 outgoing connections at a time. The same limit would be there in Windows Phone but i am not sure. Therefore more than 2 requests made either from the same webclient or different would get queued. Check here
Related
App Engine Python standard environment (threadsafe)
Cloud SQL MySQL Second Generation
All request handlers need database and are user-facing
no ORM, just plain SQL
According to Cloud SQL: Pricing and Access Limits with our machine type Cloud SQL has a limit of max 4,000 concurrent connections per app, and more importantly in our case, when connecting from App Engine standard environment, a limit of max 60 concurrent connections per app instance to the Cloud SQL instance. (The other limits aren't close to become a bottleneck)
The most likely bottleneck would be the max 60 concurrent connections per app instance. I have no numbers available, so I'm not sure whether a single app instance (F1/B1) could even serve more than 60 users at the same time (probably not).
The current code makes sure that for every request (where database is needed), a connection is created and finally closed (even when an exception occurred earlier. The connection might be closed earlier depending on the situation. I'm not sure if this is the best approach. Simplified function as called from a webapp2.RequestHandler class:
def handle_FAW_Request_approve(reqh, ref):
try:
conn = connect_to_cloudsql()
# do queries, maybe updates
# eventually conn.close() earlier if database not needed anymore
# do something else
# return response
finally:
try:
conn.close()
except Exception as err:
# connection never existed or already closed
pass
Answers to this question suggest a (threadsafe) way to re-use database connections (or cursors). I assume this would help to save the time that is otherwise lost with any overhead of opening/closing connections, but yet no safe-guard that the app instance will stay within the connection limits, right?
Is App Engine able to detect the connections problem by itself and automatically spawn new instances (with auto-scaling) instead of directing more traffic to that instance?
Or does the app need to deal with the limit by itself? What would the solution be to avoid user-facing server errors occurring randomly by exceeding the given limit?
App Engine has built in limits on how many requests it will send to a single instance before it spins up a new one. With automatic scaling, the default value of max_concurrent_requests is 8 so it's unlikely you would reach 60 concurrent requests on a single instance unless something goes really wrong.
You should also consider raising the max_concurrent_requests value as it's likely you can get more out of a single instance than 8 requests.
I would like to use 1 CURL handle and do let's say 10 concurrent requests with this handle. Would that be possible? The problem is if I want for example 100 concurrent requests it opens too many connections and sometimes server refuses to answer because too many connections are already opened from the same IP. But if I had one handle and use this handle for many requests in multiple threads that would probably solve the problem. Any idea if this is possible?
If you really want to do multiple requests in parallel on the same single connection, you need to use HTTP/2 and all those requests have to be made to the same host. That's an usual situation though. You then need to ask libcurl to use HTTP/2 and you need to use the multi interface. Like in the http2-download.c example.
If you have multiple URLs to different hosts and want to limit the number of connections used to transfer those, you can use the easy interface and get the URLs one by one to keep down the number of used connections - in combination with CURLOPT_MAXCONNECTS.
If you want to use the multi interface, you can still allow libcurl to do a limited amount of parallel transfers with CURLMOPT_MAX_TOTAL_CONNECTIONS and friends, even if you add a hundred easy handles at once. Or you can just limit the number of concurrently added easy handles.
I am looking for what are the commonly used / best practices in industry.
Assume the following hypothetical scenario:
If my app server accepts 200 user requests, and each of them need DB access.
But my DB max_connections are 100.
If all 200 users request at the same time, but we have only 100 max_connections, what happens to the other requests, which were not served max connections ?
In real world:
will remaining 100 requests be stored in some sort of a queue on apps servers, and kept waiting for DB connections ?
do we error out ?
Basically, if your database server can only handle 100 connections, and all of your web connections "require database access," you must ensure that no more than 100 requests are allowed to be active at any one instant. This is the “ruling constraint” of this scenario. (It may be one of many.)
You can accept "up to 200 simultaneous connections" on your server, but you must enqueue these requests so that the limit of 100 active requests is not exceeded.
There are many, many ways to do that: load balancers, application servers, even Apache/nginix directives. Sometimes, the web-page is only a front-end to a process that is broken-out among many different servers working in parallel. No matter how you do it, there must be a way to regulate how many requests are active, and to queue the remainder.
Also note that, even though you might have “200 active connections” to your web server, it is highly unlikely that all 200 of these clients “clicked their mouse at precisely the same time.” Requests come in at random rates and therefore might never encounter any sort of delay at all. But your system must nonetheless be engineered to handle the worst case.
https://ringpop.readthedocs.org/en/latest/
To my understanding, the sharding can be implemented in some library routines, and the application programs are just linked with the library. If the library is a RPC client, the sharding can be queried from the server side in real-time. So, even if there is a new partition, it is transparent to the applications.
Ringpop is application-layer sharding strategy, based on SWIM membership protocol. I wonder what is the major advantage at the application layer?
What is the other side, say the sharding in the system layer?
Thanks!
Maybe a bit late for this reply, but maybe someone still needs this information.
Ringpop has introduced the idea of 'sharding' inside application rather then data. It works more or less like an application level middleware, but with the advantage that it offers an easy way to build scalabale and fault-tolerance applications.
The things that Ringpop shards are the requests coming from clients to a specific service. This is one of its major advantages (there are mores, keep reading).
In a traditional SOA architecure, all requests for a specific serveice goes to a unique system that dispatch them among the workers for load balancing. These workers do not know each other, they are indipendent entities and cannot communicate between them. They do their job and sent back a reply.
Ringpop is the opposite: the workers know each other and can discover new ones, regularly talk among them to check their healthy status, and spread this information with the other workers.
How Ringpop shard the request?
It uses the concept of keyspaces. A keyspace is just a range of number, e.g. you are free to choice the range you like, but the obvious choice is hash the IDs of the objects in the application and use the hashing-function's codomain as range.
A keyspace can be imaginated as an hash "ring", but in practice is just a 4 or 8 byte integer.
A worker, e.g. a node that can serve a request for a specific service, is 'virtually' placed on this ring, e.g. it owns a contiguous portion of the ring. In practice, it has assigned a sub-range. A worker is in charge to handle all the requests belonging to its sub-range. Handle a request means two things:
- process the request and provide a response, or
- forward the request to another service that actually knows how to serve it
Every application is build with this behaviour embedded. There is the logic to handle a request or just forward it to another service that can handle it. The forwarding mechanism is nothing more than a remote call procedure, which is actually made using TChannel, the Uber's high performance forwarding for general RPC.
If you think on this, you can figure out that Ringpop is actually offering a very nice thing that traditionals SOA architecture do not have. The clients don't need to know or care about the correct instance that can serve their request. They can just send a request anywhere in Ringpop, and the receiver worker will serve it or forward to the rigth owner.
Ringpop has another interesting feature. New workers can dinamically enter the ring and old workers can leave the ring (e.g. because a crash or just a shutdown) without any service interrputions.
Ringpop implements a membership protocol based on SWIM.
It enable workers to discover each another and exclude a broken worker from the ring using a tcp-based gossip protocol. When a new worker is discovered by another worker, a new connection is established between them. Every worker map the status of the other workers sending a ping request at regular time intervals, and spread the status information with the other workers if a ping does not get a reply (e.g. piggyback membership update on a ping / gossip based)
These 3 elements consistent hashing, request forwarding and a membership protocol, make Ringpop an interesting solution to promote scalability and fault tolerance at application layer while keeping the complexity and operational overhead to a minimum.
I have a j2ee webapp that's being used internally by ~20-30 people.
There is no chance of significant growth in the number of users.
From what I understood there's a trade-off between opening a new DB connection for each request made to the webapp (expensive, but doesn't block other users when the DB is in use), to using the singleton pattern (doesn't open new connections but only allows one user at a time).
I thought that since I know that only 30 users will ever use my webapp at the same time, maybe the simplest and best solution would be to store the connection as a session attribute, thus reducing to a minimum the amount of openings made, while still allocating one connection per user.
What do you think?
From what I understood there's a
trade-off between opening a new DB
connection for each request made to
the webapp
That is what connection pools are for. If you use a connection pool in your application, the pool once initialized, is in charge of providing connections for use in the application as and when needed. In a properly tuned connection pool, there are going to be enough connections created on reserve that can be provided to the application, mitigating the need to create and open a connection only when the application requests for it.
I thought that since I know that only
30 users will ever use my webapp at
the same time, maybe the simplest and
best solution would be to store the
connection as a session attribute
Per-user connections are not a good idea, primarily when a web application is concerned. In a web application, it is perfectly possible for users to initiate multiple requests to the server (think multi-tabbed browsing). In such a case, the use of a single connection per user will result in weird application behavior, unless you synchronize access to the connection.
One must also consider the side-effect of putting transient attributes into the session - Connection objects are not serializable and hence must be marked transient. If the session is deserialized at some point, one has to account for the fact that the Connection object will not be available, and must be re-initialized.
I think you're getting into premature optimization especially given the scale of the application. Opening a new connection is not that expensive and like Makach says, most modern RDBMSs handle connection pooling and will hold connections open for subsequent requests. You'd be trying to write better code than the compiler, so to speak.
No. Don't do that. It's perfectly ok to reconnect to the database every time you need to. Any database management system will do their own connection pool caching I think.
If you want to try to keep open connections you'll make it incredible hard for yourself to manage this in a secure, bug-free, safe etc way.