If my connection pooling is set to 10 and 100 users hit a page utilizing connection to DB at almost the same time.
Than in this case will the 90 users have to wait for connections to get free?
OR
More connections would be created for 90 users but they shall not be returned to pool?
FYI: I know connection pooling and related concepts. The query is in relation to page which generate large reports.
They will have to wait for a connection to be returned to the pool if the maximum of 10 has reached, see: http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
The connection pooler satisfies
requests for connections by
reallocating connections as they are
released back into the pool. If the
maximum pool size has been reached and
no usable connection is available, the
request is queued. The pooler then
tries to reclaim any connections until
the time-out is reached (the default
is 15 seconds). If the pooler cannot
satisfy the request before the
connection times out, an exception is
thrown.
Related
I am using a SQL Server database with Nodejs. I am using connection pool to perform various queries. When I run sp_who2, I can see that the there are almost 20 processes which have status sleeping and command awaiting command.
Should I go ahead and delete these processes? I read in some other post that this happens when you create a transaction in SQL Server but do not close / commit / rollback that transaction. I do not see any point in my application where I did not commit or rollback transaction on error. So I am not sure where the error came from.
I have a feeling that leaving those processes there is going to cause query timeout issues in the future. Is there a way to see what query caused the sleeping but waiting state?
I normally see many sleeping connections. I consider it normal. If you have sleeping connections with open transactions and locks, then you need to investigate. I would try to identify the host and PID holding the lock. In some cases the resolution is a polite talk with the person responsible for not closing their transaction.
A connection pool is a pool of connections to SQL Server. They will be idle and sleeping unless they are in use. Generally, there is a timeout for the connections in the pool. (For example, if you look at the ODBC control panel, the connection pooling tab will generally show a 60 second timeout. It might also always keep a minimum number of idle connections.) Check if you have a minimum number of idle connections. Once you know your timeout, verify that the connections are timing out as expected...eventually. If not, I would look for a connection leak or a connection pool issue. Is the application releasing the connection when done? Does GC have to run before the connection goes away?
Years ago there was an issue where a connection could go back into the pool with an open transaction. It was not until the connection was being prepared for reuse that it was finally reset. This issue has been fixed.
Another past issue was a broken connection. For example, if the SQL Server was rebooted, all idle connections are broken. However, it was not until the connection was requested that this was checked. A connection failure timeout was required for each connection in the pool before it was replaced. This was a PITA.
The docs describe avg_wait_time as:
Time spent by clients waiting for a server in microseconds (average per second).
We see occasional spikes in avg_wait_time (normally it's 0). During those spikes, best I can tell, there are available/idle servers. What could be causing the wait time to be greater than 0 in these cases?
Reading the flow from hackernoon leads to that your connection pools are exhausted, and the new connections need to wait until a free spot becomes available for either connecting to a pool, either to get to the executing phase.
These server connections, that clients get linked to, are
“pooled” — limited in number and reused. Because of that it might
occur that while client sends some request (beginning a transaction or
performing a query) a corresponding server connection pool is
exhausted, i.e. pgbouncer oppened as many connections as were allowed
it and all of them are occupied by (linked to) some other clients.
PgBouncer in this scenario puts client into a queue, and this client’s
connection goes to a CL_WAITING state. This might happen as well while
client only logging in, so there’s CL_WAITING_LOGIN for that also:
in c3p0, maxIdleTime means:
maxIdleTime: (Default: 0) Seconds a Connection can remain pooled but unused before being discarded. Zero means idle connections never expire.
I don't get it. Let's say, if there are 10 awaiting connections in the connection pool, and after maxIdleTime has passed, there are still no new database requests, then all these connections should be discarded? then there will be 0 awaiting connections in the pool?
If there are 10 Connections in the pool and no activity, then after maxIdleTime has passed, yes, they will all be expired.
But that does not mean there will be no Connections left in the pool. At the same time as the pool expires old Connections, it will acquire begin new Connections from the DBMS in order to uphold the minPoolSize configuration parameter.
If you have kept a large pool size (i.e. greater than what you anticipate as 99th percentile load), this will mean that you will have lots of idle connections lying around and it is better to clean them up periodically via maxIdleTime and idleConnectionTestPeriod parameter
May be, you will need to add too idleConnectionTestPeriod and preferredTestQuery be sure that connection are valid or not
I am using version 2.3.7 for Java 6. I have set maximumPoolSize to 200 and connectionTimeout to 30 s. I run into SQLTimeoutExceptions from BaseHikariPool.getConnection in one of our load test cases involving 70 simultaneous users uploading 10 files each. I turned on debug logging and obtained pool stats. So it would seem that the pool isn't being exhausted. Rather, HikariCP takes longer than connectionTimeout to create new connections. How can I debug this part of the process? The underlying data source is SQLServerDataSource version 4.1.
connectionTimeout is the maximum time to wait for obtaining connection from pool.
it is NOT time out for creating connection from data source. there is none.
You may want to consider reducing pool size. begin load testing with minimum and gradually increasing till SqlServer begins to take much longer to create connection.
check about pool size
HTH
It might be because in HikariCP opening a connection is a blocking call (
https://github.com/brettwooldridge/HikariCP/issues/1287)
You might find this option useful com.zaxxer.hikari.blockUntilFilled. With this option connection pool will open minimumIdle connections in parallel during initialization instead of lazy initializing connections.
I was wondering if anyone knows how to set up connection pooling for Sitecore 6 running on SQLServer 2005 ?
And is this a good idea to setup on a Sitecore solution? Or probably more correct, will there be any problems if setup incorrectly?
Any other comments or tips about this are also greatly appreciated.
The answers from Sitecores support:
By default, connection pooling is
enabled in ADO.NET (it means that we
don't need any connection string
parameters to enable it, parameters
are needed only to disable or to tune
the connection pooling). Thats why
there are no connection pooling
parameters in the default connection
strings. For more information please
refer to the following MSDN articles:
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.connectionstring.aspx
http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
If you wish to tune the Database
Connection Pooling, you can add
appropriate parameters to the
connection strings.
Second reply after I asked some more about it:
Please see
http://msdn.microsoft.com/en-us/library/8xx3tyca%28v=vs.80%29.aspx
When a connection is first opened, a
connection pool is created based on an
exact matching algorithm that
associates the pool with the
connection string in the connection.
If MinPoolSize is either not specified
in the connection string or is
specified as zero, the connections in
the pool will be closed after a period
of inactivity. However, if the
specified MinPoolSize is greater than
zero, the connection pool is not
destroyed until the AppDomain is
unloaded and the process ends.
Maintenance of inactive or empty pools
involves minimal system overhead.
Following connection strings
parameters can be applied.
# Max Pool Size - specifies the
maximum size of your connection pool.
Default is 100. Most Web sites do not
use more than 40 connections under the
heaviest load but it depends on how
long your database operations take to
complete. # Min Pool Size - initial
number of connections that will be
added to the pool upon its creation.
Default is zero; however, you may
chose to set this to a small number
such as 5 if your application needs
consistent response times even after
it was idle for hours. In this case
the first user requests won't have to
wait for those database connections to
establish. # Pooling - controls if
your connection pooling on or off.
Default as you may've guessed is true.
I hope this helps others.
The only mentioning about negative impacts I'm aware about is this.