I was wondering what is maxPoolSize for? what is minPoolSize for?
How do I know which property value to use for my database?
EDITED
I am using Oracle 10g, connecting with Hibernate and bitronix on windows OS.
minPoolSize is the minimum number of database connections being opened all the time, even when none of them are used. maxPoolSize on the other hand represents the maximum number of concurrent connection. Now:
Use minPoolSize to always keep few connections opened (even when idle) so that you don't have to wait for establishing a new network connection when request arrives to the system with low load. This is basically a set of connections waiting "for emergency"
maxPoolSize is much more important. If the system is under heavy load and some request tries to open maxPoolSize + 1 connection, the connection pool will refuse, in turns causing the whole request to be discarded. On the other hand setting this parameter to to high value shift the bottleneck from your application to the database server, as it has limited capacity.
Related
The docs describe avg_wait_time as:
Time spent by clients waiting for a server in microseconds (average per second).
We see occasional spikes in avg_wait_time (normally it's 0). During those spikes, best I can tell, there are available/idle servers. What could be causing the wait time to be greater than 0 in these cases?
Reading the flow from hackernoon leads to that your connection pools are exhausted, and the new connections need to wait until a free spot becomes available for either connecting to a pool, either to get to the executing phase.
These server connections, that clients get linked to, are
“pooled” — limited in number and reused. Because of that it might
occur that while client sends some request (beginning a transaction or
performing a query) a corresponding server connection pool is
exhausted, i.e. pgbouncer oppened as many connections as were allowed
it and all of them are occupied by (linked to) some other clients.
PgBouncer in this scenario puts client into a queue, and this client’s
connection goes to a CL_WAITING state. This might happen as well while
client only logging in, so there’s CL_WAITING_LOGIN for that also:
I am using version 2.3.7 for Java 6. I have set maximumPoolSize to 200 and connectionTimeout to 30 s. I run into SQLTimeoutExceptions from BaseHikariPool.getConnection in one of our load test cases involving 70 simultaneous users uploading 10 files each. I turned on debug logging and obtained pool stats. So it would seem that the pool isn't being exhausted. Rather, HikariCP takes longer than connectionTimeout to create new connections. How can I debug this part of the process? The underlying data source is SQLServerDataSource version 4.1.
connectionTimeout is the maximum time to wait for obtaining connection from pool.
it is NOT time out for creating connection from data source. there is none.
You may want to consider reducing pool size. begin load testing with minimum and gradually increasing till SqlServer begins to take much longer to create connection.
check about pool size
HTH
It might be because in HikariCP opening a connection is a blocking call (
https://github.com/brettwooldridge/HikariCP/issues/1287)
You might find this option useful com.zaxxer.hikari.blockUntilFilled. With this option connection pool will open minimumIdle connections in parallel during initialization instead of lazy initializing connections.
I was wondering if anyone knows how to set up connection pooling for Sitecore 6 running on SQLServer 2005 ?
And is this a good idea to setup on a Sitecore solution? Or probably more correct, will there be any problems if setup incorrectly?
Any other comments or tips about this are also greatly appreciated.
The answers from Sitecores support:
By default, connection pooling is
enabled in ADO.NET (it means that we
don't need any connection string
parameters to enable it, parameters
are needed only to disable or to tune
the connection pooling). Thats why
there are no connection pooling
parameters in the default connection
strings. For more information please
refer to the following MSDN articles:
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.connectionstring.aspx
http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
If you wish to tune the Database
Connection Pooling, you can add
appropriate parameters to the
connection strings.
Second reply after I asked some more about it:
Please see
http://msdn.microsoft.com/en-us/library/8xx3tyca%28v=vs.80%29.aspx
When a connection is first opened, a
connection pool is created based on an
exact matching algorithm that
associates the pool with the
connection string in the connection.
If MinPoolSize is either not specified
in the connection string or is
specified as zero, the connections in
the pool will be closed after a period
of inactivity. However, if the
specified MinPoolSize is greater than
zero, the connection pool is not
destroyed until the AppDomain is
unloaded and the process ends.
Maintenance of inactive or empty pools
involves minimal system overhead.
Following connection strings
parameters can be applied.
# Max Pool Size - specifies the
maximum size of your connection pool.
Default is 100. Most Web sites do not
use more than 40 connections under the
heaviest load but it depends on how
long your database operations take to
complete. # Min Pool Size - initial
number of connections that will be
added to the pool upon its creation.
Default is zero; however, you may
chose to set this to a small number
such as 5 if your application needs
consistent response times even after
it was idle for hours. In this case
the first user requests won't have to
wait for those database connections to
establish. # Pooling - controls if
your connection pooling on or off.
Default as you may've guessed is true.
I hope this helps others.
The only mentioning about negative impacts I'm aware about is this.
For some time now our flagship application has been having mysterious errors. The error message is the generic
[DBNETLIB][ConnectionWrite (send()).]General network error. Check your network documentation.
This is reliably reproduced by leaving the app open for the night and resuming work in the morning. Since it's a backend server app this is a normal scenario.
The funny thing is - we've migrated from SQL Server 7 to 2000 to 2008 and the issue is present on all of them. But what seems to matter is the OS on which we run the app. On WinXP it works fine, on Vista/7 it fails. So the problem is at the client end.
The results of Google on the error message cover a very wide spectrum of different causes (since this is a very generic error) and none of the scenarios found there are similar to ours.
So perhaps someone around here will know what the problem is in our case?
You should be able to reproduce this error condition on demand by:
1. Opening a database connection (in your client application)
2. Unplugging the network cable
3. Plugging network cable back in (wait until the network connection is restored)
4. Using the previously opened connection to query the database
As far as I can tell from experience, client side ADO code is not able to consistently determine if an underlying network connection is actually valid or not. Checking if the database connection is open (in the client code) returns true. However, performing any operations on that connection results in a General network error.
The connection pool appears to be able to determine when a connection goes 'bad' so it never returns a bad connection to the application. It simply opens a new connection instead.
So, if a database connection is kept alive for a long time (used or unused) by the application, the underlying TCP/IP connectivity can get broken.
The bottom line is that database connections should be closed and returned back to the connection pool when not in use.
Edit
Also, depending on the number of clients connecting to the db, not using the connection pool can cause another issue. You may hit the maximum number of sockets open on the server side. This is from memory. Once a connection is closed on the client side, the connection on the server goes into a TIME_WAIT state. By default, the server socket takes about 4 minutes to close, so it is not available to other clients during that time. The bottom line is that there is a limited number of available sockets on the server. Keeping too many connections open can create a problem.
One project I worked on easily hit this socket limit with around 120 users. A new 'feature' was added that absolutely hammered the server, and after a few hours of using the app, things would suddenly slow to a crawl for everyone. SQL server was not closing enough sockets in time for new connection requests. Although there are 65K sockets altogether, only the first 5000 are made available to the ADO (this is a default registry setting thing, so can be changed).
The number of sockets in TIME_WAIT state would slowly build up until the OS would not allocate any more. So clients had to wait until server side sockets closed and a new connection could then be created.
Have you tried disabling SNP/TCP Chimneying?
Had a similar error. For me it was indirectly caused by mismatched calls to WSACleanup and WSAStartup.
The program called WSACleanup more times than WSAStartup. This would cause a reference counter (somewhere in the sockets library) to reach zero too early.
I think effectively from that moment on all sockets owned by the process are broken.
And this would also kill the SQL client since it uses sockets to 'talk' to the SQL server as well.
In a asp.net/Sqlserver project, we create connections using ado.net (sql authentication) and we see a behaviour where there are a lot of active connections in "sleeping","Awaiting command" status
The code does the following - Get a connection from common function, update db, Commit transaction, close & dispose transaction, close connection.
1) In sqlserver 2008 when our program runs for sometime (it updates the db every few secs), the number of active connections increases dramatically and sqlserver starts refusing new connections (as the default connections is 100)
2) In sqlserver 2005, we see that the connections are getting reused and work fine. Our max connections does not go above 15-20.
We found an issue from MSFT on 2008 and conveyed to the client.
https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=383517 - Talks about 2008 not releasing closed connections immediately.
In the client place, we see the same issue in sql2005 too.
My question, is when the .net program calls close() on a connection, how long does sqlserver keeps it active ?
Thanks a lot for any hints
Regards
Anand
Connections are going to the pool. If they are not reused from there, and the number of connections increases, you certainly did not clean them up properly. Make use of using blocks for any disposable type (also transactions, commands and whatever).
To clear the connection pool you can call this static method:
SqlConnection.ClearAllPools();
This should remove all connections and are not used in the pool. The others are still in use.