I read this question on relating to disposing of SQL connections.
My question is, how bad is it to simply close a sql connection, but not dispose of it?
We have a function which is simply closed, but never disposed, and it is used 1000s times a day.
Is it better to simply close it, or would it be better to close and dispose of it?
I am aware that dispose() also closes the connection, however I would like to know why close doesn't dispose of the connection.
The important thing about connections is to close them so they are returned to the connection pool.
As such, there is little difference between disposing and closing a connection, so long as you are disciplined about closing and not reusing connections.
However, being in the habit of wrapping the creation of a connection in a using statement means you never forget to close it.
It is a general good idiom to follow - creation of any object that implements IDisposable should be wrapped in a using statement, and as such an idiom it is a good one to follow with connections as well.
Depends on whether it goes out of scope or not.
If it does it will be closed anyway, and returned the connection pool (given you haven't disabled that).
So calling close implicitly or explicitly as you are doing makes your intent clear and makes that connection "instantly" available for reuse in the pool.
Idea was to persuade developers to get in and out of the db quick. Lots of small transactions. Not the old style open up a connection and then hide it so no one else could get at it, just in case it was needed again.
If Connection pooling is on the a using clause is equivalent to Create, open, Close.
If it's off then a using clause is equivalent to create, open,close,Dispose.
In either scenario, the real deal is to make sure it goes out of scope. Apart from very rare circumstances the connection should be a local reference and have a lifetime of this particular use. You wouldn't generally instantiate one at run time and make it a property of your mainform for intance.
Related
my question seems very simple, but there are some subquestions which requires deeper inspection.
My Question:
What's the best practice/architecture for handle the database connection?
My found options:
For each rest ful service with database(DB) requests create a new
connection to the DB and close this again after the queries.
Create a connection outside the REST Service and use this for each
query.
Option 1.:
One negativ point of this are the costs for etablish and close the connection for each request.
Option 2.:
Don't know whether it's work. I've researched for the Web Service Lifecycle for checking how this could be work, but don't know whether the instances will stay alive after the finish of the Web Service. Also don't know whether it's a good practice because there could be events which destroy the connection. A last issue is I think the requests could be block each other (So it destroys the concept of threads).
Hope you could help me a little bit with this architecture.
Greets,
Nik
If you create one per query / transaction, it is much easier to manage "closing" the connections.
I can see why common sense dictates that you should open one and use it throughout, but you will run into problems with dropped connections and multithreading. So your next step will be to open a pool, say of 50, connections and keep them all open, doling them out to different processes.
If you open a connection when you need it and dispose of it when you've finished, that will not actually close the connection, it'll just return it to the connection pool to be used again.
I am getting the following exception in the website which has to a large extent deal with data entry operations. It also has Indexes defined on the tables in the associated database. The database calls are made via SQLHelper. For e.g. SQLHelper.ExecuteNonQuery() etc. I cannot see anywhere that the Close() or Dispose() method of the SQLConnection is invoked. So I am assuming that SQLHelper must be taking care of it as I have also read about it on various sites. Also, to check the code in combination with Close() or Dispose() is also very tedious as SQLHelper is used in many places and there are many classes where business logic is defined. The exception that I am getting is:
The record was not updated Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
For now I have tested the code with putting GC.Collect in the Application_EndRequest Method of Global.asax and everything as of now is working fine. But I know that it is strictly not recommend to use the same.
Any help will be GREATLY appreciated as I am stuck # present..
Not sure which version of your SQLHelper is, but if you can't see any connection.Close() been called then you need manually call it to make sure connection closed. Garbage collector will not close the connection for you.
EDIT
Also about the connection pooling it's by default enabled by .Net itself, you call connection.Close() does not mean the connection between your application to SQL Server really been closed, it's just return that connection back to connection pool and wait for others to grab. Only if after a while nobody open new connection that connection them will be physically closed, so you should not need to worry about call connection.Close() too many times, instead of that you need to call it asap to release the resource for other threads to use.
For more detail please check how Microsoft saying about connection pool: http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
ANOTHER EDIT
I suggest you find an updated version of SQLHelper or go ahead change SQLHelper to add Close() in it. Even if you found GC help you closed the connection but you should not use it that way, GC is not designed for release database connection but just memory, also GC.Collect() is not guaranteed it will immediately do it right away to start garbage collection.
Also you are coding web application so you need to think concurrency, what if when you calling GC.Collect() when another thread is running, will that slow down your system to other users?
It's a common sense that those limited resources (like db connection, TCP/IP port, file read/write handler etc)need to be released as soon as possible. If you are looking for an easy way to make you coding easily without even using connection.Close then you are going to a wrong direction, I understand you want simply you code and not add that line everywhere but you at least need to make sure SQLHelper doing the job to close connection.
A commonly problem I see in lots of web languages is that database connections need to be closed otherwise the number of total connections gradually increases and then it grinds to a halt in whatever form.
HTTP is stateless, when the request has finished processing why can't these languages just drop any connections that request opened? Are there any legitimate reasons for why you might keep it open?
Because the cost of opening, authenticating and authorising access to a database is quite expensive. That is why normally everybody uses a databases connection pool. Connections are still open while request handlers pick up a available-already-opened connection from a pool. When one closes a connection what is really happening is that the connection is being freed for others to use.
To answer ...
why can't these languages just drop
any connections that request opened?
Are there any legitimate reasons for
why you might keep it open?
Connections might stay opened after the request is complete and use for other purposes. For instance asynchronous updates of data. But I am with you, in 90% of the cases when the request is finished the connections opened should be returned back to the pool. Depending on the Web Framework you use (Spring, DJANGO, ...) this kind of behaviour can be configured or at least implemented with minimum effort.
Checking for an open connection while closing an http connection gives more overhead so I guess that's why some languages don't close it by default.
And if you don't close it explicitly, it will have to be done by the garbage collector which can take a while.
I know this has been asked before here there and everywhere but i can't get a clear explanation so i'm going to pitch it again. So what is all of the fuss about using a singleton to control the db connection in your web app? Some like it some hate it i don't understand it. From what I've read, "it's to ensure that there is always only one active connection to your DB". I mean why is that a good thing? 1 active DB connection on a data driven web app processing multiple requests per second spells trouble doesn't it? For whatever reason nobody can properly explain this. I've been all over the web. I know i'm thick.
Assuming Java here, but is relevant to most other technologies as well.
I'm not sure whether you've confused the use of a plain singleton with a service locator. Both of them are design patterns. The service locator pattern is used by applications to ensure that there is a single class entrusted with the responsibility of obtaining and providing access to databases, files, JMS queues, etc.
Most service locators are implemented as singletons, since there is no need for multiple service locators to do the same job. Besides, it is useful to cache information obtained from the first lookup that can be later used by other clients of the service locator.
By the way, the argument about
"it's to ensure that there is always
only one active connection to your DB"
is false and misleading. It is quite possible that the connection can be closed/reclaimed if left inactive for quite a long period of time. So caching a connection to the database is frowned upon. There is one deviation from this argument; "re-using" the connection obtained from the connection pool is encouraged as long as you do so with the same context, i.e. within the same HTTP request, or user request (whichever is applicable). This done obviously, from the point of view of performance, since establishing new connections can prove to be an expensive operation.
High-performance (or even medium-performance) web apps use database connection pooling, so one DB connection can be shared among many web requests. The singleton is usually the object which manages this pool. I think the motivation for using a singleton is to idiot-proof against maintenance programmers that might otherwise instantiate many of these objects needlessly.
"it's to ensure that there is always only one active connection to your DB." I think that would be better stated as to ensure each CLIENT has only one active connection to your DB. The reason why this is incredibly important is because you want to prevent deadlocks. If I have TWO open database connections (as a client) I might be updating on one connection, then I might try to update the same row in another connection. This will a deadlock which the database cannot detect. So, the idea of the singleton is basically to make sure that there is ONE object who is charge of handing out database connections to each client. Basically. You don't HAVE to have a singleton for this, but most people will tell you it just makes sense that the system only has one.
You're right--usually this isn't what you want.
However, there are plenty of cases where you need to throttle yourself down to a single connection. By serializing your access to the database through a singleton, you can address other issues or constraints like load, bandwidth, etc.
I've done something similar in the past for a bulk processing app. Instead, though, I used a semaphore to synchronize access to the database so I could allow n concurrent db operations.
One might want to use a singleton due to database server constraints, for example, a server might limit the number of connections.
My main conscious reason is that you know what connections can be managed/closed etc., just makes things a bit more organised when you don't have unnecessary, redundant connections.
I don't think it's a simple answer. For instance on ASP.NET, the platform implements connection pooling by default, so it will automatically adjust a "pool" of connections and re-use them so you're not constantly creating and destroying expensive objects.
However, let's say you were writing a data collection application that monitored 200 separate input sources. Every time one of those inputs changed, you fire off a thread that records the event to the database. I would say that could be a bad design if there's a chance that even a fraction of those could fire off at the same time. Suddenly having 20 or 40 active database connections is inefficient. It might be better to queue the updates, and as long as there are updates left in the queue, a singleton connection picks them off the queue and executes them on the server. It's more efficient because you only have to negotiate the connection and authentication once. Once there's no activity for a while you could choose to close down the connection. This kind of behavior would be hard to implement without a central resource manager like a singleton.
"only one active connection" is a very narrow statement for illustration. It could just as well be a singleton managing a pool of connection. The point of a singleton for database connections is that you don't want every consumer making it's own connection or set of connections.
I think you might want to be more specific about, "using a singleton to control the db connection in your web app." Ideally, a java.sql.Connection object will not be thread safe, but your javax.sql.DataSource may want to pool connections, so you should go to a single instance of it to share the pooling.
you are more looking for one connection per request, not one connection for the entire application. you can still control access to it through a singleton though (storing the connection in the HttpContext.Items collection).
It guarantees that each client using your site only gets one connection to the db.
You really do not want a new connection being made everytime a user does an action that will create a db query. Not only for performance reasons with the connection handshaking involved, but to decrease load on the db server.
DB connections are a precious commodity, and this technique helps minimize the amount used at any given time.
Im currently creating a webportal with ASP.NET which relies heavily on database usage. Basically, every (well almost every :P ) GET query from any user will result in a query to the database from the webserver.
Now, I'm really new at this, and I'm very concerned about performance. Due to my lack of experience in this area, I don't really know what to expect.
My question is, using ADO.NET, would it be a smarter choice to just leave a static connection open from the webserver to the database, and then check the integrety of this connection serverside before each query to the database? - Or, would I be better off opening the connection before each query and then close it afterwards?
In my head the first option would be the better as you save time handshaking etc. before each query and you save memory both on the database and the server side since you only have one connection, but are there any downfalls to this approach? Could 2 queries send at the same time potentially destroy each others integrity or mix the returned dataset?
I've tried searching everywhere in here and on the web to find some best-practices about this, but with no luck. Closest I got was this: is it safe to keep database connections open for long time , but that seems to be more fitting for distributed systems where you have more than one user of the database, whereas I only got my webserver..
You're way early to be worrying about performance.
Anyhow, connections are pooled by the framework. You should be opening them, using them, and disposing of them ASAP.
Something like...
public object Load()
{
using (SqlConnection cn = new SqlConnection(connectionString))
using (SqlCommand cm = new SqlCommand(commandString, cn))
{
cn.Open();
return cm.ExecuteScalar();
}
}
It's better to let ADO.NET handle the connection pooling. It'll persist the connection if it thinks it needs to, but don't use a static connection object. That just smells. It would be better to pass the connection object around to methods that need it, and create the connection in a using block.
You should always close your connection after finishing your DB interaction. ADO.NET has connection pooling which will take care of efficient connection reuse. Whenever you open 2nd, 3rd and subsequent connections - they'll be taken from a pool with almost no overhead.
Hope this helps.
I'd be thinking more about caching than advanced connection pooling. Every get requires a database hit?
If its a portal you've got common content and user specific content, using the Cache you can store common items as well as with a mangled key (with the users id) you can store user specific items.
ADO.NET does connection pooling. When you call close on the connection object it will keep the connection in the pool making the next connection much faster.
Your initial hunch is correct. What you need is database connection pooling.
You definitely don't want to open a connection for every database call, that will result in extremely poor performance very quickly. Establishing a database connecting is very expensive.
Instead what you should be using is a connection pool. The pool will manage your connections, and try to re-use existing connections when possible.
I don't know your platform, but look into Connection Pooling - there must be a library or utility available (either in the base system or as an add-on, or supplied with the database drivers) that will provide the means to pool several active connections to the database, which are ready and raring to be used when you obtain one from the pool.
To be honest, I would expect the pooling to occur by default in any database abstraction library (with an available option to disable it). It appears that ADO.NET does this.
Really the first question to ask is why are you very concerned about performance? What is your expected workload? Have you tried it yet?
But in general, yes, it's smarter to have an open connection that you keep around for a while than to re-open a database connection each time; depending on the kind of connection, network issues, and phase of the moon, it can take a good part of a second or more to make an initial connection; if your workload is such that you expect more than a GET every five seconds or so, you'll be happier with a standing connection.