I am upgrading an MS Access application to work with SQL server as the back-end.
Wrote and test ran a class for establishing/managing a connection to the server and retrieving a recordset with the requested data from the server (table, view, procedure or query), besides the connection management the retrieved recordset gets available as a property of the class.
However, some of my form's procedures open more than one recordset/table while executing.
I am wondering what's best:
opening a separate instance of my connectclass for each needed recordset i.e. a separate connection per opened recordset
or modifying my ConnectClass to handle multiple recordsets/commands on the same connection i.e. within the same class.
Seems to me one class instance per recordset is the cleanest (with less risk of interference and bugs) but at the expense of one open connection per open recordset and some more memory usage, are there other consequences I need to take into account? What's your advice?
Andre
modifying my ConnectClass to handle multiple recordsets/commands on the same connection i.e. within the same class.
This. Generally you want to open a connection, perform a single "unit of work", which may be any number of separate queries and then close the connection.
In a web application a "unit of work" is typically scoped to a single web request, but in a desktop client application (like Access) the connection can be scoped to either a single method, or the lifetime of a UI element, like a form.
Related
Currently we have our Asp mvc LOB web application talking to an SQL server database. This is setup through the a connection string in the web.config as usual.
We are having performance issues with some of our bigger customers that are running some really large reports and kpi's on the database which choke it up and cause performance issues for the rest of the users.
Our solution so far is to setup replication on the database and pass all the report and kpi data calls off to the replicated server and leave the main server for the common critical use.
Without having add another connection string to the config for the replicated server and go through the application and direct the report, kpi and other read only calls to the secondary db is there a way I can point the web.config connection string to an intermediary node that will analyse the data request and shuffle it off to the appropriate db accordingly? i.e. If the data call is a standard update process on the db it will shuffle that to the main db and if there is a report being loaded it will pass it off to the secondary replicated server.
We will only need to add this node in for the bigger customers with larger db's, so if we can get away with adding a node outside the current application setup it will save us a lot of code changes and testing needed.
Thanks in advance
I would say it may be easier for you to add a second connection string for reports, etc. instead of trying to analyse the request.
The reasons are as follows:
You probably have a fairly good idea which areas of your system need to go the second database. Once you identify them, you can just point them to to the second database and not worry about switching them back and forth.
You can just create 2 connection string in you config file. If you have only one database for smaller customers, you can point both connections to the same one database. For bigger customers, you can use two different connection strings. This way you will make the system flexible and configurable.
Analysing requests usually turns out to be complex and adding this additional complexity seems unwarranted in this case.
All my comments are based on what you wrote above and may not be absolutely valid - you know they system better, just use them if you want.
I have come across an issue that seems to be somehow connected to a web server configuration, and resulting in queries randomly taking a long time to execute. The application is created using old plain Classic ASP and ADODB Connection is used.
The scenario goes as follows:
there is a single connection opened in a script at the beginning of processing each HTTP request
this connection is used to execute a query against a SQL Server, that resides on a separate box. conn.Execute is used. Connection is NOT closed afterwards
there are usually a few to a few dozens of conn.Execute in a single ASP page
All has been working well until recently, when some of the conn.Execute started to take much longer to execute, totally on random.
the difference is e.g. 15ms normal execution time vs. 2000ms long execution time
on the SQL Server side, Profiler does not show longer query execution times, so there must be something blocking the conn.Execute request
When a proper practice of closing a connection after each conn.Execute has been implemented, the issue goes away. However, as I have stated before, all has been working flawlessly until recently. This web app is a fairly large one and rewriting it to close and reopen connections properly will take some time. And I need a short-term solution.
My guess is that it could have something to do with the connection pool size, however this is not ADO.NET, therefore I am not sure, whether a connection pool issue should be taken into the consideration at all. On the SQL Server side, there is no limit on the number of concurrent connections to the server.
I need some hints. Brainstorming possible ideas.
Could be related to delays resolving the hostname in the connection string via DNS - have you tried putting an IP address in the connection string instead of the hostname?
My question is in several parts and could go long, so sorry about that.
I am using LINQ to pass on data across to Silverlight client.
Part 1: Connection pooling & optimization:
I want to know how can I ensure that LINQ establishes only one connection to database and also follow connection pooling. In the past, I used to put the connection pooling property in the SQL Connection string itself. So, if I put this setting file, where the connection string is set when I add tables in DBML file, will LINQ respect that
LINQ (DBML) forms core of my data access layer. I write classes that has methods which are basically LINQ queries.
I use the DataContext object in "using" block in my code where I handle data related part.
Is this the reason why LINQ might be using multiple connection? If I have have DataContext as a class level variable, will this ensure only one connection?
Part 2: Optimization
In not so distant past, ADO.Net days, I used to write stored procedures, execute it via a DataReader and then loop through the DataReader, fill as many objects of my Model class and pass on that collection for binding to DataGrid.
In LINQ days, I do more or less same as far creating a collection of objects is concerned. But I execute direct LINQ statements itself.
I can guess that SQL Stored procedures will give me faster performance, but if I execute Stored procedures using LINQ will it be just as fast as the older days and is it a right approach?
Ohoh.... so many bad habits.
Part 2: Optimization In not so distant
past, ADO.Net days, I used to write
stored procedures, execute it via a
DataReader and then loop through the
DataReader, fill as many objects of my
Model class and pass on that
collection for binding to DataGrid. In
LINQ days, I do more or less same as
far creating a collection of objects
is concerned. But I execute direct
LINQ statements itself. I can guess
that SQL Stored procedures will give
me faster performance, but
Basically, in the past you had some minor dlusions. Stored procedures have NO (!) performance gain for more than 10 years now. It is clearly spelled out in the documentation. SQL execution is as fast as for non stored procedures, query plans are cached and reused for both.
SP's are only good (as in: saving time) if they avoid round trips (i.e. sending multiple batches of requests from the client oto the server). And then the savings are not due to the fact that they are storped procedures, but due to the fact that round trips cost time.
Sadly, many programmers still have delusions because they get them from other people who got them.... 10 years ago when stored procedures had an intrinsic advantages. Now that this is SQL Server 6.5 time - since 7.0 this is history.
http://weblogs.asp.net/fbouma/archive/2003/11/18/38178.aspx
Basically you treated lost development time and lots of useless code against.... mostlikely no measurable advantage.
Part 1: Connection pooling &
optimization: I want to know how can I
ensure that LINQ establishes only one
connection to database and also follow
connection pooling.
You dont. Basically dont try to be smarter than is good for you. What is wrong with multiple connections IF THEY MAKE SENSE?
For pooling, yu should (sql server) not to have to put anything in the connection string normally. And yes, LINQ will not magically bypass the connection pool defined in the connection string. See, LINQ does not EVER talk to the database - it uses ADO.NET for this, and ADO.NET has not magically changed behavior just because some higher level ORM is using it instead of you. The connection string has pooling entries, ADO.NET still sees them and follows them.
Now, having only one database connection from a server pool is one thing: STUPID. It limits ou to one transaction at a time and totally destroys performance once the load gets higher (i.e. multiple requests need to be handled at the same time).
I use the DataContext object in
"using" block in my code where I
handle data related part. Is this the
reason why LINQ might be using
multiple connection? If I have have
DataContext as a class level variable,
will this ensure only one connection?
Ah - depends. It may make sense, it may not. You know, before thinking you have a problem, especially given the hugh amount of information you give here (i.e. none), do the ONLY sensible thing: grab a profiler and MEASURE whether you have one. Openng / closing a connection 100 times or 1000 times most likely will not even show up on the profiler.No problem = no reason to fix something.
That said, I dont like a per method connection opening - it normally shows a bad class design. Connections should be reused within a unit of work.
I understand that a typical .NET application that accesses a(n SQL Server) database doesn't have to do anything in particular in order to benefit from the connection pooling. Even if an application repeatedly opens and closes database connections, they do get pooled by the framework (assuming that things such as credentials do not change from call to call).
My usage scenario seems to be a bit different. When my service gets instantiated, it opens a database connection once, does some work, closes the connection and returns the result. Then it gets torn down by the WCF, and the next incoming call creates a new instance of the service.
In other words, my service gets instantiated per client call, as in [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]. The service accesses an SQL Server 2008 database. I'm using .NET framework 3.5 SP1.
Does the connection pooling still work in this scenario, or I need to roll my own connection pool in form of a singleton or by some other means (IInstanceContextProvider?). I would rather avoid reinventing the wheel, if possible.
Typical WCF application that accesses a(n SQL Server) database doesn't have to do anything in particular in order to benefit from the connection pooling. Even if an application repeatedly opens and closes database connections, they do get pooled by the framework (assuming that things such as credentials do not change from call to call).
The service instancing model creates and tears down an instance of your class, not an entire appdomain. The SqlClient connection pool is per AppDomain, so you'll get your free lunch.
Even though this is an old post, I feel it is important to add to it.
ADO.NET database connection pooling does NOT work in per-call WCF services, if you follow the typical scenario (instantiating ADO.NET objects within the service object).
While I do understand the above theory and arguments, they are just that: theory.
A simple Windows form application which goes through the step of open, query, close multiple times will show you that the first Open() call takes quite long such as 2 or 3 seconds, and subsequent calls and queries are fast - the effect of connection pooling.
If you put identical code into a per-call WCF service, you get the 2-3 seconds delay on EVERY CALL, the first call and all subsequent calls.
Conclusion - ADO.NET database connection pooling does NOT work in per-call WCF services if you do the typical ADO instantiation within the service.
You would have to instantiate ADO objects in a custom service host, and add appropriate synchronization code if you need, or live with no database connection pooling.
Consider a classic ASP site running on IIS6 with a dedicated SQL Server 2008 backend...
Scenario 1:
Open Connection
Do 15 queries, updates etc all through the ASP-page
Close Connection
Scenario 2:
For each query, update etc, open and close the connection
With connection pooling, my money would be on scenario 2 being the most effective and scalable.
Would I be correct in that assumption?
Edit: More information
This is database operations spread over a lot of asp-code in separate functions, doing separate things etc. It is not 15 queries done in rapid succession. Think a big site with many functions, includes etc.
Fundamentally, ASP pages are synchronous. So why not open a connection once per page load, and close it once per page load? All other opens/closes seem to be unnecessary.
If I understand you correctly you are considering sharing a connection object across complex code held in various functions in various includes.
In such a scenario this would be a bad idea. It becomes difficult to guarantee the correct state and settings on the connection if other code may have seen the need to modify them. Also you may at times have code that fetches a firehose recordset and hasn't finished processing when another piece of code is invoked that also needs a connection. In such a case you could not share a connection.
Having each atomic chunk of code acquire its own connection would be better. The connection would be in a clean known state. Multiple connections when necessary can operate in parrallel. As others have pointed out the cost of connection creation is almost entirely mitigated by the underlying connection pooling.
in your Scenario 2, there is a round-trip between your application and SQLServer for executing each query which consumes your server's resources and time of total executions will raise.
but in Scenario 1, there is only one round-trip and also SQLServer will run all of the queries in just one time. so it is faster and less resource-consuming
EDIT: well, I thought you mean multiple queries in one time..
so, with connection pooling enabled, there is exactly no problem in closing connection after each transaction. so go with Scenario 2
Best practice is to open the connection once, read all your data and close the connection as soon as possible. AFTER you've closed the connection, you can do what you like with the data you retrieved. In this scenario, you don't open too many connections and you don't open the connection for too long.
Even though your code has database calls in several places, the overhead of creating the connection will make things worse than waiting - unless you're saying your page takes many seconds to create on the server side? Usually, even without controlled data access and with many functions, your page should be well under a second to generate on the server.
I believe the default connection pool is about 20 connections but SQLServer can handle alot more. Getting a connection from the server takes the longest time (assuming you are not doing anything daft with your commands) so I see nothing wrong with getting a connection per page and killing it if used afterwards.
For scalability you could run into problems where your connection pool gets too busy and time outs while your script waits for a connection to be come available while your DB is sat there with a 100 spare connections but no one using them.
Create and kill on the same page gets my vote.
From a performance point of view there is no notable difference. ADODB connection pooling manages the actual connections with the db. Adodb.connection .open and .close are just a façade to the connection pool. Instantiating either 1 or 15 adodb.connection objects doesn't really matter performance wise. Before we where using transactions we used the connection string in combination with adodb.command (.activeConnection) and never opened or closed connections explicitly.
Reasons to explicitly keep reference to a adodb.connection are transactions or connection-based functions like mysql last_inserted_id(). In these cases you must be absolutely certain that you are getting the same connection for every query.