Scenario C in This Microsoft Doc describes how temp tables scoped to a connection can be replaced with Memory-Optimized Tables. The scheme uses a Filter Security Policy which calls a function to determine if ##spid matches the SpidFilter column in the Memory-Optimized table.
Will this work with .NET connection pooling? I would expect ##spid will return the same number as a connection is re-used over and over again. .NET clears the session scoped temp tables by calling sp_reset_connection, but that will not clear Memory-Optimized tables, or change ##spid. Maybe sys.dm_exec_sessions's session_id could be added to make it work in a connection pooling environment?
With the help of Microsoft Support, I was able to get the necessary details about ASP.NET Connection Pooling to answer this concern. It is true that ASP.NET threads will share the same SPID, but never at the same time. A thread gets assigned a connection only after the connection is no longer being used by the previous thread. Connection Pooling does not reduce the number of connections needed, it only reduces the number of times connections need to be opened and closed.
This is good documentation on Connection Pooling, although it does not make that distinction. https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql-server-connection-pooling
Notice that Scenario C has special note: "Replace the CREATE TABLE #tempSessionC statements in your code with DELETE FROM dbo.soSessionC, to ensure a session is not exposed to table contents inserted by a previous session with the same session_id." – i-one
Since only one thread will be using the connection at a time, this is sufficient. If the table is not also deleted after being used, it will continue consuming memory (especially precious in Azure) until another thread happens to use a connection with the same SPID.
Related
We use global temporary tables in our program, written in C++ using ATL as a DB interface. Sessions are established with the SQL OLE DB provider.
This global temporary tables are held for a long time, maybe for the complete time of a session. Such temporary tables are explicitly deleted by us, when the specific action/activity ends. So we always clean up the tables.
Now we see an effect on people that are using a slow or unstable VPN connection that the global temporary is deleted. A query that should read some content returns an error
##tblTemp... is not a valid object name
For me it is an indicator that SQL Server terminated the session.
But how can it be? Our program has internal functions that access the server at least every 5 minutes (even if the user is inactive). Usually the SQL Server is accessed much more frequently. But the program may be minimized in the background.
What timeout is responsible that SQL Server terminates a session and deletes the temporary tables?
I see the Remote Query Timeout in the server settings. But this seams to be wrong for me, because we have no open query here... also the queries to the table are real simple. Insert an record, delete an record.
Questions:
Where do I find the settings for this session timeout?
Is there a way for the client to find out that the session got terminated? Strange for me the SQL query itself was transferred to SQL Server and finally failed because the temporary table did no longer exist. We got other error on the client.
Is there a way to protocol this on the server?
EDIT:
Here more details how we work with this tables.
The tables are created in my main thread. This thread has a SQL session that is created at start of the program and ends with the program end.
Other threads use the temporary tables. We pass the names through it.
So due to the fact that the creating SQL session is still alive and doesn't show an error when executing a statement that uses the temporary table, it tells me that the session is still alive. But my problem is the object seams to be deleted.
Again: We only have this problem on machines with a slow / bad VPN connection to the server!
To quote the manual:
Global temporary tables are automatically dropped when the session that created the table ends and all other tasks have stopped referencing them. The association between a task and a table is maintained only for the life of a single Transact-SQL statement. This means that a global temporary table is dropped at the completion of the last Transact-SQL statement that was actively referencing the table when the creating session ended.
source
So it's not about the server being accessed every few minutes, but the specific object being referenced.
If the session that created the global temporary table is ended (e.g. a timeout) and nothing else is actively referencing the same table, it is dropped!
I have two applications.
One inserts data into database continuously like it is having an infinity loop.
When the second application inserts data to same database and table what will happen.
If it waits till the other application to complete inserting which will handle this?
Or it will say it is busy?
Or code throws an exception?
SQL servers have something called a connection pool which means that more than once connection to the database can be made at any particular time, and that's where the easy bit ends.
If you were to for example connect to the database on two applications at the same time and insert data in to different tables from each application then the two could happily happen at the same time without issue.
If however those applications wanted to do something like edit the same row then there's an issue with "locking" ...
Essentially any operation on a SQL database requires "acquiring a lock" on a "set" or "row" or "cell" depending on the configuration of the server its hard to say what might happen in your case.
So the simple answer is:
Yes, SQL can make stuff happen (like inserts) at the same time but with some clauses.
And long answer ...
requires in depth knowledge of locking and your database and server configuration.
I would like to get the opinions on situation I'm facing regarding the connection pools.
I'm SW developer, working on some multitenant application. We've one DB and each tenant (client) has it's own schema. For each connected tenant, the solo process is started and it gets a solo DB connection. In the near future, I would need to use it in 300+ simultaneous tenant environment.
From what I read, using a lot of connections (100+) to postgres is not advised. The one solution is to use the connection pool. The other one, to use more DB servers.
I was thinking about the connection pooler (pgBouncer, pgPool). But in current state of our application, it is a bit of problematic. Here is the list of the "problematic" items and proposed solution:
Single connection to the server for whole lifetime of the process - This is because, we're using the temp tables and prepared statements heavily. The temp tables duration are variable but in most circumstances, they span multiple transactions.
Because connection pooler will return the "free" connection I cannot be sure if the given temp table was created in the returned connection. But I think I could workaround it creating "temp tables" in predefined schema (but then I would need to have some background task to clean the orphaned temp tables from the processes that aren't cleanly closed or crashed - the postgres temp tables are dropped on connection close automatically). For prepared statements I haven't found the workaround.
Use of "set search_path="custom_schema",public;" - This is done on application start for each tenant, so the correct tables are used.
This could be fixed by issuing the set search_path=... command on each transaction. It should be cheap/fast.
Use of triggers that depends on the temp tables - This is used for automatic logging of some stuff. It use the temp tables that are automatically created on the application start.
Don't have the solution yet as cannot use the "custom table approach" mentioned above because of the table name should be constant (multiple tenants would then create the same table more than once, which is bad).
So, I don't know if I should start to think about redesigning or not. If I could just add more DB servers and everything would run, than no need to redesign it.
What do you think?
Thank You
These things are not a problem in pgbouncer with pool_mode = session.
Consider a classic ASP site running on IIS6 with a dedicated SQL Server 2008 backend...
Scenario 1:
Open Connection
Do 15 queries, updates etc all through the ASP-page
Close Connection
Scenario 2:
For each query, update etc, open and close the connection
With connection pooling, my money would be on scenario 2 being the most effective and scalable.
Would I be correct in that assumption?
Edit: More information
This is database operations spread over a lot of asp-code in separate functions, doing separate things etc. It is not 15 queries done in rapid succession. Think a big site with many functions, includes etc.
Fundamentally, ASP pages are synchronous. So why not open a connection once per page load, and close it once per page load? All other opens/closes seem to be unnecessary.
If I understand you correctly you are considering sharing a connection object across complex code held in various functions in various includes.
In such a scenario this would be a bad idea. It becomes difficult to guarantee the correct state and settings on the connection if other code may have seen the need to modify them. Also you may at times have code that fetches a firehose recordset and hasn't finished processing when another piece of code is invoked that also needs a connection. In such a case you could not share a connection.
Having each atomic chunk of code acquire its own connection would be better. The connection would be in a clean known state. Multiple connections when necessary can operate in parrallel. As others have pointed out the cost of connection creation is almost entirely mitigated by the underlying connection pooling.
in your Scenario 2, there is a round-trip between your application and SQLServer for executing each query which consumes your server's resources and time of total executions will raise.
but in Scenario 1, there is only one round-trip and also SQLServer will run all of the queries in just one time. so it is faster and less resource-consuming
EDIT: well, I thought you mean multiple queries in one time..
so, with connection pooling enabled, there is exactly no problem in closing connection after each transaction. so go with Scenario 2
Best practice is to open the connection once, read all your data and close the connection as soon as possible. AFTER you've closed the connection, you can do what you like with the data you retrieved. In this scenario, you don't open too many connections and you don't open the connection for too long.
Even though your code has database calls in several places, the overhead of creating the connection will make things worse than waiting - unless you're saying your page takes many seconds to create on the server side? Usually, even without controlled data access and with many functions, your page should be well under a second to generate on the server.
I believe the default connection pool is about 20 connections but SQLServer can handle alot more. Getting a connection from the server takes the longest time (assuming you are not doing anything daft with your commands) so I see nothing wrong with getting a connection per page and killing it if used afterwards.
For scalability you could run into problems where your connection pool gets too busy and time outs while your script waits for a connection to be come available while your DB is sat there with a 100 spare connections but no one using them.
Create and kill on the same page gets my vote.
From a performance point of view there is no notable difference. ADODB connection pooling manages the actual connections with the db. Adodb.connection .open and .close are just a façade to the connection pool. Instantiating either 1 or 15 adodb.connection objects doesn't really matter performance wise. Before we where using transactions we used the connection string in combination with adodb.command (.activeConnection) and never opened or closed connections explicitly.
Reasons to explicitly keep reference to a adodb.connection are transactions or connection-based functions like mysql last_inserted_id(). In these cases you must be absolutely certain that you are getting the same connection for every query.
I have a database system developed in VB6, and we have a scenario where more than one user may register at the same time triggering an insert in the database. I have used normal sqlconnection and recordset to make the insert and i initialize it with a pessimistic lock. Now how can i check in my application before inserting a record, if the table has been locked or not, thus if the table being inserted to has been locked currently i can alert the user that the table is in use or i can store his data temporarily and insert it once the lock is released. The underlying database is Access and the application is across multiple systems with database on a server.
You might want to read through Locking Shared Data by Using Recordset Objects in VBA. Most of it applies to VB6 as well as VBA.
It isn't really "normal" to lock a whole table, and you can't even do it via ADO and the Jet OLE DB Provider. Your question doesn't provide enough information to suggest any specific course of action.
You don't "check before inserting" either. Applications should be designed to stumble over locks relatively rarely. When they do, you deal with this as an exception. This is reflected in both the DAO and ADO APIs.