I am trying to simulate autonomous transaction in SQL Server, but the problem is, that using CLR DLL procedure (that is using different session) slows down the performance about 5 times.
To be clear:
Let's assume that in one transaction I am calling procedure for every of 100k rows in table, which gives 100k proc calls in one transaction. If any of this procedure fails, I want to rollback the entire transaction (AFTER all procedures calls), but I need to keep logs from the procedures that fails (in case of failure, insert to ErrorLog table).
The problem is, that in such case, I perform 100k connections, and it cost in terms of performance.
Using table variable is not a solution, because I am not able to control every transaction (some are controlled by frontend), using Loopback (to the same server) is not recommended in production (from what I read), so the solution was to use CLR for different session purpose.
Is there any solution to maybe create altered session for every session, and use that session for all those insert instead of creating new connection every time, or is my understanding of using CLR wrong, and it must open new connection every time. (From what I read, context_session uses the same session from what it was called, so in case of rollback, it will delete my logs from ErrorLog table).
You can use "SAVE TRAN XXXX" in the procedure and a "ROLLBACK TRAN XXXX" where an exception occurs. The Insert into the ERRORLOG table should be after the "ROLLBACK TRAN XXXX in the Procedure not to be rolled back.
Hope this helps...
Related
I have a long running stored procedure that is executed from IIS. On average this stored procedure takes between two and five minutes to complete because it is searching through a large dataset. (although it has take around 20 minutes in some cases)
Most of the time the stored procedure works fine but every now and then the SPIDS go into a sleeping state and never recover. The only solution I have found is to restart the SQL Server and re-run the stored procedure
The are no table inserts in the proc (only table variable inserts), and the other statements are selects on a large table.
I'm stuck for where to start debugging this issue. Any hints one what it might be or suggestions on tools that would help me find the issue would be most helpful
EDIT: More info added:
The actual issue is the proc doesn't return the resultset. My first thought was to look at the spids, they were sleeping but the cputime was still increasing
It's a .Net app so .Net Core 3.1 with ASP.NET Core and a Blazor UI. The libary used for db connection is System.data.SqlClient I believe System.data.SqlClient uses it's own custom driver. Calling code below:
The stored procedure doesn't return multiple result sets, however obviously different instances of the proc run at the same time.
No limits to connection pooling in IIS
#RichardWatts when you say " re-run the stored procedure" you mean that the same stored proc with the same parameter and data works once you restart SQL Server ?
If so look over your loc (sp_loc} inside your table probably another process loc some data and doesnt release it properly, specialy if you have transaction accessing the same tables.
What is your your isolation level on your connexion ? If you can, try to change it to READ UNCOMMITTED to see if that solve your problem.
as an alternate you can also add a WITH (NOLOCK) or (READUNCOMMITTED) to your sql command.
Know that you will need to hold query with a read uncommited or nolock if you have some modification on the structure of your table or index re construction for example or they will in turn block its execution
Nevertheless be cautious this solution depend on your environment, specially if your tables gots lots of update, delete, insert,... this kind of isolation can lead to a Dirty read and doesnt adress the root cause of your problem wich I would bet is uncomited transaction (good article that explain it)
Make also a DBCC CHECKTABLE just to be sure on this side
Let's say I have a stored procedure that is doing 3 inserts. To make sure everything is working fine, I add a begin and commit tran in the stored procedure.
Then from the code side (.NET, C#), the programmer is also creating a transaction.
What will be the best approach for that?
Having in both places?
Having that in the C# code only?
Having that in the stored procedure only?
It's better to only do it in the stored procedure for a number of reasons:
The procedure can keep better control of when to begin and commit the transaction.
It can also control the isolation level, which should usually be set before the transaction starts.
It keeps database code close to the database (somewhat subjective).
If the connection is severed and the server does not realize, a transaction opened by the client may not be committed or rolled back for some time, and could leave locks hanging, causing a huge chain of blocking
The client starting and committing the transaction requires two extra round-trips over the network, which in a low-latency app may be problematic. (SET NOCOUNT ON should be used for the same reason.) The transaction and associated locking is also extended for that time, casuing further blocking problems.
Do use SET XACT_ABORT ON, in case of exceptions this will cause an automatic rollback and prevent them from leaving hanging transactions.
It may still may sense to use client-side transactions, especially with distributed transactions.
Having transactions in both client code and the procedure is silly and wasteful. Choose one or the other option and stick to it.
I have some SQL statements in a batch that I want to profile for performance. To that end, I have created a stored procedure that logs execution times.
However, I also want to be able to roll back the changes the main batch performs, while still retaining the performance logs.
The alternative is to run the batch, copy the performance data to another database, restore the database from backup, re-apply all the changes made that I want to profile, plus any more, and start again. That is rather more time-consuming than not including the act of logging in the transaction.
Let us say we have this situation:
BEGIN TRANSACTION
SET #StartTime = SYSDATETIME
-- Do stuff here
UPDATE ABC SET x = fn_LongRunningFunction(x)
EXECUTE usp_Log 'Do stuff', #StartTime
SET #StartTime = SYSDATETIME
-- Do more stuff here
EXEC usp_LongRunningSproc()
EXECUTE usp_Log 'Do more stuff', #StartTime
ROLLBACK
How can I persist the results that usp_Log saves to a table without rolling them back along with the changes that take place elsewhere in the transaction?
It seems to me that ideally usp_Log would somehow not enlist itself into the transaction that may be rolled back.
I'm looking for a solution that can be implemented in the most reliable way, with the least coding or work possible, and with the least impact on the performance of the script being profiled.
EDIT
The script that is being profiled is extremely time-consuming - taking from an hour to several days - and I need to be able to see intermediate profiling results before the transaction completes or is rolled back. I cannot afford to wait for the end of the batch before being able to view the logs.
You can use a table variable for this. Table variables, like normal variables, are not affected by ROLLBACK. You would need to insert your performance log data into a table variable, then insert it into a normal table at the end of the procedure, after all COMMIT and ROLLBACK statements.
It might sound a bit overkill (given the purpose), but you can create a CLR stored procedure which will take over the progress logging, and open a separate connection inside it for writing log data.
Normally, it is recommended to use context connection within CLR objects whenever possible, as it simplifies many things. In your particular case however, you wish to disentangle from the context (especially from the current transaction), so regular connection is a way to go.
Caveat emptor: if you never dabbled with CLR programming within SQL Server before, you may find the learning curve a bit too steep. That, and the amount of server reconfiguration (both the SQL Server instance and the underlying OS) required to make it work might also seem to be prohibitively expensive, and not worth the hassle. Still, I would consider it a viable approach.
So, as Roger mentions above, SQLCLR is one option. However, since SQLCLR is not permitted, you are out of luck.
In SQL Server 2017 there is another option and that is to use the SQL Server extensibility framework and the support for Python.
You can use this to have Python code which calls back into your SQL Server instance and executes the usp_log procedure.
Another, rather obscure, option is to bind other sessions to the long-running transaction for monitoring.
At the beginning of the transaction call sp_getbindtoken and display the bind token.
Then in another session call sp_bindsession, and you can examine the intermediate state of the transaction.
Or you can read the logs with (NOLOCK).
Or you can use RAISERROR WITH LOG to send debug messages to the client and mirror them to the SQL Log.
Or you can use custom user-configurable trace events, and monitor them in SQL Trace or XEvents.
Or you can use a Loopback linked server configured to not propagate the transaction.
I have SQL Server 2017 Express database that is accessed by up to 6 tablets that connect via an Angular 7 app using REST web services.
I have a stored procedure that allows a new user to be inserted into a specific database table. The insert will always only insert 1 record at a time, but with 6 clients, the stored procedure could be called by each client almost simultaneously.
The last step of the process is to print an agreement form to a specific printer. Initially this was going to be handled on the client side, but the tablets do not have the ability to print to a network printer, so that functionality now needs to reside on the server side.
With this new requirement, the agreement form is an RTF document that is being read, placeholder values replaced with values from the insert statement, written to a temporary file and then printed to the network printer via the application (Wordpad most likely) that is associated with the RTF file format.
There is also an MS Access front-end app that uses linked servers to connect to the database, but doesn't have the ability to create new users, but will need to be able to initiate the "print agreement" operation in the case of an agreement not being printed due to printer issue, network issue, etc.
I have the C# code written to perform the action of reading/modifying/writing/printing of the form that uses the UseShellExecute StartInfo property with the Process.Start method.
Since the file read/modify/write/print process takes a few seconds, I am concerned about having the stored procedure for adding the registration blocking for that length of time.
I am pretty sure that I am going to need a CLR stored procedure so that the MS Access front-end can initiate the print operation, so what I have come up with is either the "Add_Registration" stored procedure (Transact-SQL) will call the CLR stored procedure to do the read/modify/write/print operation, or an insert trigger (either CLR or Transact-SQL) on the table that calls the CLR stored procedure to read/modify/write/print.
I could avoid the call from the trigger to the stored procedure by duplicating the code in both the CLR trigger and the CLR stored procedure if there is a compelling reason to do so, but was trying to avoid having duplicate code if possible.
The solutions that I am currently considering are as follows, but am unsure of how SQL Server handles various scenarios:
A CLR or Transact-SQL Insert trigger on the registration table that calls a CLR stored procedure that does the reading/modifying/writing/printing process.
A CLR stored procedure that does the reading/modifying/writing/printing process, being called from the current add_registration Transact-SQL stored procedure
The questions I keep coming back to are:
How are Insert CLR triggers executed if multiple inserts are done at the same or nearly the same time (only 1 per operation), are they queued up an then processed synchronously or are they executed immediately?
Same question as #1 except with a Transact-SQL trigger
How are CLR stored procedures handled if they are called by multiple clients at the same or nearly the same time, are they queued up an then processed synchronously, or is each call to the stored procedure executed immediately?
Same question as #3 except with a Transact-SQL stored procedure
If a CLR stored procedure is called from a Transact-SQL trigger, is the trigger blocked until the stored procedure returns or is the call to the stored procedure spawned out to it's own process (or similar method) with the trigger returning immediately?
Same question as #5 except with a CLR trigger calling the CLR stored procedure
I am looking for any other suggestions and/or clarifications on how SQL Server handles these scenarios.
There is no queuing unless you implement it yourself, and there are a few ways to do that. So for multiple concurrent sessions, they are all acting independently. Of course, when it comes to writing to the DB (INSERT / UPDATE / DELETE / etc) then they clearly operate in the order in which they submit their requests.
I'm not sure why you are including the trigger in any of this, but as it relates to concurrency, triggers execute within a system-generated transaction initiated by the DML statement that fired the trigger. This doesn't mean that you will have single-threaded INSERTs, but if the trigger is calling a SQLCLR stored procedure that takes a second or two to complete (and the trigger cannot continue until the stored procedure completes / exits) then there are locks being held on the table for the duration of that operation. Those locks might not prevent other sessions from inserting at the same time, but you might have other operations attempting to modify that table that require a conflicting lock that will need to wait until the insert + trigger + SQLCLR proc operation completes. Of course, that might only be a problem if you have frequent inserts, but that depends on how often you expect new users, and that might not be frequent enough to worry about.
I'm also not sure why you need to print anything in that moment. It might be much simpler on a few levels if you simply have a flag / BIT column indicating whether or not the agreement has been printed, defaulted to 0. Then, you can have an external console app, scheduled via SQL Server Agent or Windows Scheduled Tasks, executed once every few minutes, that reads from the Users table WHERE HasPrintedAgreement = 0. Each row has the necessary fields for the replacement values, it prints each one, and upon printing, it updates that UserID setting HasPrintedAgreement = 1. You can even schedule this console app to execute once every minute if you always want the agreements immediately.
i have two questions
1.i have two stored procedures. is it possible to commit/rollback another procedure's transaction in my current procedure.
2.i have two webservices two services connected with same database or linked server database. one webservices gotsucceed it transactions. when moving to the second webservice some error got occured. if error occured i have to rollback the previous webservice transactions.? is it possible. if anyone explain related to banking transactions like ATM
is it possible?
how?
explain related to banking sector with little understandable coding.
No, a commit must be issued from the same connection that the begin transaction statement was issued.
In this case you would first need to append the data tables with a "transaction" field or something similar that will uniquely identify each transaction. If the second webservice needed to issue a rollback that touched the work of the first webservice, it would have to call a custom process would then issue deletes looking for the transaction identifier that you have built into the tables. There's no built-in functionality of the db engine to accommodate this out of the box.