What is the best approach of breaking long running stored procedures (up to 20 minutes)?
Inside the Stored procedure is wrapped in a transaction. If I close connection will this transaction be rolled back?
Another approach is to start a transaction in C# before I start the stored procedure and when I want to cancel the stored procedure I just need to rollback the C# transaction.
If you close the connection, SQL Server will rollback the transaction if it notices the disconnect before the transaction commits. There'll be a (very) small time window where the transaction might complete just when you disconnect.
A custom transaction adds complexity and has few benefits for a single stored procedure call. So I'd go for the disconnect.
You can set a time out in three place-- the connection, the command, and if you are ultimately programming a web page-- in the page time out.
The relevant time out in this case is the command time out.
Update: Cancelling by a user's event:
To cancel your command by a user event, call Cancel() on the command. I haven't written code to test this, but I suspect that once you call ExecuteReader() it will block, so you'd need an async call-- BeginExecuteNonQuery(), which really is a pain to set up-- it requires extra things on the query string and I think it requires SQL2005+
UPDATE: Re: Transactions
C# (or ADO.NET) transaction code adds about two lines of code and guarantees invocations of stored procedures (which may have more than one statement in them, in not today, maybe a year from now) succeed or fail as a unit. They normally are not a source of poor performance and can be a source of poor performance when not used-- e.g. a long series of inserts runs faster in a transaction.
If you do not call CommitTrans() the transaction will rollback, you do not have to explicitly call Rollback()
Set Timeout at the time of beginning the transaction.
Related
I have a question but I can never get a clear answer. Any stored
procedure that used a transaction that I have looked at up until my recent job always had a commit transaction + a roll back in case of error. However I have seen a lot of code
at my new job that just has a begin transaction and then a commit at the end with no roll back. I understand why you would use a transaction with a rollback but why would you want to begin a transaction with no roll back? Is it so when you run that code you want to lock the table up so no values can be changed why your code is updating? If so why would you not want the added security of a roll back in case something goes wrong? Is this proper use of the transaction statement? Any thoughts or ideas would be great!
For Example:
BEGIN TRANSACTION [Tran1]
INSERT INTO [Test].[dbo].[T1]
([Title], [AVG])
VALUES ('Tidd130', 130), ('Tidd230', 230)
UPDATE [Test].[dbo].[T1]
SET [Title] = N'az2' ,[AVG] = 1
WHERE [dbo].[T1].[Title] = N'az'
COMMIT TRANSACTION [Tran1]
GO
shouldn't this code be using a roll back syntax for proper use of the begin transaction statement?
The idea is that if that set of transactions needs to be "all or nothing", wrapping the lot in a transaction is the way to ensure that is what will happen. You're not seeing an explicit rollback because that's not what they're guarding against. Imagine the ff scenario with your contrived example:
The insert happens
The server crashes (or the log fills up or some other external reason why things can't continue) before the update can happen
If they're both wrapped in the same transaction, the insert won't be reflected in the table data. Which is the desired behavior.
When transactions are not explicitly declared, SQL Server will automatically BEGIN and COMMIT a TRANSACTION for each command. This frees up each command's lock as soon as the command executes.
When executing multiple commands inside a single transaction (as in the example you posted), locks from all commands are held until the transaction is committed.
Depending on the desired behavior, the script you posted may be correct. However, I would be cautious to ensure that the developer did not mistakenly believe that the transaction would be automatically rolled back on error. If that behavior is desired, you do indeed need to explicitly ROLLBACK or SET XACT_ABORT ON
You use transaction when you need the outcome to be atomic, you would see this alot in financial related procedures where you are gravely worried about data acid consistency . Otherwise it is not necessary and introduces a great deal of locking overhead. There is a good question here and here that goes into great depth.
Edit
The takeaway point is if the procedure is a all or none and must either succeed or fail the correct decision is to use a transaction. If the procedure is not a all or none transaction such as simple insert update etc using a transaction is a) unnecessary and b) can introduce an undue performance overhead due to additional locking.
If a stored procedure fails in middle, are changes at that point from the beginning of SP rolled back implicitly or do we have to write any explicit code to make sure that SP runs in a database transaction only?
Strictly speaking, Postgres did not have stored procedures as defined in the ISO/IEC standard before version 11. The term is often used incorrectly to refer to functions, which provide much of the same functionality (and more) as other RDBMS provide with "stored procedures". The main difference being transaction handling.
What are the differences between “Stored Procedures” and “Stored Functions”?
True stored procedures were finally introduced with Postgres 11:
When to use stored procedure / user-defined function?
Functions are atomic in Postgres and automatically run inside their own transaction unless called within an outer transaction. They always run inside a single transaction and succeed or fail completely. Consequently, one cannot begin or commit transactions within the function. And commands like VACUUM, CREATE DATABASE, or CREATE INDEX CONCURRENTLY which do not run in a transaction context are not allowed.
The manual on PL/pgSQL:
Functions and trigger procedures are always executed within a
transaction established by an outer query — they cannot start or
commit that transaction, since there would be no context for them to
execute in. However, a block containing an EXCEPTION clause
effectively forms a subtransaction that can be rolled back without
affecting the outer transaction.
Error handling:
By default, any error occurring in a PL/pgSQL function aborts
execution of the function, and indeed of the surrounding transaction
as well. You can trap errors and recover from them by using a BEGIN
block with an EXCEPTION clause.
There are exceptions, including but not limited to:
data written to log files
changes made to a sequence
Important: Some PostgreSQL data types and functions have special rules
regarding transactional behavior. In particular, changes made to a
sequence (and therefore the counter of a column declared using serial)
are immediately visible to all other transactions and are not rolled
back if the transaction that made the changes aborts.
prepared statements
SQL Fiddle demo
dblink calls (or similar)
Does Postgres support nested or autonomous transactions?
If you are using Postgres 14 procedure like below:
CREATE OR REPLACE PROCEDURE test_error(schema_name text)
LANGUAGE plpgsql
AS
$$
declare
<declare any vars that you need>
BEGIN
<do your thing>
END
$$;
For all practical purposes, code written in between the BEGIN and END block is executed in a single transaction. Hence, if any of the statements in the block fail, all the previous statements will be rolled back automatically. You do not need to explicitly write any roll back code.
However, there are special cases where one can have fine grained control over when to start/commit/rollback transactions. Refer to : https://www.postgresql.org/docs/current/plpgsql-transactions.html for details.
From the official document of Postgresql:
In procedures invoked by the CALL command as well as in anonymous code
blocks (DO command), it is possible to end transactions using the
commands COMMIT and ROLLBACK. A new transaction is started
automatically after a transaction is ended using these commands, so
there is no separate START TRANSACTION command. (Note that BEGIN and
END have different meanings in PL/pgSQL.)
https://www.postgresql.org/docs/11/plpgsql-transactions.html
I'm playing with the idea of rerouting every end-user stored procedure call of my database through a logging stored procedure. Essentially it will wrap the stored procedure call in some simple logging logic, who made the call, how long did it take etc.
Can this potentially create a bottleneck? I'm concerned that when the amount of total stored procedure calls grows this could become a serious problem.
Routing everything through a single point of entry is not optimal. Even if there are no performance issues, it is still something of a maintenance problem as you will need to expose the full range of Input Parameters that the real procs are accepting in the controller proc. Adding procs to this controller over time will require a bit of testing each time to make sure that you mapped the parameters correctly. Removing procs over time might leave unused input parameters. This method also requires that the app code pass in the params it needs to for the intended proc, but also the name (or ID?) of the intended proc, and this is another potential source of bugs, even if a minor one.
It would be better to have a general logging proc that gets called as the first thing of each of those procs. That is a standard template that can be added to any new proc quite easily. This leaves a clean API to the app code such that the app code is likewise maintainable.
SQL can run the same stored procedure concurrently, as long as it doesn't cause blocking or deadlocks on the resources it is using. For example:
CREATE PROCEDURE ##test
AS
BEGIN
SELECT 1
WAITFOR DELAY '00:00:10'
SELECT 2
END
Now execute this stored procedure quickly in two different query windows to see it running at the same time:
--Query window 1
EXEC ##test
--Query window 2
EXEC ##test
So you can see there won't be a line of calls waiting to EXECUTE the stored procedure. The only problem you may run into is if you are logging the sproc details to a certain table, depending on the isolation level, you could get blocking as the logging sproc locks pages in the table for recording the data. I don't believe this would be a problem unless you are running the logging stored procedure extremely heavily, but you'd want to run some tests to be sure.
I have a long-running SP (it can run for up to several minutes) that basically performs a number of cleanup operations on various tables within a transaction. I'm trying to determine the best way to somehow pass human-readable status information back to the caller on what step of the process the SP is currently performing.
Because the entire SP runs inside a single transaction, I can't write this information back to a status table and then read it from another thread unless I use NOLOCK to read it, which I consider a last resort since:
NOLOCK can cause other data inconsistency issues; and
this places the onus on anyone wanting to read the status table that they need to use NOLOCK because the table or row(s) could be locked for quite a while.
Is there any way to issue a single command (or EXEC a second SP) within a transaction and tell specify that that particular command shouldn't be part of the transaction? Or is there some other way for ADO.NET to gain insight into this long-running SP to see what it is currently doing?
You can PRINT messages in T-SQL and get them delivered to your SqlConnection in ADO.NET via the "InfoMessage" event. See
http://msdn.microsoft.com/en-us/library/a0hee08w.aspx
for details.
You could try using RAISERROR (use a severity of 10 or lower) within the procedure to return informational messages.
Example:
RAISERROR(N'Step 5 completed.', 10, 1) WITH NOWAIT;
I am running a bunch of database migration scripts. I find myself with a rather pressing problem, that business is waking up and expected to see their data, and their data has not finished migrating. I also took the applications offline and they really need to be started back up. In reality "the business" is a number of companies, and therefore I have a number of scripts running SPs in one query window like so:
EXEC [dbo].[MigrateCompanyById] 34
GO
EXEC [dbo].[MigrateCompanyById] 75
GO
EXEC [dbo].[MigrateCompanyById] 12
GO
EXEC [dbo].[MigrateCompanyById] 66
GO
Each SP calls a large number of other sub SPs to migrate all of the data required. I am considering cancelling the query, but I'm not sure at what point the execution will be cancelled. If it cancels nicely at the next GO then I'll be happy. If it cancels mid way through one of the company migrations, then I'm screwed.
If I cannot cancel, could I ALTER the MigrateCompanyById SP and comment all the sub SP calls out? Would that also prevent the next one from running, whilst completing the one that is currently running?
Any thoughts?
One way to acheive a controlled cancellation is to add a table containing a cancel flag. You can set this flag when you want to cancel exceution and your SP's can check this at regular intervals and stop executing if appropriate.
I was forced to cancel the script anyway.
When doing so, I noted that it cancels after the current executing statement, regardless of where it is in the SP execution chain.
Are you bracketing the code within each migration stored proc with transaction handling (BEGIN, COMMIT, etc.)? That would enable you to roll back the changes relatively easily depending on what you're doing within the procs.
One solution I've seen, you have a table with a single record having a bit value of 0 or 1, if that record is 0, your production application disallows access by the user population, enabling you to do whatever you need to and then set that flag to 1 after your task is complete to enable production to continue. This might not be practical given your environment, but can give you assurance that no users will be messing with your data through your app until you decide that it's ready to be messed with.
you can use this method to report execution progress of your script.
the way you have it now is every sproc is it's own transaction. so if you cancel the script you will get it update only partly up to the point of the last successfuly executed sproc.
you cna however put it all in a singel transaction if you need all or nothign update.