I’m trying to kill a process in Azure and I’m getting this status message.
“SPID 55: transaction rollback in progress. Estimated rollback completion: 0%. Estimated time remaining: 0 seconds.”.
Apparently, there is a rollback on a table taken place and it’s been going on for 3 days now, it has over 13M records.
I tried renaming, deleting, dropping, truncating the table, but to no avail.
Is there anything else one can do, to resolve this issue?
wait for transaction to be completed
if you are willing to take a risk - you can delete the DB and restore to a specific point in time just before you executed the transaction that you killed.
regarding contacting azure support, i am afraid it will probably take more time then for the transaction to rollback
Please contact support. It's not clear from your description if you've hit an issue with out service. The best way to resolve this is with a support ticket. You can create a ticket directly in portal.azure.com.
Related
I am shipping transaction logs to another database that we will be using for certain reports that don't need real-time data. This works fine until we start directing traffic to it, then it lasts for a day or so and then just gets stuck in a Restoring state.
I am shipping transaction logs every 15 minutes from the production server and on the standby server, I have it set to to Standby mode with 'Disconnect users in the database when restoring backups' set to true (if I don't do this, the restore will be put off sometimes by a day or so, I am ok with killing the active sessions). Also on the standby server I have it set to run LSRestore every 10 minutes.
The problem is that I don't know what is causing the database to hang, nor do I know where I can even look to get some diagnostics that may tell me something.
Does anyone know where I can look?
It sounds like the Secondary cannot keep up with the processing of the restores of the t-log backups on the secondary. I would try decreasing the time of the backups, copies, and restores and see if you can find an optimal time interval where the secondary database does not stay in the restoring mode. I see you already have it down to every 15 minutes. Also, check this article and use the t-sql in it to dive down deep and try and see where the delays are occurring in more detail - https://www.sqlshack.com/monitor-transaction-log-shipping-using-t-sql-and-ssms/. I hope this helps.
I created a stored procedure which sends an email and accidentally called the stored procedure within itself creating an endless loop. Within a few seconds of executing the stored procedure I realized what I had done and fixed the loop, but it had already created 517 processes. I killed all the SPID's but they are stuck in a KILLED/ROLLBACK state.
This code shows me the processes:
select session_id,handle.percent_complete, *
from sys.dm_exec_requests handle
outer apply sys.fn_get_sql(handle.sql_handle) spname
where cast(handle.start_time as date) = '2022-01-10'
spname.text is showing 'xp_sysmail_format_query' for all the SPID's. It's been two days, and all 517 processes have been stuck in this rollback state with 0% progress. We are still able to use all our business applications and execute queries, with the exception of EXEC msdb.dbo.sp_send_dbmail which, when starting even a test email, gets stuck executing and has to be cancelled. This is not good because any auto generated email warnings will not be sent, and all other sql email functions are blocked. I'm not sure what other jobs are being blocked at this time.
This is a huge problem and I cannot find a solution. I've read every post I can find about this. I've tried everything I can think of except restarting the SQL server. Some posts state that restarting the SQL server can fix this and some state not to restart it or that the tasks will just resume in the killed/rollback state when restarted. I tried killing the spids again with statusonly but that just informs me that they are in a rollback state with 0% complete.
Should I restart the server and will this fix anything? Is there another solution other than restoring the DB to a backup that is more than 2 days old and losing all the work the entire business has done in the last couple days?
Any assistance will be greatly appreciated.
As robust as it is, sometimes (fortunately rarely) SQL Server leaves us no choice when killing a process to adopt the IT mantra of turning it off an on again when the rollback does not complete in a timely fashion.
This can be more prevalent when a transaction enlists external methods or functions, email is notorious for this inparticular.
As unwelcome as it is, it's often the least-expensive in terms of time and should be considered an option soon in the diagnosis process when the low-hanging fruit options have been exhausted.
I know it is possible to kill Postgres idle transaction, however is it possible to force it to commit. The commit needs to come from some kind of an agent that monitors idle transactions and not from the module that started the transaction.
Thanks
Avi
There is no way to do this, and that's a good thing too. How could you be certain that the transaction's work is done and the database is in a consistent state afterwards?
The correct solution is to fix the application so that it does not leave transactions “hanging”.
I'm a new "accidental" DBA and I'm currently trying to resolve a lockup caused by a trigger I created on a production database supporting a front end application.
I created a trigger, and then I decided I'd be best off creating a job to do the work instead, so tried to delete the trigger in object explorer. The delete failed with the message:
An exception occurred while executing a Transact-SQL statement or batch.
Lock request time out period exceeded.
I then tried to manually drop it and it failed at 0%, 0s left to go. I checked for the longest running transaction and then tried to kill the process in activity monitor. Since then the process has been stuck on "Task State:RUNNING and Command:KILLED/ROLLBACK". After some googling it sounds like I have two options.
Option 1: Restart DTC on the SQL server.... didn't work, still stuck.
Option 2: Restart the SQL service. Uh-oh.
This is the first time I've ever had to do anything like this and I'm pretty nervous being the only SQL guy in the office. Please can anyone let me know what the potential implications of restarting the service are, in terms of data loss and impact to front end users? Am I better off waiting to restart after business hours?
Thanks, and apologies if I've asked this question badly, first time for everything.
Cheers
Wait. It's rolling back and has to finish the rollback. Don't restart SQL, that will just result in the rollback continuing after the restart, possibly with the database offline.
If this is a production system and you do bounce the database, all users of your user interface will get weird and wonderful errors. Unless your application can handle it, your users will have a bad experience and then you will start getting phone calls from the boss....
As a side note, check for locking\blocking processes. The message in the question "Lock request time out period exceeded. " seems to suggest there is locking/blocking happening.
I have recently tried a big merge of 2 databases. We recreated the schema from Database 2 into Database 1 and created a script to transfer all data from database 2 into Database 1. This script takes about 35 min to run and have transaction handling with:
BEGIN TRANSACTION
...
IF(##error<>0)
COMMIT TRANSACTION
ELSE
ROLLBACK TRANSACTION
The full script is a bit sensitive but here is some SQL that have the same structure: http://pastebin.com/GWJ3ZnkF
We ran the script and all data was transfered without errors. We tested the systems running with the new combined database (removed access rights to the old database).
But as a last task we wanted to take the old database offline to make sure no one used that database. To do this we used:
ALTER DATABASE <dbname> SET OFFLINE WITH ROLLBACK IMMEDIATE
This was bad. After this line of SQL code all data in the combined database that we just copied was suddenly gone. I first asumed it wasn't really finished so the "Rollback immediate" sounds like it have performed a rollback on my transaction..
But why? Wasn't the transaction allready committed?
Also I tried running the same script again a few times but after every attempt no data was copied even if it said the script was successfull. I have no idea why... did it remember my offline rollback somehow?
What is really happening to my connections?
Sounds like you had a pending transaction uncommitted and you forced it to rollback, loosing some of the work. The rest is explained by how your scripts are structured. Is unlikely your script had a single transaction from start to bottom. Only the last transaction was rolled back, so the database was left now in a state in which it is 'half copied'. Probably your script does various checks and this intermediate state sends the script on the 'ELSE' branches where it does not do the proper work (ie. apparently does nothing).
W/o posting the exact script, is all speculation anyway.
Right now you need to restore the database to a consistent state, the one before your data copy. Use the backup you took before the data move (you did take a backup, right?). for extra credit, make sure your script is idempotent and works correctly on a half-updated database.
I'd double-check to make sure that there are no outstanding transactions. Either go through the file and count the number of BEGIN TRANSACTION vs COMMIT TRANSACTION lines, or add a statement to the end of it to SELECT ##TRANCOUNT to ensure that there are no open transactions remaining.
If your data has been committed, there should be no way it can be lost by disconnecting you.
WITH ROLLBACK IMMEDIATE:
All incomplete transactions will be rolled back and any other
connections to the database will be
immediately disconnected.
Sounds like someone got the 2 databases mixed up or maybe there is an outstanding transaction?.... Can you post your entire script?
Ref: ALTER DATABASE.
Rather than only checking ##ERROR, inspect ##TRANCOUNT as well.