I'm a new "accidental" DBA and I'm currently trying to resolve a lockup caused by a trigger I created on a production database supporting a front end application.
I created a trigger, and then I decided I'd be best off creating a job to do the work instead, so tried to delete the trigger in object explorer. The delete failed with the message:
An exception occurred while executing a Transact-SQL statement or batch.
Lock request time out period exceeded.
I then tried to manually drop it and it failed at 0%, 0s left to go. I checked for the longest running transaction and then tried to kill the process in activity monitor. Since then the process has been stuck on "Task State:RUNNING and Command:KILLED/ROLLBACK". After some googling it sounds like I have two options.
Option 1: Restart DTC on the SQL server.... didn't work, still stuck.
Option 2: Restart the SQL service. Uh-oh.
This is the first time I've ever had to do anything like this and I'm pretty nervous being the only SQL guy in the office. Please can anyone let me know what the potential implications of restarting the service are, in terms of data loss and impact to front end users? Am I better off waiting to restart after business hours?
Thanks, and apologies if I've asked this question badly, first time for everything.
Cheers
Wait. It's rolling back and has to finish the rollback. Don't restart SQL, that will just result in the rollback continuing after the restart, possibly with the database offline.
If this is a production system and you do bounce the database, all users of your user interface will get weird and wonderful errors. Unless your application can handle it, your users will have a bad experience and then you will start getting phone calls from the boss....
As a side note, check for locking\blocking processes. The message in the question "Lock request time out period exceeded. " seems to suggest there is locking/blocking happening.
Related
I created a stored procedure which sends an email and accidentally called the stored procedure within itself creating an endless loop. Within a few seconds of executing the stored procedure I realized what I had done and fixed the loop, but it had already created 517 processes. I killed all the SPID's but they are stuck in a KILLED/ROLLBACK state.
This code shows me the processes:
select session_id,handle.percent_complete, *
from sys.dm_exec_requests handle
outer apply sys.fn_get_sql(handle.sql_handle) spname
where cast(handle.start_time as date) = '2022-01-10'
spname.text is showing 'xp_sysmail_format_query' for all the SPID's. It's been two days, and all 517 processes have been stuck in this rollback state with 0% progress. We are still able to use all our business applications and execute queries, with the exception of EXEC msdb.dbo.sp_send_dbmail which, when starting even a test email, gets stuck executing and has to be cancelled. This is not good because any auto generated email warnings will not be sent, and all other sql email functions are blocked. I'm not sure what other jobs are being blocked at this time.
This is a huge problem and I cannot find a solution. I've read every post I can find about this. I've tried everything I can think of except restarting the SQL server. Some posts state that restarting the SQL server can fix this and some state not to restart it or that the tasks will just resume in the killed/rollback state when restarted. I tried killing the spids again with statusonly but that just informs me that they are in a rollback state with 0% complete.
Should I restart the server and will this fix anything? Is there another solution other than restoring the DB to a backup that is more than 2 days old and losing all the work the entire business has done in the last couple days?
Any assistance will be greatly appreciated.
As robust as it is, sometimes (fortunately rarely) SQL Server leaves us no choice when killing a process to adopt the IT mantra of turning it off an on again when the rollback does not complete in a timely fashion.
This can be more prevalent when a transaction enlists external methods or functions, email is notorious for this inparticular.
As unwelcome as it is, it's often the least-expensive in terms of time and should be considered an option soon in the diagnosis process when the low-hanging fruit options have been exhausted.
We have a transaction that has a fair number of updates and inserts that run inside it as sent by our server-side code. We've run into an issue where all work in the transaction up to a given point is rolled back, and then later updates/inserts are run and end up being committed when the transaction is closed.
We narrowed it down to a bit of code where it would always happen and pulled that code out. Then the same behavior started happening elsewhere in the transaction. Nothing in our code is telling the transaction to roll back, and we haven't changed our code on our prod servers for quite a while before it started happening.
We finally restarted our prod db server and the problem went away, for a little while. Then it started happening again, and happened consistently after that.
We're on SQL Server 2016 and our web server is running ColdFusion 11 hotfix 18. Queries are being issued via <cfquery> inside a <cftransaction>.
Has anyone run into anything like this or have any way to diagnose the issue?
Update w/ more info: Apparently SQL Server was processing hardly any updates. We finally discovered that a log file was out of space. This usually will produce notifications/errors that this has happened, but this time it didn't for some reason. I realize this question is probably not the very highest quality, but in hopes that anyone who may run into this will find this and gain some benefit, I'm leaving it here.
I’m trying to kill a process in Azure and I’m getting this status message.
“SPID 55: transaction rollback in progress. Estimated rollback completion: 0%. Estimated time remaining: 0 seconds.”.
Apparently, there is a rollback on a table taken place and it’s been going on for 3 days now, it has over 13M records.
I tried renaming, deleting, dropping, truncating the table, but to no avail.
Is there anything else one can do, to resolve this issue?
wait for transaction to be completed
if you are willing to take a risk - you can delete the DB and restore to a specific point in time just before you executed the transaction that you killed.
regarding contacting azure support, i am afraid it will probably take more time then for the transaction to rollback
Please contact support. It's not clear from your description if you've hit an issue with out service. The best way to resolve this is with a support ticket. You can create a ticket directly in portal.azure.com.
We are using SQL Server 2012 Enterprise edition.
Normally we get hardly any blocked processes, but last weekend we experienced very unusual situation. Within 2 hours we got more "blocked process" alerts than we did in the last year together. There were a few hundred alerts within this time. Then suddenly without any interference from anyone everything went back to norm and we didn't get any blocked processes ever since. I want to prevent this situation from occurring again.
I am well aware how to find what can be causing blocking at present, but I have very little idea how to find what caused the block in the past, which is currently resolved.
I checked error logs in SQL Server Management Studio, but there is nothing there under the date when blocking occurred. There is also nothing unusual in the Windows event viewer. Where else should I check?
Could you please help?
From what you describe, I'm not too sure you will actually find the cause of the previously blocking processes if you did not actively setup tracing i.e. have your blocked process threshold set and configured with an alert to provide said trace information. The situation you described is interesting and definitely worth monitoring.
Here is an article on blocked process threshold configuration in SQL Server and a link through to Alerts configuration.
Hope this helps
I'm just debugging this error which I have suddenly seem to have gotten when I'm writing a row to a table.
Timeout expired. The timeout period
elapsed prior to completion of the
operation or the server is not
responding. The statement has been
terminated.
Does it have something to do with transaction scopes? I was wondering if I create a transaction scope and I debug through and I literally stop the application in the middle of debugging before it's reached the end of the trasnaction scope - is there a chance it'll block the database?
If so, how do I unblock it?
NOTE - This might help, right now, I'm having trouble inserting a row in tables but have accessing and updating existing rows
UPDATE - Well I reset the SQL Server service and it seems to have done the trick. Still, I'm curious to heard how it could've locked in the first place - I do not want some part of my code to be doing that and happening in production.
If opening a DB transaction would lock the entire database, that would be massively disappointing. Generally speaking, SQL Server locks on a per-row basis and then escalates these locks as neccessary (I'm simplifying matters here significantly).
Each transaction has a timeout it's given to complete. If this time elapses and you nether commit nor roll back, you'll get that "Timeout expired" exception.
As far as "unblocking" goes, you don't usually have to worry. Everything's unlocked as soon as you close the connection.