I have already completed this job, but wondering whether what I did was correct and / or whether you would have performed the job differently.
A couple of the infrastructure guys came to me saying that they were moving the storage for one of the SQL servers and they wanted to check whether there was any break in connection or performance while they were doing it - they had some company in that said they could move the storage without having any impact on the business and this is the first try.
To test this out, they wanted me to write a query that would use the storage and report any breaks, so I wrote the following script ad hoc:
CREATE TABLE [dbo].[TomAndRickysSpecialTable]
( [Counter] INT IDENTITY(1,1) NOT NULL
,[Result] VARCHAR(10) NOT NULL
,[TimeStamp] DATETIME2(0) NOT NULL DEFAULT(GETDATE())
,CONSTRAINT PK_TikkiTavi PRIMARY KEY CLUSTERED ([counter] ASC)
)
exec sp_configure 'remote query timeout', 0
go
reconfigure with override
go
CREATE PROC RICKYRICKYRICKYRICKYrickyTom
AS
TRUNCATE TABLE TomAndRickysSpecialTable;
INSERT TomAndRickysSpecialTable(Result) VALUES ('Start');
SET NOCOUNT ON;
TryAgain:
WHILE 1 = 1
BEGIN
BEGIN TRY
WAITFOR DELAY '00:00:01';
IF EXISTS (SELECT TOP 1 [Counter] FROM TomAndRickysSpecialTable)
INSERT TomAndRickysSpecialTable(Result) VALUES ('Yup');
RAISERROR('Yup',10,1) WITH NOWAIT;
END TRY
BEGIN CATCH
RAISERROR('Nop',10,1) WITH NOWAIT;
Goto TryAgain;
END CATCH
END
I created the table and sproc and they executed it from the server, leaving it to run and observing any breaks in the second by second timestamp.
I figured I should write something that queried and wrote to the memory, so that we could see if there was a problem, but I have no idea whether that reasoning was correct...
My question is this - would my script have actually done what they wanted? Was there any point in running it or would it for instance have failed so badly if there was a break that there was no reportable result? Could I have written a simpler or better script to do the same job?
PS. I hated writing the While loop and the goto, but figured it would do the job in this weird case!
would my script have actually done what they wanted?
Yes. SQL Server caches data in memory, but changes must be hardened to disk on commit, and by default SQL Server commits after each statement. So the INSERT would have failed if the disk containing the database log was unavailable.
BTW 'remote query timeout' is irrelevant.
I hated writing the While loop and the goto,
It's not needed. After the CATCH block the WHILE loop would have run again.
Related
I have a stored procedure where I send in an user defined type which is a table. I have simplified to make it easier to read.
CREATE TYPE [dbo].[ProjectTableType] AS TABLE(
[DbId] [uniqueidentifier] NOT NULL,
[DbParentId] [uniqueidentifier] NULL,
[Description] [text] NULL
)
CREATE PROCEDURE [dbo].[udsp_ProjectDUI] (#cmd varchar(10),
#tblProjects ProjectTableType READONLY) AS BEGIN
DECLARE #myNewPKTable TABLE (myNewPK uniqueidentifier)
IF(LOWER(#cmd) = 'insert')
BEGIN
INSERT INTO
dbo.Project
(
DbId,
DbParentId,
Description)
OUTPUT INSERTED.DbId INTO #myNewPKTable
SELECT NEWID(),
DbParentId,
Description
FROM #tblProjects;
SELECT * FROM dbo.Project WHERE dbid IN (SELECT myNewPK FROM #myNewPKTable);
END
This is for a DLL that other applications will use so we aren't in charge on validation necessarily. I want to mimic BULK INSERT where if one rows fails to insert but the other rows are fine, the correct ones will still insert. Is there a way to do this? I want to do it for UPDATE as well where if one fails, the stored procedure will continue to try updating the others.
The only option I can think of is to only do one at a time (either a loop in the code where the stored proc is called multiple times or a loop in the stored procedure), but was wondering what the performance hit would be for that or if there is a better solution.
Not sure which errors you're wanting to continue on, but unless you're running into all manner of unexpected errors, I'd try to avoid devolving into RBAR just yet.
Check Explicit Violations
The main thing I would think would be PK volations which you can avoid by just checking for existence before insert (and update). If there are other business logic failure conditions, you can check those here as well.
insert into dbo.Project
(
DbId,
DbParentId,
Description
)
output insert.DbId
into #myNewPKTable (DbId)
select
DbId = newid(),
DbParentId = s.DbParentId,
Description = s.Description
from #tblProjects s -- source
-- LOJ null check makes sure we don't violate PK
-- NOTE: I'm pretending this is an alternate key of the table.
left outer join dbo.Project t -- target
on s.dbParentId = t.dbParentId
where t.dbParentId is null
If at all possible, I'd try to stick with a batch update, and use join predicates to eliminate the possibility of most errors you expect to see. Changing to RBAR processing because you're worried you "might" get a system shutdown failure is probably a waste of time. Then, if you hit a really nasty error you can't recover from, fail the batch, legitimately.
RBAR
Alternatively, if you absolutely need row-by-row granularity of successes or failures, you could do try/catch around each statement and have the catch block do nothing (or log something).
declare
#DBParentId int,
#Description nvarchar(1000),
#Ident int
declare c cursor local fast_forward for
select
DbParentId = s.DbParentId,
Description = s.Description
from #tblProjects
open c
fetch next from c into #DBParentId, #Description
while ##fetch_status = 0
begin
begin try
insert into dbo.Project
(
DbId,
DbParentId,
Description
)
output insert.DbId
into #myNewPKTable (DbId)
select
newid(),
#DBParentId,
#Description
end try
begin catch
-- log something if you want
print error_message()
end catch
fetch next from c into #DBParentId, #Description
end
Hybrid
You might be able to get clever and hybridize things. One option might be to make the web-facing insert procedure actually insert into a lightweight, minimally keyed/constrainted table (like a queue). Then, every minute or so, have an Agent job run through the logged calls, and operate on them in a batch. Doesn't fundamentally change any of the patterns here, but it makes the processing asynchronous so the caller doesn't have to wait, and by batching requests together, you can save processing power piggybacking on what SQL does best; set based ops.
Another option might be to do the most set-based processing you can (using checks to prevent business rule or constraint violations). If anything fails, you could then spin off an RBAR process for the remaining rows. If all succeeds though, that RBAR process is never hit.
Conclusion
Several ways to go about this. I'd try to use set-based operations as much as possible unless you have a really strong reason for needing row-by-row granularity.
You can avoid most errors just by constructing your insert/update statement correctly.
If you need to, you can use a try/catch with an "empty" catch block so failures don't stop the total processing
Depending on random odds and ends of your specific situation, you might want or need to hybridize these two approaches
EDIT This questions is no longer valid as the issue was something else. Please see my explanation below in my answer.
I'm not sure of the etiquette so i'l leave this question in its' current state
I have a stored procedure that writes some data to a table.
I'm using Microsoft Practices Enterprise library for making my stored procedure call.
I invoke the stored procedure using a call to ExecuteNonQuery.
After ExecuteNonQuery returns i invoke a 3rd party library. It calls back to me on a separate thread in about 100 ms.
I then invoke another stored procedure to pull the data I had just written.
In about 99% of cases the data is returned. Once in a while it returns no rows( ie it can't find the data). If I put a conditional break point to detect this condition in the debugger and manually rerun the stored procedure it always returns my data.
This makes me believe the writing stored procedure is working just not committing when its called.
I'm fairly novice when it comes to sql, so its entirely possible that I'm doing something wrong. I would have thought that the writing stored procedure would block until its contents were committed to the db.
Writing Stored Procedure
ALTER PROCEDURE [dbo].[spWrite]
#guid varchar(50),
#data varchar(50)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- see if this guid has already been added to the table
DECLARE #foundGuid varchar(50);
SELECT #foundGuid = [guid] from [dbo].[Details] where [guid] = #guid;
IF #foundGuid IS NULL
-- first time we've seen this guid
INSERT INTO [dbo].[Details] ( [guid], data ) VALUES (#guid, #data)
ELSE
-- updaeting or verifying order
UPDATE [dbo].[Details] SET data =#data WHERE [guid] = #guid
END
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
Reading Stored Procedure
ALTER PROCEDURE [dbo].[spRead]
#guid varchar(50)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SELECT * from [dbo].[Details] where [guid] = #guid;
END
To actually block other transactions and manually commit,
maybe adding
BEGIN TRANSACTION
--place your
--transactions you wish to do here
--if everything was okay
COMMIT TRANSACTION
--or
--ROLLBACK TRANSACTION if something went wrong
could help you?
I’m not familiar with the data access tools you mention, but from your description I would guess that either the process does not wait for the stored procedure to complete execution before proceeding to the next steps, or ye olde “something else” is messing with the data in between your write and read calls.
One way to tell what’s going on is to use SQL Profiler. Fire it up, monitor all possible query execution events on the database (including stored procedure and stored procedures line start/stop events), watch the Text and Started/Ended columns, correlate this with the times you are seeing while tracing the application, and that should help you figure out what’s going on there. (SQL Profiler can be complex to use, but there are many sources on the web that explain it, and it is well worth learning how to use it.)
I'll leave my answer below as there are comments on it...
Ok, I feel shame I had simplified my question too much. What was actually happening is two things:
1) the inserting procedure is actually running on a separate machine( distributed system).
2) the inserting procedure actually inserts data into two tables without a transaction.
This means the query can run at the same time and find the tables in a state where one has been written to and the second table hasn't' yet had its write committed.
A simple transaction fixes this as the reading query can handle either case of no write or full write but couldn't handle the case of one table written to and the other having a pending commit.
Well it turns out that when I created the stored procedure the MSSQLadmin tool added a line to it by default:
SET NOCOUNT ON;
If I turn that to:
SET NOCOUNT OFF;
then my procedure actually commits to the database properly. Strange that this default would actually end up causing problems.
Easy way using try-catch, like it if useful
BEGIN TRAN
BEGIN try
INSERT INTO meals
(
...
)
Values(...)
COMMIT TRAN
END try
BEGIN catch
ROLLBACK TRAN
SET #resp = (convert(varchar,ERROR_LINE()), ERROR_MESSAGE() )
END catch
I've got a simple SQL command that is supposed to read all of the records in from a table and then delete them all. Because there's a chance someone else could be writing to this table at the exact moment, I want to lock the table so that I'm sure that everything I delete is also everything I read.
BEGIN TRAN T1;
SELECT LotID FROM fsScannerIOInvalidCachedLots WITH (TABLOCK, HOLDLOCK);
DELETE FROM fsInvalidCachedLots;
COMMIT TRAN T1;
The really strange thing is, this USED to work. It worked for a while through testing, but now I guess something has changed because it's reading everything in, but it's not deleting any of the records. Consequently, SQL Server is spinning up high CPU usage because when this runs the next time it takes significantly longer to execute, which I assume has something to do with the lock.
Any idea what could be going on here? I've tried both TABLOCK and TABLOCKX
Update: Oh yea, something I forgot to mention, I can't query that table until after the next read the program does. What I mean is, after that statement is executed in code (and the command and connection are disposed of) if I try to query that table from within Management Studio it just hangs, which I assume means it's still locked. But then if I step through the calling program until I hit the next database connection, the moment after the first read the table is no longer locked.
Your SELECT retrieves data from a table named fsScannerIOInvalidCachedLots, but the delete is from a different table named fsInvalidCachedLots.
If you run this query in set xact_abort off, the transaction will not be aborted by the error from the invalid table name. In fact, select ##trancount will show you that there is an active transaction, and select xact_state() will return 1 meaning that it is active and no error has occurred.
On the other hand, with set xact_abort on, the transaction is aborted. select ##trancount will return 0, and select xact_state() will return 0 (no active transaction).
See ##trancount and xact_state() on MSDN for more information about them.
I'm trying to write to a log file inside a transaction so that the log survives even if the transaction is rolled back.
--start code
begin tran
insert [something] into dbo.logtable
[[main code here]]
rollback
commit
-- end code
You could say just do the log before the transaction starts but that is not as easy because the transaction starts before this S-Proc is run (i.e. the code is part of a bigger transaction)
So, in short, is there a way to write a special statement inside a transaction that is not part of the transaction. I hope my question makes sense.
Use a table variable (#temp) to hold the log info. Table variables survive a transaction rollback.
See this article.
I do this one of two ways, depending on my needs at the time. Both involve using a variable, which retain their value following a rollback.
1) Create a DECLARE #Log varchar(max) value and use this: #SET #Log=ISNULL(#Log+'; ','')+'Your new log info here'. Keep appending to this as you go through the transaction. I'll insert this into the log after the commit or the rollback as necessary. I'll usually only insert the #Log value into the real log table when there is an error (in theCATCH` block) or If I'm trying to debug a problem.
2) create a DECLARE #LogTable table (RowID int identity(1,1) primary key, RowValue varchar(5000). I insert into this as you progress through your transaction. I like using the OUTPUT clause to insert the actual IDs (and other columns with messages, like 'DELETE item 1234') of rows used in the transaction into this table with. I will insert this table into the actual log table after the commit or the rollback as necessary.
If the parent transaction rolls back the logging data will roll back as well - SQL server does not support proper nested transactions. One possibility is to use a CLR stored procedure to do the logging. This can open its own connection to the database outside the transaction and enter and commit the log data.
Log output to a table, use a time delay, and use WITH(NOLOCK) to see it.
It looks like #arvid wanted to debug the operation of the stored procedure, and is able to alter the stored proc.
The c# code starts a transaction, then calls a s-proc, and at the end it commits or rolls back the transaction. I only have easy access to the s-proc
I had a similar situation. So I modified the stored procedure to log my desired output to a table. Then I put a time delay at the end of the stored procedure
WAITFOR DELAY '00:00:12'; -- 12 second delay, adjust as desired
and in another SSMS window, quickly read the table with READ UNCOMMITTED isolation level (the "WITH(NOLOCK)" below
SELECT * FROM dbo.NicksLogTable WITH(NOLOCK);
It's not the solution you want if you need a permanent record of the logs (edit: including where transactions get rolled back), but it suits my purpose to be able to debug the code in a temporary fashion, especially when linked servers, xp_cmdshell, and creating file tables are all disabled :-(
Apologies for bumping a 12-year old thread, but Microsoft deserves an equal caning for not implementing nested transactions or autonomous transactions in that time period.
If you want to emulate nested transaction behaviour you can use named transactions:
begin transaction a
create table #a (i int)
select * from #a
save transaction b
create table #b (i int)
select * from #a
select * from #b
rollback transaction b
select * from #a
rollback transaction a
In SQL Server if you want a ‘sub-transaction’ you should use save transaction xxxx which works like an oracle checkpoint.
I have a table I'm using as a work queue. Essentially, it consists of a primary key, a piece of data, and a status flag (processed/unprocessed). I have multiple processes trying to grab the next unprocessed row, so I need to make sure that they observe proper lock and update semantics to avoid race condition nastiness. To that end, I've defined a stored procedure they can call:
CREATE PROCEDURE get_from_q
AS
DECLARE #queueid INT;
BEGIN TRANSACTION TRAN1;
SELECT TOP 1
#queueid = id
FROM
MSG_Q WITH (updlock, readpast)
WHERE
MSG_Q.status=0;
SELECT TOP 1 *
FROM
MSG_Q
WHERE
MSG_Q.id=#queueid;
UPDATE MSG_Q
SET status=1
WHERE id=#queueid;
COMMIT TRANSACTION TRAN1;
Note the use of "WITH (updlock, readpast)" to make sure that I lock the target row and ignore rows that are similarly locked already.
Now, the procedure works as listed above, which is great. While I was putting this together, however, I found that if the second SELECT and the UPDATE are reversed in order (i.e. UPDATE first then SELECT), I got no data back at all. And no, it didn't matter whether the second SELECT was before or after the final COMMIT.
My question is thus why the order of the second SELECT and UPDATE makes a difference. I suspect that there is something subtle going on there that I don't understand, and I'm worried that it's going to bite me later on.
Any hints?
by default transactions are READ COMMITTED :
"Specifies that shared locks are held while the data is being read to avoid dirty reads, but the data can be changed before the end of the transaction, resulting in nonrepeatable reads or phantom data. This option is the SQL Server default."
http://msdn.microsoft.com/en-us/library/aa259216.aspx
I think you are getting nothing in the select because the record is still marked as dirty. You'd have to change the transaction isolation level OR, what I do is do the update first and then read the record, but to do this you have to flag the record w/ a unique value (I use a getdate() for batchs but a GUID would be what you probably want to use).
Although not directly answering your question here, rather than reinventing the wheel and making life difficult for yourself, unless you enjoy it of course ;-), may I suggest that you look at using SQL Server Service Broker.
It provides an existing framework for using queues etc.
To find out more visit.
Service Broker Link
Now back to the question, I am not able to replicate your problem, as you will see if you execute the code below, data is returned regardless of the order os the select/update statement.
So your example above then.
create table #MSG_Q
(id int identity(1,1) primary key,status int)
insert into #MSG_Q select 0
DECLARE #queueid INT
BEGIN TRANSACTION TRAN1
SELECT TOP 1 #queueid = id FROM #MSG_Q WITH (updlock, readpast) WHERE #MSG_Q.status=0
UPDATE #MSG_Q SET status=1 WHERE id=#queueid
SELECT TOP 1 * FROM #MSG_Q WHERE #MSG_Q.id=#queueid
COMMIT TRANSACTION TRAN1
select * from #MSG_Q
drop table #MSG_Q
Returns the Results (1,1) and (1,1)
Now swapping the statement order.
create table #MSG_Q
(id int identity(1,1) primary key,status int)
insert into #MSG_Q select 0
DECLARE #queueid INT
BEGIN TRANSACTION TRAN1
SELECT TOP 1 #queueid = id FROM #MSG_Q WITH (updlock, readpast) WHERE #MSG_Q.status=0
SELECT TOP 1 * FROM #MSG_Q WHERE #MSG_Q.id=#queueid
UPDATE #MSG_Q SET status=1 WHERE id=#queueid
COMMIT TRANSACTION TRAN1
select * from #MSG_Q
drop table #MSG_Q
Results in: (1,0), (1,1) as expected.
Perhaps you could qualify your issue further?
More experimentation leads me to conclude that I was chasing a red herring, brought about by the tools I was using to exec my stored procedure. I was initially using DBVisualizer (free edition) and Netbeans, and they both appear to be confused by something about the format of the results. DBVisualizer suggests that I'm getting multiple result sets back, and that the free edition doesn't handle that.
Since then, I grabbed the free MS SQL Server Management Studio Express and things work perfectly. For those interested, the URL to SMSE is here:
MS SQL Server SMSE
Don't forget to install the MSXML6 service pack, too:
MSXML Service Pack 1
So, totally my bad in this case. :-(
Major thanks and kudos to you guys for your answers though. You helped me confirm that what I was doing should work, which lead me to the change I had to make to actually "solve" the issue. Thanks ever so much!
One more point-- including a "SET NOCOUNT ON" in the stored procedure fixed things for all ODBC clients. Apparently the rowcounts for the first select was confusing the ODBC clients, and telling SQL Server to not return that value makes things work perfectly...