Let's say I have a table (let's call it myTable) on a database server, and I'm writing a desktop client application that allows you to modify records in myTable. I don't want two users to be able to edit the same record, i.e., if user A edits record 1 and user B tries to do the same, he gets notified that the record is currently "locked" by user A.
I'm sure that this is a common problem, so I'm wondering if there is a canonical solution to it.
There is an obvious solution, but it also has an obvious drawback:
Add a field inUseBy to myTable, set it to the user's name once the user starts editing the record, and clear it once the user finishes, e.g.
function editRecord(recordId):
begin transaction
if (select lockedBy from myTable where id = recordId) is not empty:
commit
show "Sorry, record already in use by ..."
else
update myTable set lockedBy = current_user() where id = recordId
commit
show UI window to let user edit and update record
update myTable set lockedBy = empty where id = recordId
Drawback: If user A's application crashes, the record stays locked.
The following approach might seem suitable at a first glance, but won't solve the problem:
Use database locks to lock record 1. This will just cause user B to run into a timeout. I need to lock the record on the application level not on the database level.
A common approach is to use ROWVERSION for optimistic concurrency.
Rowversion is a datatype (Which you'd add as a new column) that updates when the row is updated.
So, you select your row, including the rowversion column. When you send your update back, you make sure the rowversions match - By doing your update and adding "WHERE Rowversion = TheRowversionIGotEarlier" and seeing if ##ROWCOUNT is not 0 - If it is zero, then you can assume someone has modified the row since you read it, and you can return that message to the user.
http://www.mssqltips.com/sqlservertip/1501/optimistic-locking-in-sql-server-using-the-rowversion-data-type/
You're asking about pessimistic concurrency, and as this answer and the comment on your question say - Do consider optimistic concurrency. Consider your model, how would it handle someone starting to edit a record then going to lunch? Or if they never came back from lunch? What if a record has to be edited in the meantime, for a critical business reason?
Related
This is more of a concurrent use of software by multiple users. Consider the tables below
Table: ShippingList
ID Status
-- ------
1 PENDING SUBMIT
2 COMPLETE
3 DRAFT
Table: ShippingListItems
ID ShippingListId Value
-- -------------- -----
1 1 100
2 1 110
3 2 350
4 3 125
In the above table, ShippingList can be updated if its in not-complete state, i.e, items can be modified (even added/deleted) till its shipping status is not COMPLETE.
If multiple users have the application opened simultaneously and are modifying the same ShippingList, they can have different state of Shipping Items locally. If first user confirms the ShippingList and moves it to COMPLETE state and then the 2nd user tries to confirm the stock, he'll end up adding more ShippingItems onto a ShippingList which is already closed.
Need here is to somehow check for user 2 before committing that the stock its currently trying to update is already closed/completed.
How can I achieve that? Am using MS SQL server.
EDIT : How can I write a trigger on ShippingListItems table to do a check for status of ShippingList on insert/update?
To do this correctly, you'd need to implement a concurrency pattern. Which one you choose depends upon how the system is utilized.
It sounds like you want to utilize the Optimistic Locking pattern. I'd recommend setting a modification date column on the table you're updating. Then, each consumer of the data would fetch that date along with the data, and when an update is made, they'd pass the date back in. If the date has changed on the row they're attempting to update, you should raise an error and abort instead of making the update.
If you're looking for a simple solution, though, you could just check the status of that row before committing the change. Same deal, just bail out before making the change if your status is already closed. This would probably catch 99.9% of your woes.
NOTE: I have limited access to code, so had to push it down to the DB level which is also the right place for multiple instance of the program running and writing concurrently.
Ended up using a trigger on ShippingListItem and checking the Status of the ShippingList on insert/update. That seems to solve the problem.
CREATE TRIGGER [dbo].[ValidateShippingListItems]
ON [dbo].[ShippingListItems]
FOR INSERT, UPDATE
AS
BEGIN
if exists (select 1 from [dbo].[ShippingList] g inner join inserted i on i.ShippingList_Id = g.Id where g.Status = 3)
BEGIN
RAISERROR ('Shipping List is already in COMPLETE status',16,1)
ROLLBACK TRANSACTION
END
END
I've been trying for weeks to figure out an issue that happens once every 100,000+ transactions. I've tried dozens of variations and have run out of ideas, so I'm hoping someone has seen this before.
In summary, I have a table that acts like a queue. Records are inserted either singly or in transaction'ed batches, and sometimes one record is "dependent" on another (so that it is not eligible to be removed from the queue until the record it's dependent on is first removed). The basic structure of the tables includes these columns:
item_id - a GUID that uniquely identifies the record
depend_id - a GUID that identifies the record that this record is dependent on (or NULL if it's not dependent on anything)
item_lock - a GUID that starts out NULL, but is set to the "owner process" when the record is "locked down" to be worked on (when the work is done, the record is deleted from the table)
A simplified version of the polling query that is called to "lock down" the next "ready" record is:
UPDATE TOP(1) Q1 SET
lock_id = #lock,
FROM item_queue Q1
WHERE (lock_id IS NULL)
AND (depend_id IS NULL OR depend_id NOT IN (SELECT item_id FROM item_queue))
AND execute_at < GETUTCDATE()
My objective here (and this works almost all of the time) is that the NOT IN SELECT will simply check to see if the item that is otherwise eligible to have its lock_id set, that it won't be chosen if it's depend_id matches another item that's still in the table. But 1 out of 100,000+ calls to the stored procedure, that constraint fails and the record with the depend_id that does match an item_id that's still in the table gets chosen.
I have tried various alternatives to the NOT IN SELECT; all methods "work" but all fail in the same way. It is always the case that "dependent" records are inserted with their dependencies within a committed transaction.
Any and all ideas welcomed...I'm stumped.
PS - I should mention that there are many different threads on different client machines adding to and polling/locking/deleting records in this table. One of my working theories is that there is some sort of locking/contention that occasionally causes the record that is being depended on to not show up in the NOT IN SELECT subquery, causing the dependent record to become eligible (but I have not been able to come up with the specific scenario for that to happen).
EDIT: More on transaction isolation level: I'm running with the default READ COMMITTED isolation level. Is it possible that this is causing the "depended on" record to be omitted from the NOT IN SELECT subquery in the "race condition" case where another thread has just updated it? If so, I'm not entirely clear on what isolation level I need to ensure that any record that's still in the table (whether it's being updated or not) comes back in that query.
have you tried doing as a correlated update? By left-joining to the item queue table the second time by the depend_id to the item_id, you should not be locking anything in the second instance, and have the criteria just see if that left join result is NULL. If the depend_id was null, then it won't find a match via the left-join. If it DID have a value in depend_id, and not in as an item_id value, then it too would result in null. Only if the depend_id MATCHED and ITEM_ID would it NOT BE NULL and thus excluded from consideration.
UPDATE TOP(1) item_queue
SET lock_id = #lock
FROM item_queue
LEFT JOIN item_queue IQ2
ON item_queue.depend_id = IQ2.item_id
WHERE
lock_id is null
AND IQ2.item_id is null
AND execute_at < GETUTCDATE()
Based on the clarification of your Top(1), I might switch where the locking attempt might be done by doing the following... Pseudocode within your thread...
Select All Pending Possible Item_Queue entries.
scan through the list of records returned
update the lock_id = uniqueGUID
where item queue ID is the one you are working with LOCALLY
AND lock_id IS NULL
if the number of records returned = 1, then you got it.
process the item queue ID you successfully "locked"
else
if it comes back 0 records updated, someone else hit it
before you... continue with next available LOCAL record.
end scan of available LOCAL POSSIBLE queue records
The premise is this. Everyone queries all POSSIBLE queue items, and everyone tries to hit an update with their GUID. Since the update call here is specifically looking to update where the LOCK_ID IS NULL, if someone else hit it first, it is no longer null and won't update it (0 records updated). If so, try the next one and go through the list.
If no records are available after scanning through the entire available list, you may want to wrap that in a loop of say 2-3 times by getting a fresh list each cycle to remove any "in process" items now being processed and a fresh list of available to try with.
Other similar approaches have been to update with a numeric counter for a lock column and to set it to 1 more than the last value where the value equals what you originally retrieved (vs null in this case). This way, it allows a sort of tracker to how many TIMES a given record has been locked for updating.
We have an application that was originally written as a desktop app, lo these many years ago. It starts a transaction whenever you open an edit screen, and commits if you click OK, or rolls back if you click Cancel. This worked okay for a desktop app, but now we're trying to move to ADO.NET and SQL Server, and the long-running transactions are problematic.
I found that we'll have a problem when multiple users are all trying to edit (different subsets of) the same table at the same time. In our old database, each user's transaction would acquire record-level locks to every record they modified during their transaction; since different users were editing different records, everyone gets their own locks and everything works. But in SQL Server, as soon as one user edits a record inside a transaction, SQL Server appears to get a lock on the entire table. When a second user tries to edit a different record in the same table, the second user's app simply locks up, because the SqlConnection blocks until the first user either commits or rolls back.
I'm aware that long-running transactions are bad, and I know that the best solution would be to change these screens so that they no longer keep transactions open for a long time. But since that would mean some invasive and risky changes, I also want to research whether there's a way to get this code up and running as-is, just so I know what my options are.
How can I get two different users' transactions in SQL Server to lock individual records instead of the entire table?
Here's a quick-and-dirty console app that illustrates the issue. I've created a database called "test1", with one table called "Values" that just has ID (int) and Value (nvarchar) columns. If you run the app, it asks for an ID to modify, starts a transaction, modifies that record, and then leaves the transaction open until you press ENTER. I want to be able to
start the program and tell it to update ID 1;
let it get its transaction and modify the record;
start a second copy of the program and tell it to update ID 2;
have it able to update (and commit) while the first app's transaction is still open.
Currently it freezes at step 4, until I go back to the first copy of the app and close it or press ENTER so it commits. The call to command.ExecuteNonQuery blocks until the first connection is closed.
public static void Main()
{
Console.Write("ID to update: ");
var id = int.Parse(Console.ReadLine());
Console.WriteLine("Starting transaction");
using (var scope = new TransactionScope())
using (var connection = new SqlConnection(#"Data Source=localhost\sqlexpress;Initial Catalog=test1;Integrated Security=True"))
{
connection.Open();
var command = connection.CreateCommand();
command.CommandText = "UPDATE [Values] SET Value = 'Value' WHERE ID = " + id;
Console.WriteLine("Updating record");
command.ExecuteNonQuery();
Console.Write("Press ENTER to end transaction: ");
Console.ReadLine();
scope.Complete();
}
}
Here are some things I've already tried, with no change in behavior:
Changing the transaction isolation level to "read uncommitted"
Specifying a "WITH (ROWLOCK)" on the UPDATE statement
Just checking, but do you have a primary key or unique index on the ID column?
Look into optimistic versus pessimistic locking.
Edit:
Previous article linked to classic ado...sorry.
http://msdn.microsoft.com/en-us/library/cs6hb8k4(VS.71).aspx
Probably the index was created with row locks set to "off".
"WITH (ROWLOCK)" in a query would have no effect in that case.
You can turn them back on with ALTER INDEX, e.g.:
ALTER INDEX [PK_Values] ON [Values] SET (ALLOW_ROW_LOCKS = ON)
I'm trying to create a LINQ to SQL class that represents the "latest" version of itself.
Right now, the table that this entity represents has a single auto-incrementing ID, and I was thinking that I would add a version number to the primary key. I've never done anything like this, so I'm not sure how to proceed. I would like to be able to abstract the idea of the object's version away from whoever is using it. In other words, you have an instance of this entity that represents the most current version, and whenever any changes are submitted, a new copy of the object is stored with an incremented version number.
How should I proceed with this?
If you can avoid keeping a history, do. It's a pain.
If a complete history is unavoidable (regulated financial and medical data or the like), consider adding history tables. Use a trigger to 'version' into the history tables. That way, you're not dependent on your application to ensure a version is recorded - all inserts/updates/deletes are captured regardless of the source.
If your app needs to interact with historical data, make sure it's readonly. There's no sense capturing transaction histories if someone can simply change them.
If your concern is concurrent updates, consider using a record change timestamp. When both User A and User B view a record at noon, they fetch the record's timestamp. When User A updates the record, her timestamp matches the record's so the update goes through and the timestamp is updated as well. When User B updates the record five minutes later, his timestamp doesn't match the record's so he's warned that the record has changed since he last viewed it. Maybe it's automatically reloaded...
Whatever you decide, I would avoid inter-mingling current and historic data.
Trigger resources per comments:
MSDN
A SQL Team Introduction
Stackoverflow's Jon Galloway describes a general data-change logging trigger
The keys to an auditing trigger are the virtual tables 'inserted' and 'deleted'. These tables contain the rows effected by an INSERT, UPDATE, or DELETE. You can use them to audit changes. Something like:
CREATE TRIGGER tr_TheTrigger
ON [YourTable]
FOR INSERT, UPDATE, DELETE
AS
IF EXISTS(SELECT * FROM inserted)
BEGIN
--this is an insert or update
--your actual action will vary but something like this
INSERT INTO [YourTable_Audit]
SELECT * FROM inserted
END
IF EXISTS(SELECT * FROM deleted)
BEGIN
--this is a delete, mark [YourTable_Audit] as required
END
GO
The best way to proceed is to stop and seriously rethink your approach.
If you are going to keep different versions of the "object" around, then you are better off serializing it into an xml format and storing that in an XML column with a field for the version number.
There are serious considerations when trying to maintain versioned data in sql server revolving around application maintenance.
UPDATE per comment:
Those considerations include: the inability to remove a field or change the data type of a field in future "versions". New fields are required to be nullable or, at the very least, have a default value stored in the DB for them. As such you will not be able to use them in a unique index or as part of the primary keys.
In short, the only thing your application can do is expand. Provided the expansion can be ignored by previous layers of code.
This is the classic problem of Backwards Compatibility which desktop software makers have struggled with for years. And is the reason you might want to stay away from it.
I have designed database tables (normalised, on an MS SQL server) and created a standalone windows front end for an application that will be used by a handful of users to add and edit information. We will add a web interface to allow searching accross our production area at a later date.
I am concerned that if two users start editing the same record then the last to commit the update would be the 'winner' and important information may be lost. A number of solutions come to mind but I'm not sure if I am going to create a bigger headache.
Do nothing and hope that two users are never going to be editing the same record at the same time. - Might never happed but what if it does?
Editing routine could store a copy of the original data as well as the updates and then compare when the user has finished editing. If they differ show user and comfirm update - Would require two copies of data to be stored.
Add last updated DATETIME column and check it matches when we update, if not then show differences. - requires new column in each of the relevant tables.
Create an editing table that registers when users start editing a record that will be checked and prevent other users from editing same record. - would require carful thought of program flow to prevent deadlocks and records becoming locked if a user crashes out of the program.
Are there any better solutions or should I go for one of these?
If you expect infrequent collisions, Optimistic Concurrency is probably your best bet.
Scott Mitchell wrote a comprehensive tutorial on implementing that pattern:
Implementing Optimistic Concurrency
A classic approach is as follows:
add a boolean field , "locked" to each table.
set this to false by default.
when a user starts editing, you do this:
lock the row (or the whole table if you can't lock the row)
check the flag on the row you want to edit
if the flag is true then
inform the user that they cannot edit that row at the moment
else
set the flag to true
release the lock
when saving the record, set the flag back to false
# Mark Harrison : SQL Server does not support that syntax (SELECT ... FOR UPDATE).
The SQL Server equivalent is the SELECT statement hint UPDLOCK.
See SQL Server Books Online for more information.
-first create filed (update time) to store last update record
-when any user select record save select time,
compare between select time and update time field if( update time) > (select time) that mean another user update this record after select record
SELECT FOR UPDATE and equivalents are good providing you hold the lock for a microscopic amount of time, but for a macroscopic amount (e.g. the user has the data loaded and hasn't pressed 'save' you should use optimistic concurrency as above. (Which I always think is misnamed - it's more pessimistic than 'last writer wins', which is usually the only other alternative considered.)
Another option is to test that the values in the record that you are changing are the still the same as they were when you started:
SELECT
customer_nm,
customer_nm AS customer_nm_orig
FROM demo_customer
WHERE customer_id = #p_customer_id
(display the customer_nm field and the user changes it)
UPDATE demo_customer
SET customer_nm = #p_customer_name_new
WHERE customer_id = #p_customer_id
AND customer_name = #p_customer_nm_old
IF ##ROWCOUNT = 0
RAISERROR( 'Update failed: Data changed' );
You don't have to add a new column to your table (and keep it up to date), but you do have to create more verbose SQL statements and pass new and old fields to the stored procedure.
It also has the advantage that you are not locking the records - because we all know that records will end up staying locked when they should not be...
The database will do this for you. Look at "select ... for update", which is designed just for this kind of thing. It will give you a write lock on the selected rows, which you can then commit or roll back.
With me, the best way i have a column lastupdate (timetamp datatype).
when select and update just compare this value
another advance of this solution is that you can use this column to track down the time data has change.
I think it is not good if you just create a colum like isLock for check update.