I am trying to understand how some code with a transaction will work if run by multiple threads. There is a unique 5 character ID that needs to be inserted per record and must be unique for each "job" which is checked by a unique index.
We currently have code that is single threaded and catches duplicate errors, creates a new random ID and tries again. However we are considering moving the ID creation to a later step of processing which is multiple threaded. There is also code in the transaction that uses the random ID but we want to be sure it has the ID that went into the record and that if the code at the end of the transaction fails that the record update does not occur.
What I am trying to understand is if the transaction can only fail when the update query is run and if another error is thrown by the additional code, or if it could possibly fail on commit after the additional code has run.
Here is the code outline....
until transaction completes successfully
try
begin transaction
create randomid
update an existing record with randomid
do something that should only be done once
commit transaction
catch duplicate id error
I have tried some tests that didn't show a problem but not sure if my tests were sufficient so would like to better understand what would be going on in this situation.
Also, I assume the update queries can't happen in parallel but from my testing it appears that the additional code does. Is that always the case or does it depend on other considerations?
If you have a unique index on a column and you update this column, SQL Server will put a lock on the index (key range). This will prevent all other processes from updating the same record. They will have to wait for the lock to be released and will be blocked until that time. Once the lock is released, the unique index will prevent them from updating the table.
As other processes have to wait, depending on your timeout setting and the length of the execution, they may time out. You may wish to handle that in addition to handling duplicate exception.
If your processes try to insert different values into the column with the unique index, that may happen in parallel, depending on other locks on the table.
Related
I have been asked to check a production issue for which I need help. I am trying to understand the isolation levels and different locks available in SQL server.
I have a table JOB_STATUS having columns job_name (string, primary key), job_status (string), is_job_locked (string)
Sample data as below:
job_name
job_status
is_job_locked
JOB_A
INACTIVE
N
JOB_B
RUNNING
N
JOB_C
SUCCEEDED
N
JOB_D
RUNNING
N
JOB_E
INACTIVE
N
Multiple processes can update the table at the same time by calling a stored procedure and passing the job_name as input parameter. It is fine if two different rows are getting updated by separate processes at the same time.
BUT, two processes should not update the same row at the same time.
Sample update query is as follows:
update JOB_STATUS set is_job_locked='Y' where job_name='JOB_A' and is_job_locked='N';
Here if two processes are updating the same row, then one process should wait for the other one to complete. Also, if the is_job_locked column value is changed to Y by one process, then the other process should not update it again (which my update statement should handle if locking is proper).
So how can I do this row level locking and make sure the update query reads the latest data from the row before making an update using a stored procedure.
Also, would like to get the return value whether the update query updated the row or it did not as per the condition, so that I can use this value in my further application flow.
RE: "Here if two processes are updating the same row, then one process should wait for the other one to complete. "
That is how locking works in SQL Server. An UPDATE takes an exclusive lock on the row -- where "exclusive" means the English meaning of the word: the UPDATE process has excluded (locked out) all other processes while it is running. The other processes now wait for the UPDATE to complete. This includes READ processes for transaction isolation levels READ COMMITTED and above. When the UPDATE lock is released, then the next statement can access the value.
IF what you are looking for is that 2 processes cannot change the same row in a single table at the same time, then SQL Server does that for you out of the box and you do not need to add your own "is_job_locked" column.
However, typically an is_job_locked column is used to control access beyond a single table. For example, it may be used to prevent a second process from starting a job that is already running. Process A would mark is_job_locked, then start the job. Process B would check the flag before trying to start the job.
I did not had to use explicit lock or provide any isolation level as it was a single update query in the stored procedure.
At a time SQL server is only allowing one process to update a row which is then read committed by second process and not updated again.
Also, I used ##ROWCOUNT to get the No. of rows updated. My issue is solved now.
Thanks for the answers and comments.
I have a conditional split in an SSIS job that inserts or updates based on the CDC operation. The deletes dont actually delete, they just mark the row deleted (so it is also an update statement).
This is what it looks like:
And the error message associated with the red x is
"Transaction (Process ID 67) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.".
I tried to put an additional input arrow in there, so it only runs one update at a time, but it won't let me.
I suspect FactWaybillTrans is has table lock checked in the destination. That's usually what you want when loading large amounts of data. However, since you also want to update the same thing, that is going to conflict with the lock and thus, you get a deadlock. Even without checking table lock, the default lock could escalate to a full lock.
I would look to stage my updates to a table (stage.CDCWaybillUpdates) and then have an Execute SQL Task fire after the Data Flow. Much cleaner and no opportunity for deadlocks.
You might be able to fake it but it'd be totally unreliable. Add a Sort Operation between the NumRowsUpdated and Update. That might be able to induce enough drag so that the OLE DB Destination finishes and releases its lock before the update begins firing. If it doesn't slow it down enough, then sort the same data in the opposite direction. Terrible, hackish approach but sometimes you have to do the dumb.
I want to place DB2 Triggers for Insert, Update and Delete on DB2 Tables heavily used in parallel online Transactions. The tables are shared by several members on a Sysplex, DB2 Version 10.
In each of the DB2 Triggers I want to insert a row into a central table and have one background process calling a Stored Procedure to read this table every second to process the newly inserted rows, ordered by sequence of the insert (sequence number or timestamp).
I'm very concerned about DB2 Index locking contention and want to make sure that I do not introduce Deadlocks/Timeouts to the applications with these Triggers.
Obviously I would take advantage of DB2 Features to reduce locking like rowlevel locking, but still see no real good approach how to avoid index contention.
I see three different options to select the newly inserted rows.
Put a sequence number in the table and the store the last processed sequence number in the background process. I would do the following select Statement:
SELECT COLUMN_1, .... Column_n
FROM CENTRAL_TABLE
WHERE SEQ_NO > 'last-seq-number'
ORDER BY SEQ_NO;
Locking Level must be CS to avoid selecting uncommited rows, which will be later rolled back.
I think I need one Index on the table with SEQ_NO ASC
Pro: Background process only reads rows and makes no updates/deletes (only shared locks)
Neg: Index contention because of ascending key used.
I can clean-up processed records later (e.g. by rolling partions).
Put a Status field in the table (processed and unprocessed) and change the Select as follows:
SELECT COLUMN_1, .... Column_n
FROM CENTRAL_TABLE
WHERE STATUS = 'unprocessed'
ORDER BY TIMESTAMP;
Later I would update the STATUS on the selected rows to "processed"
I think I need an Index on STATUS
Pro: No ascending sequence number in the index and no direct deletes
Cons: Concurrent updates by online transactions and the background process
Clean-up would happen in off-hours
DELETE the processed records instead of the status field update.
SELECT COLUMN_1, .... Column_n
FROM CENTRAL_TABLE
ORDER BY TIMESTAMP;
Since the table contains very few records, no index is required which could create a hot spot.
Also I think I could SELECT with Isolation Level UR, because I would detect potential uncommitted data on the later delete of this row.
For a Primary Key index I could use GENERATE_UNIQUE,which is random an not ascending.
Pro: No Index hot spot and the Inserts can be spread across the tablespace by random UNIQUE_ID
Con: Tablespace scan and sort on every call of the Stored Procedure and deleting records in parallel to the online inserts.
Looking forward what the community thinks about this problem. This must be a pretty common problem e.g. SAP should have a similar issue on their Batch Input tables.
I tend to favour Option 3, because it avoids index contention.
May be there is still another solution in your minds out there.
I think you are going to have numerous performance problems with your various solutions.
(I know premature optimazation is a sin, but experience tells us that some things are just not going to work in a busy system).
You should be able to use DB2s autoincrement feature to get your sequence number, with little or know performance implications.
For the rest perhaps you should look at a Queue based solution.
Have your trigger drop the operation (INSERT/UPDATE/DELETE) and the keys of the row into a MQ queue,
Then have a long running backgound task (in CICS?) do your post processing as its processing one update at a time you should not trip over yourself. Having a single loaded and active task with the ability to batch up units of work should give you a throughput in the order of 3 to 5 hundred updates a second.
In my application, I have couple of threads that execute some logic.
At the end they adding new row to some table.
Before adding the new row, they check if a previous entry with the same details does not already exist. If one found - they updating instead of adding.
The problem is when some thread A do the check, see that no previous entity with the same details exist, and just before he add a new row, the thread B search the DB for the same entity. Thread B see that no such entity exist so he add new row too.
The result is that there are two rows with the same data in the table.
Note: no table key violated, because the thread get the next sequence just before adding the row, and the table key is some ID that does not related to the data.
Even if I will change the table key so it will be a combination of the data, It will prevent two rows with the same data, but will cause a DB error when the second thread will try to add the row.
Thank you in advance for the help, Roy.
You should be using a queue, possibly blocking queue. Threads A and B (producers) would add objects to the queue and another thread C (consumer) would poll the queue and remove the oldest object from the queue persisting it to the DB. This will prevent the problem when both A and B in the same time want to persist equal objects
You speak of "rows" so presumably this is a SQL database?
If so, why not just use transactions?
(Unless the threads are sharing a database connection, in which case a mutex might help, but I would prefer to give each thread a separate connection.)
I would recommend avoid locking in the client layer. Synchronized only works within one process, later you may scale so that your threads are across several JVMs or indeed machines.
I would enforce uniqueness in the DB, as you suggest this will then cause an exception for the second inserter. Catch that exception and do an update if that's the business logic you need.
But consider this argument:
Sometimes either of the following sequences may occur:
A insert Values VA, B updates to values VB.
B insert VB, A updates to VA.
If the two threads are racing either of these two outcomes VA or VB is equally valid. So you can't distinguish the second case from A inserts VA and B just fails!
So in fact there may be no need for the "fail and then update" case.
I think this is a job for SQL constraints, namely "UNIQUE" on the set of columns that have the data + the appropriate error handling.
Most database frameworks (Hibernate in Java, ActiveRecord etc in Ruby) have a form of optimistic locking. What this means is that you execute each operation on the assumption that it will work without conflict. In the special case where there is a conflict, you check this atomically at the point where you do the database operation, throw an exception, or error return code, and retry the operation in your client code after requerying etc.
This is usually implemented using a version number on each record. When a database operation is done, the row is read (including the version number), the client code updates the data, then saves it back to the database with a where clause specifying the primary key ID AND the version number being the same as it was when it was read. If it is different - this means another process has updated the row, and the operation should be retried. Usually this means re-reading the record, and doing that operation again on it with the new data from the other process.
In the case of adding, you would also want a unique index on the table, so the database refuses the operation, and you can handle that in the same code.
Pseudo code would look something like
do {
read row from database
if no row {
result_code = insert new row with data
} else {
result_code = update row with data
}
} while result_code != conflict_code
The benefit of this is that you don't need complicated synchronization/locking in your client code - each thread just executes in isolation, and uses the database as the consistency check (which it is very quick, and good at). Because you're not locking on some shared resource for every operation, the code can run much faster.
It also means that you can run multiple separate operating system processes to split the load and/or scale the operation over multiple servers as well without any code changes to handle conflicts.
You need to wrap the calls to check and write the row in a critical section or mutex.
With a critical section, interrupts and thread-switching are disabled while you perform the check and write, so both threads can't write at once.
With a mutex, the first thread would lock the mutex, perform its operations, then unlock the mutex. The second thread would attempt to do the same but the mutex lock would block until the first thread released the mutex.
Specific implementations of critical section or mutex functionality would depend on your platform.
You need to perform the act of checking for existing rows and then updating / adding rows inside a single transaction.
When you perform your check you should also acquire an update lock on those records, to indicate that you are going to write to the database based on the information that you have just read, and that no-one else should be allowed to change it.
In pseudo T-SQL (for Microsoft SQL Server):
BEGIN TRANSACTION
SELECT id FROM MyTable WHERE SomeColumn = #SomeValue WITH UPDLOCK
-- Perform your update here
END TRANSACTION
The update lock wont prevent people reading from those records, but it will prevent people from writing anything which might change the output of your SELECT
Multi Threading is always mind-bending ^^.
Main thing to do is to delimit the critical resources and critical operations.
Critical resource : your table.
Critical operation : adding yes, but
the whole procedure
You need to lock access to your table from the beginning of the check, until the end of the add.
If a thread attempt to do the same, while another is adding/checking, then he waits until the thread finish its operation. As simple as that.
I've got in an ASP.NET application this process :
Start a connection
Start a transaction
Insert into a table "LoadData" a lot of values with the SqlBulkCopy class with a column that contains a specific LoadId.
Call a stored procedure that :
read the table "LoadData" for the specific LoadId.
For each line does a lot of calculations which implies reading dozens of tables and write the results into a temporary (#temp) table (process that last several minutes).
Deletes the lines in "LoadDate" for the specific LoadId.
Once everything is done, write the result in the result table.
Commit transaction or rollback if something fails.
My problem is that if I have 2 users that start the process, the second one will have to wait that the previous has finished (because the insert seems to put an exclusive lock on the table) and my application sometimes falls in timeout (and the users are not happy to wait :) ).
I'm looking for a way to be able to have the users that does everything in parallel as there is no interaction, except the last one: writing the result. I think that what is blocking me is the inserts / deletes in the "LoadData" table.
I checked the other transaction isolation levels but it seems that nothing could help me.
What would be perfect would be to be able to remove the exclusive lock on the "LoadData" table (is it possible to force SqlServer to only lock rows and not table ?) when the Insert is finished, but without ending the transaction.
Any suggestion?
Look up SET TRANSACTION ISOLATION LEVEL READ COMMITTED SNAPSHOT in Books OnLine.
Transactions should cover small and fast-executing pieces of SQL / code. They have a tendancy to be implemented differently on different platforms. They will lock tables and then expand the lock as the modifications grow thus locking out the other users from querying or updating the same row / page / table.
Why not forget the transaction, and handle processing errors in another way? Is your data integrity truely being secured by the transaction, or can you do without it?
if you're sure that there is no issue with cioncurrent operations except the last part, why not start the transaction just before those last statements, Whichever they are that DO require isolation), and commit immediately after they succeed.. Then all the upfront read operations will not block each other...