I just want to know what happen if a table is being truncated by another connection and the other connection is retrieving the data out of it?
Let say, the first connection is retrieving thousands of rows, then after a few seconds another connection is truncating the said table. does the first connection locks the table and restrict the second connection from truncating until the first connection is done or otherwise?
I know the use of #temptables to avoid this complicated scenario but I want to know how SQL Server handle this kind of situation.
Thank you.
Actually it is the lock mechanism that controls the different tasks:
Let's say, connection 1 reads tableX, and connection2 also reads tableX, then both connection hold the lock called shared lock, and they are compatible with each other, so no prob.
Another scenario, connection 1 reads the tableX, so it holds the shared lock; then connetion 2 tries to update or delete the rows, then what happens is that it waits for the first connection 1 to finish reading, then able to continue.
does the first connection locks the table and restrict the second
connection from truncating until the first connection is done or
otherwise?
Yes, exactly so.
It's impossible to truncate a table while some other session does a select from it.
It does not depend on transaction isolation level because table truncation needs Sch-M on a table and this lock is incompatible with all the other locks.
In case of read committed the first session already has IS on a table and this conflicts with Sch-M.
In case of read uncommitted the first session has Sch-S on a table that conflicts with Sch-M.
In case of read committed snapshot/snapshot the reading session also takes Sch-S lock that conflicts with Sch-M so in every case the truncate operation waits the select session to release its lock on a table level.
Related
I am having a nodejs program which uses sequelize to create tables and insert data based on it.
Now, in future we are going to have multiple instances of the program and so we don't want multiple instances to read from the table during program startup so that only one instance can do the setup thing if required and other instance shouldn't get 'any access' to the table until the first instance has completed it's work.
I have checked 'transaction locking' - shared and exclusive but both of them seems to be giving reading access to the tables which I don't want.
My requirement is specifically that once a transaction acquires lock on a table, other transaction shouldn't be able to read from that table unless first one has completed it's work. How can I do this?
In MySQL use LOCK TABLES to lock an entire table.
In postgresql LOCK TABLE whatever IN EXCLUSIVE MODE; does the trick.
For best results have your app, when starting, look for a particular table. Do something simple and fast such as SELECT id FROM whatever LIMIT 1; to probe whether the table exists. If your app gets an exception because the table isn't there, then do
CREATE TABLE whatever ....;
LOCK TABLES whatever WRITE;
from the app creating the table. It blocks access to the table from all instances of your app except the one that gets the LOCK.
Once your table is locked, the initial SELECT I suggested will block from other clients. There's a possible race condition if two clients try to create the table more-or-less concurrently. But the extra CREATE TABLE will throw an exception.
Note: if you LOCK more than one table, and it's possible to run the code from more than one instance of the app, always always lock the tables in the same order, or you have the potential for a deadlock.
As documented in the manual the statement to lock a table is, LOCK TABLE ...
If you lock a table in exclusive mode, then no other access is allowed - not even a SELECT. Exclusive mode is the default:
If no lock mode is specified, then ACCESS EXCLUSIVE, the most restrictive mode, is used.
The manual explains the different lock modes:
ACCESS EXCLUSIVE
This mode guarantees that the holder is the only transaction accessing the table in any way.
In a database, we would not like the table to be dropped during we are modifying a row in this table. Per my understanding, a read lock on table + a write lock on row when write a row in table should be enough(based on that a write lock is needed when drop the table), why do we need a intent lock in this case? seems many databases using intent lock which confused me very much. I think pthread_rwlock should be enough.
I read here that they only exists for performance. Imagine you want to drop a table but you would have to check for every row if its locked or not - that would be time consuming, and you would have to lock every row that you checked.
Heres a citation from the blog post:
From a technical perspective the Intent Locks are not really needed by
SQL Server. They have to do with performance optimization. Let’s have
a look on that in more detail. With an Intent Lock SQL Server just
indicates at a higher level within the Lock Hierarchy that you have
acquired a Lock somewhere else. A Intent Shared Lock tells SQL Server
that there is a Shared Lock somewhere else. A Intent Update or Intent
Exclusive Lock does the same, but this time SQL Server knows that
there is an Update Lock or an Exclusive Lock somewhere. It is just an
indication, nothing more.
But how does that indication help SQL Server with performance
optimization? Imagine you want to acquire an Exclusive Lock at the
table level. In that case, SQL Server has to know if there is an
incompatible lock (like a Shared or Update Lock) somewhere else on a
record. Without Intent Locks SQL Server would have to check every
record to see if an incompatible lock has been granted.
But with an Intent Shared Lock on the table level, SQL Server knows
immediately that a Shared Lock has been granted somewhere else, and
therefore an Exclusive Lock can’t be granted at the table level.
That’s the whole reason why Intent Locks exist in SQL Server: to allow
efficient checking if an incompatible lock exists somewhere within the
Lock Hierarchy. Quite easy, isn’t it?
read lock on table + a write lock on row
This would break meaning of the read lock on the table.
Assume concurrent SELECT operation, which expects unmodified table during execution. This operation will take read lock on the table ... and it will succeed in your implementation. This is bad, as table is actually modified during row modification.
Instead, follow locks combination is used for modify row in the table:
IX(Intent eXclusive) on table + X(eXclusive, similar to "write lock") on row
This combination is compatible (that is, can be executed concurrently) with modification of another row, but it is incompatible with
S(Share, similar to "read lock") on table
used by SELECT.
Locks compatibility table can be found, e.g., on wiki.
One of the conclusions today is "intent lock can lock a parent node AND all its children nodes in a read only mode in a cheaper/safer way".
Take an example for making a table read only case, how to lock it in S-X mode?
We lock the table in S mode, then user still can modify the rows with S(table) + W(row) way. to avoid that, we need to lock every row in a S mode to make sure rows will not be updated. The cost is so huge, and it has a bug that user can insert new rows as well. -- cost too much and not safe.
We lock the table in X mode, How other can read the rows (S on table + S on row), no way, since mode_X on table blocked MODE_S on table. That's not read only.
The right solution with the intent lock is:
Lock the table in MODE_S. that's all!
any intention to modify the rows needs to take a MODE_IX lock on the table, but it is blocked by MODE_S. the solution is cheap/efficient and safe!
Here is my scenario why I need a row lock across transactions..
change the columns value to 5 (in SQL Server)
change the columns value to 5 (in another resource, this can be a file or etc.)
Of course it's the case when everything is gone well. but if any problem occurs while doing the second change operation, I need to rollback the first change. And also while doing the second change, nobody should be allowed to read or to write this row in SQL Server.
So I need to do that
lock the column
change the columns value to 5 (in SQL Server)
change the columns value to 5 (in another resource)
if the above change is successfully done
commit the column
else
rollback the column
unlock the column
And I also need something for the murphy case. If I cannot reach the database after locking the row (in order to unlock or to rollback), it should be unlocked in a few seconds.
Is it possible to have something to do that in SQL Server or what ?
Read up on distributed transactions and a compensating ressource manager. THen you realize you can do all that in ONE transaction, managed by your transaction coordinator.
I have a server application, and a database. Multiple instances of the server can run at the same time, but all data comes from the same database (on some servers it is postgresql, in other cases ms sql server).
In my application, there is a process that is performed which can take hours. I need to ensure that this process is only executed one at a time. If one server is processing, no other server instance can process until the first one has completed.
The process depends on one table (let's call it 'ProcessTable'). What I do is, before any server starts the hour-long process, I set a boolean flag in the ProcessTable which indicates that this record is 'locked' and is being processed (not all records in this table are processed / locked, so I need to specifically mark each record which is needed by the process). So when the next server instance comes along while the previous instance is still processing, it sees the boolean flags and throws an exception.
The problem is, that 2 server instances might both be activated at nearly the same time, and when both check the ProcessTable, there may not be any flags set, but both servers are actually in the process of 'setting' the flags but since the transaction hasn't yet commited for either process, neither process will see the locking done by the other process. This is because the locking mechanism itself may take a few seconds, so there is that window of opportunity where 2 servers might still be able to process at the same time.
It appears that what I need is a single record in my 'Settings' table which should store a boolean flag called 'LockInProgress'. So before even a server can lock the needed records in the ProcessTable, it first must make sure that it has full rights to do the locking by checking the 'LockInProgress' column in the Settings table.
So my question is, how do I prevent two servers from both modifying that LockInProgress column in the settings table, at the same time... or am I going about this in the wrong manner?
Please note that I need to support both postgresql and ms sql server as some servers use one database, and some servers use the other.
Thanks in advance...
How about obtaining a lock on the record first and then update the record to show "locked". This would avoid the 2nd instance to get a lock successfully and thereby the update of record fails.
The point is to make sure the lock and update as one atomic step.
Make a stored procedure that hands out the lock, and run it under 'serializable' isolation. This will guarantee that one and only one process can get at the resource at any given time.
Note that this means that the second process trying to get at the lock will block until the first process releases it. Also, if you have to get multiple locks in this manner, make sure that the design of the process guarantees that the locks will be acquired and released in the same order. This will avoid deadlock situations where two processes hold resources while waiting for each other to release locks.
Unless you can't deal with your other processes blocking this would probably be easier to implement and more robust than attempting to implement 'test and set' semantics.
I've been thinking about this, and I think this is the simplest way of doing things; I just execute a command like this:
update settings set settingsValue = '333' where settingsKey = 'ProcessLock' and settingsValue = '0'
'333' would be a unique value which each server process gets (based on date/time, server name, + random value etc).
If no other process has locked the table, then the settingsValue would be = to 0, and that statement would adjust the settingsValue.
If another process has already locked the table, then that statement becomes a no-op, and nothing get's modified.
I then immediately commit the transaction.
Finally, I requery the table for the settingsValue, and if it is the correct value, then our lock succeeded and we continue on, otherwise an exception is thrown, etc. When we're done with the lock, we reset the value back down to 0.
Since I'm using SERIALIZATION transaction mode, I can't see this causing any issues... please correct me if I'm wrong.
I've got in an ASP.NET application this process :
Start a connection
Start a transaction
Insert into a table "LoadData" a lot of values with the SqlBulkCopy class with a column that contains a specific LoadId.
Call a stored procedure that :
read the table "LoadData" for the specific LoadId.
For each line does a lot of calculations which implies reading dozens of tables and write the results into a temporary (#temp) table (process that last several minutes).
Deletes the lines in "LoadDate" for the specific LoadId.
Once everything is done, write the result in the result table.
Commit transaction or rollback if something fails.
My problem is that if I have 2 users that start the process, the second one will have to wait that the previous has finished (because the insert seems to put an exclusive lock on the table) and my application sometimes falls in timeout (and the users are not happy to wait :) ).
I'm looking for a way to be able to have the users that does everything in parallel as there is no interaction, except the last one: writing the result. I think that what is blocking me is the inserts / deletes in the "LoadData" table.
I checked the other transaction isolation levels but it seems that nothing could help me.
What would be perfect would be to be able to remove the exclusive lock on the "LoadData" table (is it possible to force SqlServer to only lock rows and not table ?) when the Insert is finished, but without ending the transaction.
Any suggestion?
Look up SET TRANSACTION ISOLATION LEVEL READ COMMITTED SNAPSHOT in Books OnLine.
Transactions should cover small and fast-executing pieces of SQL / code. They have a tendancy to be implemented differently on different platforms. They will lock tables and then expand the lock as the modifications grow thus locking out the other users from querying or updating the same row / page / table.
Why not forget the transaction, and handle processing errors in another way? Is your data integrity truely being secured by the transaction, or can you do without it?
if you're sure that there is no issue with cioncurrent operations except the last part, why not start the transaction just before those last statements, Whichever they are that DO require isolation), and commit immediately after they succeed.. Then all the upfront read operations will not block each other...