I'm helping with a deadlock hunt. The environment: Tomcat 5.5, Java 5, Microsoft SQL Server 2008, jTDS (replacing an old driver). We have a legacy connection pool.
The database code always follows this scheme:
connection = connectionPool.getConnection(); // 1
boolean previousAutoCommitStatus = connection.getAutoCommit(); // 2
connection.setAutoCommit(false); // 3
// ... use the connection ...
// execute prepared statement 4
// execute prepared statement 5
// execute prepared statement 6
connection.commit(); // 7
connection.setAutoCommit(previousAutoCommitStatus); // 8
connectionPool.releaseConnection(connection); // 9
While we hunt the bug (pardon: the software defect) I was wondering: how does the driver work? My guess: whatever I do between (3) and (7) is queued by the driver/the DBMS. Only when I connection.commit() the DBMS begins a new transaction, acquires every lock the operations need (I hope that it is smart enough to lock the smaller possible set of objects), executes the statements and releases the lock, thus closing the transaction.
Or is it that as soon as I execute a prepared statement the DBMS locks the table?
EDIT: What I want to understand is if "commit()" translates in a set of SQL statements beginning with "begin trans/lock table" and ending in "commit/unlock table" or if any Java executeStatement() acquires the lock immediately.
TIA
If you are interested in inside details,
On connection.commit of a SQLServer JDBC implementation
Following command is issued
IF ##TRANCOUNT > 0 COMMIT TRAN
##TRANCOUNT = 0 -- no open transaction
##TRANCOUNT = 1 -- 1 open transaction
##TRANCOUNT =10 -- 10 open transaction
On setting autocommit to false,
set implicit_transactions on
These are MS SQLServer specific commands.
According to this resource the transaction starts as soon as you call setAutocommit(false);
I think it might still be driver dependant but this will be typical. See also MSDN which says the same.
//Switch to manual transaction mode by setting
//autocommit to false. Note that this starts the first
//manual transaction.
con.setAutoCommit(false);
connection.setAutoCommit(false);
Triggers "BEGIN TRAN" on DB Server and
connection.commit();
Would trigger "COMMIT TRAN"
If you want to prevent locks between these two statements, set connection's isolation level to "Read Uncommited". You will have to ensure that it is acceptable in this scenario.
setTransactionIsolation(Connection.TRANSACTION_READ_UNCOMMITTED);
What exactly goes on depends entirely on the driver implementation, so you need to check the documentation for the driver you are using to get a definitive answer.
setAutoCommit(false) does not necessarily begin the transaction on the database, and the statements do not necessarily execute on the database or even acquire the locks when you call the execute function in your code, as you surmised. That being said, as far as I know, shared (read) locks are ordinarily acquired when the statement is executed; update locks, when commit is called. You may be hitting a conversion deadlock (which happens when multiple statements's read locks on a given object are waiting to be converted to write locks), I'd check if your update statements have nested selects in them that may lead to such a lock.
Related
We have a number of configuration scripts which are a bit complicated. Those scripts are using a number of available stored procedure in our database to insert the configuration data. If any script attempts to insert invalid data, stored procedures will make a call to
RAISERROR #msg, 16, 1
And then the SP will return. The SPs start their own named transactions and they will rollback/commit the named transaction, however, LB doesn't detect the error raised and it takes the execution as a successful execution. But we don't want to continue if that happens.
failOnError in the changeSets are set to true but LB still proceeds. Even in DATABASECHANGELOG that failed changeset marked as executed as if it was successful.
We also tried removing nested transactions (named transactions), no luck.
We removed the name from the transaction and just using BEGIN TRAN, the LB execution stops at the incorrect script but the problem is, LB fails to commit its own transaction and can't release the lock so it remains LOCKED.
Is there anyway to tell LB that an error happened and make it stop?
We are using Liquibase 3.5.0. and Microsoft SQL Server
=== EDIT
So after debugging Liquibase, we found two things:
When connected to MS SQL Server, if a RAISERROR occurs and there are also resultsets in the script, it won't throw exception unless we make a call to statement.getMoreResults(). The same thing happens with Sybase (we tested it with Sybase too). So we thought maybe in LB, after executing the statement we need to make a call to getMoreResults() until it throws exception or it returns false which means no error happened.
A script makes a call to a stored procedure. The stored procedure, has 'BEGIN TRAN' and at the end it either COMMIT or ROLLBACK. If a rollback occurs, it also does RAISERROR. Note that our scripts don't do any update/insert, they are only providing the data in a temp table, so we don't do transaction handling in our scripts. In this scenario, consider we added code to make a call to getMoreResults(), the exception is throws correctly but then in LB, the executor tries to database.rollback() and then later again in StandardLockService, before releasing the lock, it tries to database.rollback() which ends in exception because our SP has rolled back the transaction already. This last rollback in LB, causes the error raised by JDBC to be swallowed and as the result not only do we see the error that caused it but also the lock remained unreleased and this is the most concern because even if we re-run the script and fix it, the lock hasn't been released and we need to do it manually.
One may argue that our transaction handling is not correct but all I am trying to say is that releasing lock should not be affecting if our script is incorrect. LB should be releasing the lock and throw exception or continue if a script/changeset is not run successfully.
If anybody is facing this too: In my case I had a very complex SQL script only for MS SQL Server. This also failed to stop the execution of the LB changes if an error occures in the SQL script, anyway if I use RAISERROR or THROW.
Things I need to do, to get it to work:
remove (or comment) all places where resultsets were created (SELECT)
start the SQL script with "SET NOCOUNT ON;" to avoid results from
insert or update (... lines affected)
end the SQL script with "SET NOCOUNT OFF;" to enable LB to work properly just after executing the SQL script (set EXECTYPE)
Use the precondition https://docs.liquibase.com/concepts/advanced/preconditions.html
create another changeset and check for the execution result before proceeding to next.
I have a situation where I need to wrap an update T-SQL in a stored procedure (sp_update_queue) inside a transaction. But I'm wondering what would happen if you have two threads using the same connection but executing different queries and one rolls back a transaction it started.
For example ThreadA called sp_update_queue to update table QUEUED_TASKS but before sp_update_queue commits/rollback, transaction ThreadB executes some other update or insert SQL on a different table, say CUSTOMERS. Then after ThreadB has finished, sp_update_queue happens to encounter an error and calls rollback.
Because they are both using the same connection would the rollback also rollback changes made by ThreadB?, regardless of whether ThreadB made its changes within a transaction or not.
Each thread which acquire the resource first, will lock that resource(if you have suitable isolation level), so the second thread will wait for the required resource.
Note:each thread will have their own SessionId.
UPDATED
In your scenario, however both of the threads are using same connection, but do not use any common resources(ThreadA is dealing with table X and ThreadB is dealing with table Y). So commit or rollback of each Thread(Thread A or B) does not impact the other one.
Read more about Isolation Level
I feel like I should know this, but I can't find anything that specifically outlines this, so here goes.
The documentation for SQL Server describes REPEATABLE READ as:
Specifies that statements cannot read data that has been modified but
not yet committed by other transactions and that no other transactions
can modify data that has been read by the current transaction until
the current transaction completes
This makes sense, but what actually happens when one of these situation arises? If, for example, Transaction A reads row 1, and then Transaction B attempts to update row 1, what happens? Does Transaction B wait until Transaction A has finished and then try again? Or is an exception thrown?
REPEATABLE READ takes S-locks on all rows that have been read by query plan operators for the duration of the transaction. The answer to your question follows from that:
If the read comes first it S-locks the row and the write must wait.
If the write comes first the S-lock waits for the write to commit.
Under Hekaton it works differently because there are no locks.
I have a question but I can never get a clear answer. Any stored
procedure that used a transaction that I have looked at up until my recent job always had a commit transaction + a roll back in case of error. However I have seen a lot of code
at my new job that just has a begin transaction and then a commit at the end with no roll back. I understand why you would use a transaction with a rollback but why would you want to begin a transaction with no roll back? Is it so when you run that code you want to lock the table up so no values can be changed why your code is updating? If so why would you not want the added security of a roll back in case something goes wrong? Is this proper use of the transaction statement? Any thoughts or ideas would be great!
For Example:
BEGIN TRANSACTION [Tran1]
INSERT INTO [Test].[dbo].[T1]
([Title], [AVG])
VALUES ('Tidd130', 130), ('Tidd230', 230)
UPDATE [Test].[dbo].[T1]
SET [Title] = N'az2' ,[AVG] = 1
WHERE [dbo].[T1].[Title] = N'az'
COMMIT TRANSACTION [Tran1]
GO
shouldn't this code be using a roll back syntax for proper use of the begin transaction statement?
The idea is that if that set of transactions needs to be "all or nothing", wrapping the lot in a transaction is the way to ensure that is what will happen. You're not seeing an explicit rollback because that's not what they're guarding against. Imagine the ff scenario with your contrived example:
The insert happens
The server crashes (or the log fills up or some other external reason why things can't continue) before the update can happen
If they're both wrapped in the same transaction, the insert won't be reflected in the table data. Which is the desired behavior.
When transactions are not explicitly declared, SQL Server will automatically BEGIN and COMMIT a TRANSACTION for each command. This frees up each command's lock as soon as the command executes.
When executing multiple commands inside a single transaction (as in the example you posted), locks from all commands are held until the transaction is committed.
Depending on the desired behavior, the script you posted may be correct. However, I would be cautious to ensure that the developer did not mistakenly believe that the transaction would be automatically rolled back on error. If that behavior is desired, you do indeed need to explicitly ROLLBACK or SET XACT_ABORT ON
You use transaction when you need the outcome to be atomic, you would see this alot in financial related procedures where you are gravely worried about data acid consistency . Otherwise it is not necessary and introduces a great deal of locking overhead. There is a good question here and here that goes into great depth.
Edit
The takeaway point is if the procedure is a all or none and must either succeed or fail the correct decision is to use a transaction. If the procedure is not a all or none transaction such as simple insert update etc using a transaction is a) unnecessary and b) can introduce an undue performance overhead due to additional locking.
I did not find explicit sqlite locking commands before inserting or updating rows into the table. Does sqlite handle the locking mechanism on it own?
The pager module described in http://sqlite.org/lockingv3.html handles the locking mechanism. But I am not sure if there are any commands that the user can use to explicitly lock the tables. Please advice.
Thanks
As far as I know there are no dedicated sqlite commands to control locking. However you can get sqlite to lock the database using create transaction. For instance:
BEGIN IMMEDIATE TRANSACTION;
...
COMMIT TRANSACTION;
BEGIN EXCLUSIVE TRANSACTION;
...
COMMIT TRANSACTION;
If you read the documentation I linked you should get a better idea on the difference between IMMEDIATE & EXCLUSIVE transactions.
It might be worth noting that the locks in sqlite apply to the whole database and not just individual tables, unlike the LOCK TABLE statement in other sql databases.
SQLite does whatever locking is necessary in order to implement the transaction scheme that your SQL statements describe. In particular, if you don't describe any then you get auto-commit behavior, with a lock held for the duration of each statement and then dropped as the statement finishes. Should you need longer transactions (often true!) then you ask for them explicitly with BEGIN TRANSACTION (often shortened to BEGIN) and finish with COMMIT TRANSACTION (or ROLLBACK TRANSACTION). The transaction handling is frequently wrapped for you by your language interface (as this makes it considerably easier to get right, coupling the transaction lifetime to a code block or method call) but at the base level, it comes down to BEGIN/COMMIT/ROLLBACK.
In short, you've got transactions. Locks are used to implement transactions. You don't have raw locks (which is a good thing; they're rather harder to get right than you might think from first glance).