How do I achieve the Transaction involving multiple DB operations to >1 tables using iBatis & Spring?
Let me explain in detail:
I have 2 tables A & B with Master-details relationship. [Both tables in single database].
/* Table A: */
a_id [Primary Key]
[plus other columns]
/* Table B: */
b_id [Primary Key]
a_id [Foreign Key = PK of table A]
[plus other columns]
In my Dao I have following methods (I am using iBatis sqlMap toperform DB operations):
insertA();
insertB();
updateA();
updateB();
deleteA();
deleteB();
Each of the above operations are Atomic (& can be called by client & commited in database -via Spring/iBatis).
Up to this point everything WORKS OK! [i.e. I am able to perform INDIVIDUAL insert/update/delete on each table.]
-- NEXT, I need to perform a combination of two of above DB operations as an ATOMIC operation;
Here is what I want to achieve from SVC layer:
start Tranaction
operation on Table-A (via method of Dao class) - op #1
operation on Table-B (via method of Dao class) - op #2
end Transaction
Example1:
start Tranaction
insertA();
insertB();
end Transaction
Example2:
start Tranaction
updateA();
updateB();
end Transaction
Here, if op#2 Fails, I want op#1 also to be Rolled back. i.e. Complete Rollback.
So, I wrote additional method within the Service layer, which calls above DAO methods.
Before running the (Svc) code, I manually [via cmd-line] change some data On database, so that 2nd operation FAILS due to DB Constraints.
Now, op #2 [Table-B] FAILS, but op #1 is commited in DB. i.e. there is NO complete rollback, ONLY PARTIAL rollback.
If op #2 Fails, shouldn't op#1 also Roll back?
Here is what I am using in ApplicationContext.xml:
"DataSourceTransactionManager" [Spring] for Transaction.
iBatis 2.3.x [SqlMapClient]
Spring 3.0
DefaultAutoCommit is set to FALSE.
In "tx:method": [service method from where ATOMIC operation is to be performed)
propagation="REQUIRED" [Tried with other values also, but no use]
rollback-for=Exception-Name-for-which-to-rollback
Is there anything else that needs to be done?
Am I doing something wrong?
Is this correct way or is there a better option?
<
In my opinion, you should consider the data integrity, if op #2 make the system loose data integrity, then it should roll back according to op #1.
To achieve what you want, just make a call to op #1 and #2, wrapper #2 on try/catch block, something like:
try {
start Tranaction ;
//pkA is primary key of A
Object pkA = insertA();
updateA(pkA);
try {
Object pkB = insertB(pkA);
updateB(pkB);
}
catch(Exception e) {
logger.ERROR("Error when inserting and updating B.Ignore. ",e);
}
commit transaction;
}
catch(Exception e) {
logger.ERROR(e);
rollback Transaction;
}
HTH.
Related
I'm new to PDO transaction, I want to know is the rollback() is rollback the database or only the tables inside the beginTransaction().
For example :
$db->beginTransaction();
try{
$stmt = $db->prepare('SELECT * FROM user_detail .......');
$stmt -> bindParam(........);
......
$stmt -> execute();
while($s = $stmt->fetch(PDO::FETCH_ASSOC)){
$update = $db->prepare('UPDATE tableB .......');
$update -> execute();
$update = $db->prepare('UPDATE tableC .......');
$update -> execute();
$delete = $db->prepare('Delete tableD .......');
$delete -> execute();
}
$db->commit();
}
catch(PDOException $e){
$db->rollback();
}
At the same time, there is another query which will update tableC and tableG been submitted when the first query (shown above) still executing. Will the new query execute immediately or after the first query has done.
If both of them execute at the same time, what if there is an error
found and trigger the rollback(), will the second query (update tableC and tableG) rollback if it is done before the rollback start?
Rollback undoes all the changes made by INSERT/UPDATE/DELETE since the beginning of the transaction.
Whether you commit or rollback, that only those changes made within the current transaction, not other changes going on in other sessions. They can commit or rollback independently.
Concurrent updates running in different sessions will run at the same time, unless they are trying to update the same rows (even partially-overlapping sets of rows). One session or the other will get to those rows first, and lock the rows before it updates them. The session that acquired the locks will hold those locks until it either commits or rolls back its transaction.
There's a lot more to learn about locking. It's a very complex subject. You might like my presentation InnoDB Locking Explained with Stick Figures to get started (but it only scratches the surface).
My presentation is specifically about MySQL. You didn't say what brand of RDBMS you're using. All different brands will each have their own quirks regarding transactions and locking.
I am creating a DB synchronization engine using SQL CLR Triggers in Microsoft SQL Server 2012. These triggers do not call a stored procedure or function (and thereby have access to the INSERTED and DELETED pseudo-tables but do not have access to the ##procid).
Differences here, for reference.
This "sync engine" uses mapping tables to determine what the table and field maps are for this sync job. In order to determine the target table and fields (from my mapping table) I need to get the source table name from the trigger itself. I have come across many answers on Stack Overflow and other sites that say that this isn't possible. But, I've found one website that provides a clue:
Potential Solution:
using (SqlConnection lConnection = new SqlConnection(#"context connection=true")) {
SqlCommand cmd = new SqlCommand("SELECT object_name(resource_associated_entity_id) FROM sys.dm_tran_locks WHERE request_session_id = ##spid and resource_type = 'OBJECT'", lConnection);
cmd.CommandType = CommandType.Text;
var obj = cmd.ExecuteScalar();
}
This does in fact return the correct table name.
Question:
My question is, how reliable is this potential solution? Is the ##spid actually limited to this single trigger execution? Or is it possible that other simultaneous triggers will overlap within this process id? Will it stand up to multiple executions of the same and/or different triggers within the database?
From these sites, it seems the process Id is in fact limited to the open connection, which doesn't overlap: here, here, and here.
Will this be a safe method to get my source table?
Why?
As I've noticed similar questions, but all without a valid answer for my specific situation (except that one). Most of the comments on those sites ask "Why?", and in order to preempt that, here is why:
This synchronization engine operates on a single DB and can push changes to target tables, transforming the data with user-defined transformations, automatic source-to-target type casting and parsing and can even use the CSharpCodeProvider to execute methods also stored in those mapping tables for transforming data. It is already built, quite robust and has good performance metrics for what we are doing. I'm now trying to build it out to allow for 1:n table changes (including extension tables requiring the same Id as the 'master' table) and am trying to "genericise" the code. Previously each trigger had a "target table" definition hard coded in it and I was using my mapping tables to determine the source. Now I'd like to get the source table and use my mapping tables to determine all the target tables. This is used in a medium-load environment and pushes changes to a "Change Order Book" which a separate server process picks up to finish the CRUD operation.
Edit
As mentioned in the comments, the query listed above is quite "iffy". It will often (after a SQL Server restart, for example) return system objects like syscolpars or sysidxstats. But, it seems that in the dm_tran_locks table there's always an associated resource_type of 'RID' (Row ID) with the same object_name. My current query which works reliably so far is the following (will update if this changes or doesn't work under high load testing):
select t1.ObjectName FROM (
SELECT object_name(resource_associated_entity_id) as ObjectName
FROM sys.dm_tran_locks WHERE resource_type = 'OBJECT' and request_session_id = ##spid
) t1 inner join (
SELECT OBJECT_NAME(partitions.OBJECT_ID) as ObjectName
FROM sys.dm_tran_locks
INNER JOIN sys.partitions ON partitions.hobt_id = dm_tran_locks.resource_associated_entity_id
WHERE resource_type = 'RID'
) t2 on t1.ObjectName = t2.ObjectName
If this is always the case, I'll have to find that out during testing.
How reliable is this potential solution?
While I do not have time to set up a test case to show it not working, I find this approach (even taking into account the query in the Edit section) "iffy" (i.e. not guaranteed to always be reliable).
The main concerns are:
cascading (whether recursive or not) Trigger executions
User (i.e. Explicit / Implicit) transactions
Sub-processes (i.e. EXEC and sp_executesql)
These scenarios allow for multiple objects to be locked, all at the same time.
Is the ##SPID actually limited to this single trigger execution? Or is it possible that other simultaneous triggers will overlap within this process id?
and (from a comment on the question):
I think I can join my query up with the sys.partitions and get a dm_trans_lock that has a type of 'RID' with an object name that will match up to the one in my original query.
And here is why it shouldn't be entirely reliable: the Session ID (i.e. ##SPID) is constant for all of the requests on that Connection). So all sub-processes (i.e. EXEC calls, sp_executesql, Triggers, etc) will all be on the same ##SPID / session_id. So, between sub-processes and User Transactions, you can very easily get locks on multiple resources, all on the same Session ID.
The reason I say "resources" instead of "OBJECT" or even "RID" is that locks can occur on: rows, pages, keys, tables, schemas, stored procedures, the database itself, etc. More than one thing can be considered an "OBJECT", and it is possible that you will have page locks instead of row locks.
Will it stand up to multiple executions of the same and/or different triggers within the database?
As long as these executions occur in different Sessions, then they are a non-issue.
ALL THAT BEING SAID, I can see where simple testing would show that your current method is reliable. However, it should also be easy enough to add more detailed tests that include an explicit transaction that first does some DML on another table, or have a trigger on one table do some DML on one of these tables, etc.
Unfortunately, there is no built-in mechanism that provides the same functionality that ##PROCID does for T-SQL Triggers. I have come up with a scheme that should allow for getting the parent table for a SQLCLR Trigger (that takes into account these various issues), but haven't had a chance to test it out. It requires using a T-SQL trigger, set as the "first" trigger, to set info that can be discovered by the SQLCLR Trigger.
A simpler form can be constructed using CONTEXT_INFO, if you are not already using it for something else (and if you don't already have a "first" Trigger set). In this approach you would still create a T-SQL Trigger, and then set it as the "first" Trigger using sp_settriggerorder. In this Trigger you SET CONTEXT_INFO to the table name that is the parent of ##PROCID. You can then read CONTEXT_INFO() on a Context Connection in a SQLCLR Trigger. If there are multiple levels of Triggers then the value of CONTEXT INFO will get overwritten, so reading that value must be the first thing you do in each SQLCLR Trigger.
This is an old thread, but it is an FAQ and I think I have a better solution. Essentially it uses the schema of the inserted or deleted table to find the base table by doing a hash of the column names and comparing the hash with the hashes of tables with a CLR trigger on them.
Code snippet below - at some point I will probably put the whole solution on Git (it sends a message to Azure Service Bus when the trigger fires).
private const string colqry = "select top 1 * from inserted union all select top 1 * from deleted";
private const string hashqry = "WITH cols as ( "+
"select top 100000 c.object_id, column_id, c.[name] "+
"from sys.columns c "+
"JOIN sys.objects ot on (c.object_id= ot.parent_object_id and ot.type= 'TA') " +
"order by c.object_id, column_id ) "+
"SELECT s.[name] + '.' + o.[name] as 'TableName', CONVERT(NCHAR(32), HASHBYTES('MD5',STRING_AGG(CONVERT(NCHAR(32), HASHBYTES('MD5', cols.[name]), 2), '|')),2) as 'MD5Hash' " +
"FROM cols "+
"JOIN sys.objects o on (cols.object_id= o.object_id) "+
"JOIN sys.schemas s on (o.schema_id= s.schema_id) "+
"WHERE o.is_ms_shipped = 0 "+
"GROUP BY s.[name], o.[name]";
public static void trgSendSBMsg()
{
string table = "";
SqlCommand cmd;
SqlDataReader rdr;
SqlTriggerContext trigContxt = SqlContext.TriggerContext;
SqlPipe p = SqlContext.Pipe;
using (SqlConnection con = new SqlConnection("context connection=true"))
{
try
{
con.Open();
string tblhash = "";
using (cmd = new SqlCommand(colqry, con))
{
using (rdr = cmd.ExecuteReader(CommandBehavior.SingleResult))
{
if (rdr.Read())
{
MD5 hash = MD5.Create();
StringBuilder hashstr = new StringBuilder(250);
for (int i=0; i < rdr.FieldCount; i++)
{
if (i > 0) hashstr.Append("|");
hashstr.Append(GetMD5Hash(hash, rdr.GetName(i)));
}
tblhash = GetMD5Hash(hash, hashstr.ToString().ToUpper()).ToUpper();
}
rdr.Close();
}
}
using (cmd = new SqlCommand(hashqry, con))
{
using (rdr = cmd.ExecuteReader(CommandBehavior.SingleResult))
{
while (rdr.Read())
{
string hash = rdr.GetString(1).ToUpper();
if (hash == tblhash)
{
table = rdr.GetString(0);
break;
}
}
rdr.Close();
}
}
if (table.Length == 0)
{
p.Send("Error: Unable to find table that CLR trigger is on. Message not sent!");
return;
}
….
HTH
I'm inserting data into multiple tables, and I use the mybatis component to do that. I also need to create a temporary table before I can insert the data. High-level overview is:
Get data to insert
Create temp table
Insert data to temp table
Insert into table1 select x from temp table
Insert into table2 select y from temp table
Steps 2 to 5 should be their own single transaction, in case something fails. I've got this currently:
from(initialEndpoint)
.routeId("database-appender")
.aggregate().expression(constant(true)).completionSize(100).aggregationStrategy(new LinkListAggregator())
.transacted()
.bean(CreateTmpLinksTable.class)
.to("mybatis:prepareLinks?executorType=reuse&statementType=InsertList")
.to("mybatis:insertLinks?executorType=reuse&statementType=InsertList")
.to("mybatis:insertLinkSources?executorType=reuse&statementType=InsertList")
.end()
.log("Wrote at most ${body.size} links to the database")
The CreateTmpLinksTable needs to have access to the current connection, such that the creation of the temporary table does not happen in a different transaction (targeting PostgreSQL, if it matters).
I have this currently:
public class CreateTmpLinksTable {
public void createImportTable(Exchange exchange) throws SQLException {
final Connection conn = exchange.getIn().getHeader("TransactionConnection", Connection.class);
try (final Statement stat = conn.createStatement()) {
stat.execute("CREATE TEMPORARY TABLE tmp_links(" +
"url text, hostname text, service media, service_id bigint, user_id bigint, screen_name text, harvested_at timestamp with time zone, body text" +
") ON COMMIT DROP");
}
}
}
I also haven't setup my transaction manager. My suspicion is I have to get hold of the transaction manager, in order to correctly participate in the transaction.
Questions:
How do I get the transaction manager from a regular bean? Is it just a matter of getting the context, then from the context getting the manager through the registry?
Is there a better way to do what I need? I can see at least one: move all responsibilities into a single bean and do the work there. Any other ways?
NOTE: I'm learning Camel, and I like to do things using only code. Once I know how everything is wired up, then I can transfer that knowledge to Spring.
Q1,
If you can pass the instance of bean to the camel route, you can setup the transaction manager yourself, otherwise you have to use the registry to look up the transaction manager instance.
Q2,
You can wrap the DB update work in a single bean and use the transacted DSL in camel if you have other resources need to be managed.
Sometimes postgresql raise error deadlocks.
In trigger for table setted FOR UPDATE.
Table comment:
http://pastebin.com/L1a8dbn4
Log (INSERT sentences is cutted):
2012-01-26 17:21:06 MSK ERROR: deadlock detected
2012-01-26 17:21:06 MSK DETAIL: Process 2754 waits for ExclusiveLock on tuple (40224,15) of relation 735493 of database 734745; blocked by process 2053.
Process 2053 waits for ShareLock on transaction 25162240; blocked by process 2754.
Process 2754: INSERT INTO comment (user_id, content_id, reply_id, text) VALUES (1756235868, 935967, 11378142, 'text1') RETURNING comment.id;
Process 2053: INSERT INTO comment (user_id, content_id, reply_id, text) VALUES (4071267066, 935967, 11372945, 'text2') RETURNING comment.id;
2012-01-26 17:21:06 MSK HINT: See server log for query details.
2012-01-26 17:21:06 MSK CONTEXT: SQL statement "SELECT comments_count FROM content WHERE content.id = NEW.content_id FOR UPDATE"
PL/pgSQL function "increase_comment_counter" line 5 at SQL statement
2012-01-26 17:21:06 MSK STATEMENT: INSERT INTO comment (user_id, content_id, reply_id, text) VALUES (1756235868, 935967, 11378142, 'text1') RETURNING comment.id;
And trigger on table comment:
CREATE OR REPLACE FUNCTION increase_comment_counter() RETURNS TRIGGER AS $$
DECLARE
comments_count_var INTEGER;
BEGIN
SELECT INTO comments_count_var comments_count FROM content WHERE content.id = NEW.content_id FOR UPDATE;
UPDATE content SET comments_count = comments_count_var + 1, last_comment_dt = now() WHERE content.id = NEW.content_id;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER increase_comment_counter_trigger AFTER INSERT ON comment FOR EACH ROW EXECUTE PROCEDURE increase_comment_counter();
Why it can happens?
Thanks!
These are two comments being inserted with the same content_id. Merely inserting the comment will take out a SHARE lock on the content row, in order to stop another transaction deleting that row until the first transaction has completed.
However, the trigger then goes on to upgrade the lock to EXCLUSIVE, and this can be blocked by a concurrent transaction performing the same process. Consider the following sequence of events:
Txn 2754 Txn 2053
Insert Comment
Insert Comment
Lock Content#935967 SHARE
(performed by fkey)
Lock Content#935967 SHARE
(performed by fkey)
Trigger
Lock Content#935967 EXCLUSIVE
(blocks on 2053's share lock)
Trigger
Lock Content#935967 EXCLUSIVE
(blocks on 2754's share lock)
So- deadlock.
One solution is to immediately take an exclusive lock on the content row before inserting the comment. i.e.
SELECT 1 FROM content WHERE content.id = 935967 FOR UPDATE
INSERT INTO comment(.....)
Another solution is simply to avoid this "cached counts" pattern completely, except where you can prove it is necessary for performance. If so, consider keeping the cached count somewhere other than the content table-- e.g. a dedicated table for the counter. That will also cut down on the update traffic to the content table every time a comment gets added. Or maybe just re-select the count and use memcached in the application. There's no getting round the fact that wherever you store this cached count is going to be a choke point, it has to be updated safely.
I have a few windwos services. They get xml column from Sql server manipulate and update it.
Service A- Gets XML
Service B- Gets XML
Service A- Updates XML (it will be lost)
Service B- Updates XML
I must lock row and I use next Code:
SqlCommand cmdUpdate = new SqlCommand();
cmdUpdate.CommandText = "select MyXML from MyTable with(holdlock,rowlock) where id=#id";
cmdUpdate.Parameters.AddWithValue("#id", id);
using (SqlConnection conn = Helper.GetConnection())
{
cmdUpdate.Connection = conn;
SqlTransaction ts = conn.BeginTransaction();
cmdUpdate.Transaction = ts;
XElement elem = XElement.Parse(cmdUpdate.ExecuteScalar().ToString());
UpdateXElement(elem);
cmdUpdate.Parameters.Clear();
cmdUpdate.CommandText = "update MyTable set MyXML=#xml where id=#id";
cmdUpdate.Parameters.AddWithValue("#id", id);
cmdUpdate.Parameters.AddWithValue("#xml", elem.ToString());
cmdUpdate.ExecuteNonQuery();
ts.Commit();
}
}`
then occurs Deadlocks.
Have you got a better idea, to solve this problem ?
Thanks
The scenario you are describing is not a deadlock. It's a lock contention, in other words, exactly what the locks are for:
Service A- Gets XML - Service A locks XML
Service B- Gets XML - Services B places lock request which waits for service A to release the lock
Service A- Updates XML (it will be lost) - Service A should commit or rollback the transaction to release the lock.
Service B- Updates XML - Service B acquires the lock on the XML and updates it
Service B will be frozen between steps 2 and 3.
This means you should perform these steps as fast as possible.
Update:
You use a HOLDLOCK to lock the row in a transaction.
HOLDLOCK places a shared lock which is compatible with another shared lock but not with update lock placed by UPDATE.
Here's what happens:
Service A places a shared lock on row 1
Service B places a shared lock on row 1
Service A tries to place an update lock on row 1 which is not compatible with the shared lock placed by Service B on step 2. Service A enters wait state (while still holding a shared lock placed on step 1).
Service B tries to place an update lock on row 1 which is not compatible with the shared lock placed by Service A on step 1. Service B enters wait state. DEADLOCK.
There is no point in placing a shared lock in a SELECT clause here. You should place an UPDLOCK in a SELECT clause instead. This will make the transaction locks completely incompatible and either transaction will have to wait for completion of other transactions before acquiring any locks.
In this scenario, deadlocks are impossible.