We have a very weird problem using EF 6 with MSSQL and MassTransit with Rabbit MQ.
The scenario is as follows:
Client application inserts a row in database (via EF code first - implicit transaction only in DbContext SaveChanges)
Client application publishes Id of the row via MassTransit
Windows Service with consumers processes the message
Row is not found initially, after a few retries, the row appears
I always thought that after commit the row is persisted and becomes accessible for other connections...
We have ALLOW_SNAPSHOT_ISOLATION on in the database.
What is the reason of this and is there any way to be assured that the row is accessible before publishing the Id to MQ?
If you are dependent upon another transaction being completed before your event handler can continue, you need to make you read serializable. Otherwise, transactions are isolated from each other and the results of the write transaction are not yet available. Your write may also need to be serializable, depending upon how the query is structured.
Yes, the consumers run that quickly.
Related
On SQL Server, is it possible to begin a transaction but intentionally orphan it from an open connection yet keep it from rolling back?
The use-case it for a REST service.
I'd like to be able to link a series of HTTP requests to work under a transaction, which can be done if the service is stateful; i.e. there's a single REST API server holding a connection open (map HTTP header value to a named connection), but a flawed idea in a non-sticky farm of web servers.
If the DB supported the notion of something like named/leased transactions, kinda like a named mutex, this could be done.
I appreciate there are other RESTful designs for atomic data mutations.
Thanks.
No. A transaction lives and dies with the session it's created in, and a session lives and dies with its connection. You can keep a transaction open for as long as you like -- but only by also keeping the session, and thereby the connection open. If the session is closed before the transaction commits, it automatically rolls back. Which is a good thing, in general, because transactions tend to use pessimistic locking. You don't want to keep those locks around for longer than necessary.
While there is such a thing as a distributed transaction that you can enlist in even if the current connection did not begin the transaction, this will still not do what you want for the scenario of multiple distributed nodes performing actions in succession to complete a transaction on one database. Specifically, you'd still need to have one "master" node to keep the transaction alive and decide it should finally commit now, and you need a way to make nodes aware of the transaction so they can enlist. I don't recommend you actually go this way, as it's much more complicated than tailoring a solution to your specific scenario (typically, accumulating modifications in their own table and committing them as a batch when they're complete, which can be done in one transaction).
You could use a queue-oriented design, where the application simply adds to the queue, while SQL server agent 'pop's the queue and executes.
I need to sync(upload first to remote DB-download to mobile device next) DB tables with remote DB from mobile device (which may insert/update/delete rows from multiple tables).
The remote DB performs other operation based on uploaded sync data.When sync continues to download data to mobile device the remote DB still performing the previous tasks and leads to sync fail. something like 'critical condition' where both 'sync and DB-operations' want access remote Databse. How to solve this issue? is it possible to do sync DB and operate on same DB at a time?
Am using Sql server 2008 DB and mobilink sync.
Edit:
Operations i do in sequence:
1.A iPhone loaded with application which uses mobilink for SYNC data.
2.SYNC means UPLOAD(from device to Remote DB)followed by DOWNLOAD(from Remote DB to device).
3.Remote DB means Consolidated DB ; device Db is Ultralite DB.
4.Remote DB has some triggers to fire when certain tables are updated.
5.An UPLOAD from device to Remote will fire triggers when sync upload finished.
6.Very next moment the UPLOAD finished DOWNLOAD to device starts.
7.Exactly same moment those DB triggers will fire.
8.Now a deadlock between DB SYNC(-DOWNLOAD) and trigger(Update queries included within) operations occur.
9.Sync fails with error saying cannot access some tables.
I did a lots of work around and Google! Came out with a simple(?!) solution for the problem.
(though the exact problem cannot be solved at this point ..i tried my best).
Keep track of all clients who does a sync(kind of user details).
Create a sql job scheduler which contains all the operations to be performed when user syncs.
Announce a "maintenance period" everyday to execute the tasks of sql job with respect to saved user/client sync details.
Here keeping track of client details every time is costlier but much needed!
Remote consolidated DB "completely-updated" only after maintenance period.
Any approaches better than this would be appreciated! all Suggestions are welcome!
My understanding of your system is following:
Mobile application sends UPDATE statement to SQL Server DB.
There is ON UPDATE trigger, that updates around 30 tables (= at least 30 UPDATE statements in the trigger + 1 main update statement)
UPDATEis executed in single transaction. This transaction ends when Trigger completes all updates.
Mobile application does not wait for UPDATE to finish and sends multiple SELECT statements to get data from database.
These SELECTstatements query same tables as the Trigger above is updating.
Blocking and deadlocks occur at some query for some user as Trigger is not completing updates before selects and keeps lock on tables.
When optimizing we are trying make it our processes less easy for computer, achieve same result in less iterations and use less resources or those resources that are more available/less overloaded.
My suggestions for your design:
Use parametrized SPs. Every time SQL Server receives any statement it creates Execution plan. For 1 UPDATE statement with a trigger DB needs at least 31 execution plan. It happens on busy Production environment for every connection every time app updates DB. It is a big waste.
How SPs would help reduce blocking?
Now you have 1 transaction for 31 queries, where locks are issued against all tables involved and held until transaction commits. With SP you'll have 31 small transaction and only 1-2 tables will be locked at a time.
Another question I would like to address: how to do asynchronous updates to your database?
There is a feature in SQL Server called Service Broker. It allows to process message queue (rows from the queue table) automatically: it monitors queue, takes messages from it and does processing you specify and deletes processes messages from the queue.
For example, you save parameters for your SPs - messages - and Service Broker executes SP with parameters.
I have an application that uses hibernate to inert data into a table.
Database is SQL server. The application itself is deployed in Tomcat 6.
To insert data into DB table - I am using BasicDataSource with minimum configurations for tomcat connection pool (like MaxActive=150, maxIdle =10....)
The problem now is that - I want to add concurrency to the application. In the process - I am making concurrent calls to the business layer method that calls the dao level methods that perform DB inserts. This is resulting in the below error:
Exception occurred java.util.concurrent.ExecutionException: org.hibernate.HibernateException:
Illegal attempt to associate a collection with two open sessions
When I monitor the database, I see that multiple threads are being created but are not being closed.
I am not sure how to proceed further to debug/fix this. Any pointers would be helpful.
If Hibernate is telling you:
Illegal attempt to associate a collection with two open sessions
Basically you are opening two sessions, and have a transaction each and you are trying to save a session in one transaction into another. Ya concurrency is you major problem here. Well That can be tackled if you design your application so as to handle sessions carefully.
Stack-Trace will give you which functions are causing the exceptions. See how long you unit of work lasts with sessions and try reducing those and make sure your sessions are always closed after use.
An application implemented in hibernate can have various patterns.
You need to have session-per-request pattern. In this model, a
request from the client is sent to the server, where the Hibernate
persistence layer runs. A new Hibernate Session is opened, and all
database operations are executed in this unit of work. On completion
of the work, and once the response for the client has been prepared,
the session is flushed and closed. Use a single database transaction
to serve the clients request, starting and committing it when you open
and close the Session. The relationship between the two is one-to-one
and this model is a perfect fit for many applications
Do not use the anti-patterns session-per-user-session or session-per-application
The Transaction And Concurrency in Hibernate Documentation gives in depth analysis and examples
I have three transaction services that are executed within a transaction boundary (stratTransaction or begin transaction). All three services uses different connection (No_Transaction, Local_Transaction and XA_Transaction) for their processing respectively. Now I want to know, when I start a transaction (using javax.transaction.TransactionManager) and run these three services within the transaction boundary, I can see that the service that used NO and LOCAL transactions are able to insert data into the tables. Now I am inserting data more than the table constraints in a column using the Service XA (and I know it is supposed to fail) and calling the commit (and a rollback procedure if there are any failures). Now I have data in tables of NO and Local connection tables while XA connection table don't have any data. Now:
I want to know that when the transaction has failed at one point it is suppossed to rollback all the data from all the tables or it is just supposed to rollback data of XA Service only?
I also wanted to know: 'Transaction' as I know is a procedure of transferring data atomicly. So why connection creation includes defining the type of transaction that can be performed by connection isn't it a property of transactions?
I also want to know that why we have to define transaction type in connection properties instead we must define the type of transaction when we start a transacion and that transaction manager must perform the given type of transactions.
Thanks in advance.
Let's start with the simplest transaction mode and increase complexity.
No transaction
A 'no transaction' connection is one that does not 'commit' or 'roll back' data such as sending email. Once you have passed the message object to the email server, it is sent to the recipient and no amount of pleading will ever get the message back again. It's almost as if every call is committed by the time the call returns. Examples of this kind of connection include connection to SMTP, SMS gateways, printers and so on.
I believe that you can use a database connection in this manner if you have auto-commit on, but it begs the question on why you have a full ACID database in the first place...
'Normal' transactions
The normal connection, for example to a SQL database, has the ability to store up a series of state change commands in an internal buffer. When everything has been done, and all appears OK, then the whole buffer of changes is written to the data store and other connections can see the changes. If something goes wrong, before or even during the commit, the whole set of changes can be discarded (rolled back).
One critical limitation of this type of connection is the scope of the buffer - the buffer is part of the connection itself. In other words, it is only through the connection that you can write to the buffer.
An important responsibility of the application server is to manage these connections. When you ask the connection pool to give you the connection, you magically get the same connection each time (within a single transaction). This is true even when when one EJB calls another or when an EJB calls into a Resource Adapter (assuming you use the REQUIRES_TRANSACTION semantics. You can override this with REQUIRES_NEW). This behaviour means that one web request can multiple EJB calls, each of which can interact with multiple entity beans, and all the data manipulation occurs on a single connection with a single internal buffer. It will all be committed or rolled back together.
Transactions with multiple connections
This is great when you have a single database - but (by definition) you need separate connections if you talk to separate database instances (eg on different machines). So what happens then? Your EJB transaction ends up associated with multiple connections - each connection to a unique database. This appears to work well, except in one situation:
You have Connection A to Database A and Connection B to Database B
You execute DML statements on A and B
You commit the EJB connection. The Application Server now:
Commits Connection A - success
Commits Connection B - fail (eg constraint fails) and Connection B rolls back
This is a disaster - you have committed the transaction in Database A, and this cannot now be rolled back. However, the transaction (and the whole EJB) is rolled back on Database B.
(Interestingly, your example is almost identical to this - you have data committed to the no transaction and normal transaction, but not in the XA transaction - the last of the three connections)
XA Transactions
This is where XA comes in. It provides logic to co-ordinate transactions being committed against different data sources and simulates a single transaction over multiple data sources. XA commits with a "two-phase commit" managed by a transaction co-ordinator that manages a number of XA-connections co-opted into the XA Transaction. The co-ordinator
Sends a message to each data source through the XA Connection to see if the transaction can be committed: All constraint and database logic is executed up to the point just before a final commit. If any database reports a failure, the XA co-ordinator rolls back the whole transaction. Phase 1 is where almost all the transaction work is carried out and so takes comparatively long
When every database has reported that the transaction can be committed, the co-ordinator sends a message to every database to commit the transaction. This happens very fast.
Note that the two-phase commit can fail if something goes wrong in phase 2 (eg part of the network crashes or one of the databases is powered off between phase 1 and phase 2).
Because an XA connection behaves so differently from a normal connection, it typically needs a different ConnectionFactory object which instantiates different object instances than a non-XA ConnectionFactory. In addition, the XA ConnectionFactory needs configuration parameters for the XA transaction co-ordinator, such as XA transaction timeouts, which are in addition to the ordinary transaction properties.
Another constraint: Only Connections created by an XA ConnectionFactory can join an XA Transaction and the associated two-phase commit. You can have both XA and non-XA connections participating in a single Application Server transaction, but then the entire transaction cannot reliably commit/rollback as a single transaction (as above).
Specific answers
I want to know that when the transaction has failed at one point it is suppossed to rollback all the data from all the tables or it is just supposed to rollback data of XA Service only?
If the transaction fails before the application server attempts a commit (eg your EJB gets a NPE or you deliberately roll back), each connection will receive a rollback, and everything should be just as you expect.
However, if the transaction fails in the commit logic (eg a database constraint), then the transaction manager will attempt to roll everything back; this cannot happen if a non-XA connection has already committed.
I also wanted to know: 'Transaction' as I know is a procedure of transferring data atomicly. So why connection creation includes defining the type of transaction that can be performed by connection isn't it a property of transactions?
The XA connection uses a different library and protocol than the ordinary connection, because the connection itself needs to communicate with the XA Transaction Co-ordinator. Normal connections don't do this.
I also want to know that why we have to define transaction type in connection properties instead we must define the type of transaction when we start a transacion and that transaction manager must perform the given type of transactions.
Because the XA connection uses different code, the connection pool needs to load a different class when compared to the normal connection. This is why the connection pool (not connection) properties are different.
yes if a transaction is failed to write its commit entry in log file then it rollbacks completely(Atomic property of trxn ).
Trxn is an atomic unit of database processing.Whatever opr you perform in database using txn , that action will be atomic.
By default the transaction is of autocommit type. but if you use your own code for stating the start point and commit point of a txn then it is of explicit type.(http://msdn.microsoft.com/en-us/library/ms172353.aspx)
I have two instances of database located in two servers. I want to create an application to insert data into the first one and then update data on the second instance, if one of these process fail then I want to rollback all operations.
The database servers do not enable DTC/MSDTC. I tired to use transaction scope but no luck. Do you guys have any idea how can I do this?
if one of these process fail then I want to rollback all operations
You are describing a distributed transaction. To use distributed transactions, you need a transaction coordinator. You can't have the cake and eat it too.
There are alternatives if you consider asynchronous application of the changes, ie. Replication. This removes the distributed transaction requirement but the changes are applied asynchronously to the second server, after they are committed on the first server.
One option would be to put compensation code into your application. For example, if your application were c# based, you could have a try...catch block. In the catch block, you could add compensation code to "undo" the changes you made to the data on the first server.
The best method however, is of course to make a case to the DBAs to enable DTC