Which ORM framework can best handle an MVCC database design? - database

Question

When designing a database to use MVCC (Multi-Version Concurrency Control), you create tables with either a boolean field like "IsLatest" or an integer "VersionId", and you never do any updates, you only insert new records when things change.
MVCC gives you automatic auditing for applications that require a detailed history, and it also relieves pressure on the database with regards to update locks. The cons are that it makes your data size much bigger and slows down selects, due to the extra clause necessary to get the latest version. It also makes foreign keys more complicated.
(Note that I'm not talking about the native MVCC support in RDBMSs like SQL Server's snapshot isolation level)
This has been discussed in other posts here on Stack Overflow. [todo - links]
I am wondering, which of the prevalent entity/ORM frameworks (Linq to Sql, ADO.NET EF, Hibernate, etc) can cleanly support this type of design? This is a major change to the typical ActiveRecord design pattern, so I'm not sure if the majority of tools that are out there could help someone who decides to go this route with their data model. I'm particularly interested in how foreign keys would be handled, because I'm not even sure of the best way to data model them to support MVCC.

Answer 1

I might consider implementing the MVCC tier purely in the DB, using stored procs and views to handle my data operations. Then you could present a reasonable API to any ORM that was capable of mapping to and from stored procs, and you could let the DB deal with the data integrity issues (since it's pretty much build for that). If you went this way, you might want to look at a more pure Mapping solution like IBatis or IBatis.net.

Answer 2

I designed a database similarly (only INSERTs — no UPDATEs, no DELETEs).
Almost all of my SELECT queries were against views of only the current rows for each table (highest revision number).
The views looked like this…
SELECT
dbo.tblBook.BookId,
dbo.tblBook.RevisionId,
dbo.tblBook.Title,
dbo.tblBook.AuthorId,
dbo.tblBook.Price,
dbo.tblBook.Deleted
FROM
dbo.tblBook INNER JOIN
(
SELECT
BookId,
MAX(RevisionId) AS RevisionId
FROM
dbo.tblBook
GROUP BY
BookId
) AS CurrentBookRevision ON
dbo.tblBook.BookId = CurrentBookRevision.BookId AND
dbo.tblBook.RevisionId = CurrentBookRevision.RevisionId
WHERE
dbo.tblBook.Deleted = 0
And my inserts (and updates and deletes) were all handled by stored procedures (one per table).
The stored procedures looked like this…
ALTER procedure [dbo].[sp_Book_CreateUpdateDelete]
#BookId uniqueidentifier,
#RevisionId bigint,
#Title varchar(256),
#AuthorId uniqueidentifier,
#Price smallmoney,
#Deleted bit
as
insert into tblBook
(
BookId,
RevisionId,
Title,
AuthorId,
Price,
Deleted
)
values
(
#BookId,
#RevisionId,
#Title,
#AuthorId,
#Price,
#Deleted
)
Revision numbers were handled per-transaction in the Visual Basic code…
Shared Sub Save(ByVal UserId As Guid, ByVal Explanation As String, ByVal Commands As Collections.Generic.Queue(Of SqlCommand))
Dim Connection As SqlConnection = New SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings("Connection").ConnectionString)
Connection.Open()
Dim Transaction As SqlTransaction = Connection.BeginTransaction
Try
Dim RevisionId As Integer = Nothing
Dim RevisionCommand As SqlCommand = New SqlCommand("sp_Revision_Create", Connection)
RevisionCommand.CommandType = CommandType.StoredProcedure
RevisionCommand.Parameters.AddWithValue("#RevisionId", 0)
RevisionCommand.Parameters(0).SqlDbType = SqlDbType.BigInt
RevisionCommand.Parameters(0).Direction = ParameterDirection.Output
RevisionCommand.Parameters.AddWithValue("#UserId", UserId)
RevisionCommand.Parameters.AddWithValue("#Explanation", Explanation)
RevisionCommand.Transaction = Transaction
LogDatabaseActivity(RevisionCommand)
If RevisionCommand.ExecuteNonQuery() = 1 Then 'rows inserted
RevisionId = CInt(RevisionCommand.Parameters(0).Value) 'generated key
Else
Throw New Exception("Zero rows affected.")
End If
For Each Command As SqlCommand In Commands
Command.Connection = Connection
Command.Transaction = Transaction
Command.CommandType = CommandType.StoredProcedure
Command.Parameters.AddWithValue("#RevisionId", RevisionId)
LogDatabaseActivity(Command)
If Command.ExecuteNonQuery() < 1 Then 'rows inserted
Throw New Exception("Zero rows affected.")
End If
Next
Transaction.Commit()
Catch ex As Exception
Transaction.Rollback()
Throw New Exception("Rolled back transaction", ex)
Finally
Connection.Close()
End Try
End Sub
I created an object for each table, each with constructors, instance properties and methods, create-update-delete commands, a bunch of finder functions, and IComparable sorting functions. It was a huge amount of code.
One-to-one DB table to VB object...
Public Class Book
Implements iComparable
#Region " Constructors "
Private _BookId As Guid
Private _RevisionId As Integer
Private _Title As String
Private _AuthorId As Guid
Private _Price As Decimal
Private _Deleted As Boolean
...
Sub New(ByVal BookRow As DataRow)
Try
_BookId = New Guid(BookRow("BookId").ToString)
_RevisionId = CInt(BookRow("RevisionId"))
_Title = CStr(BookRow("Title"))
_AuthorId = New Guid(BookRow("AuthorId").ToString)
_Price = CDec(BookRow("Price"))
Catch ex As Exception
'TO DO: log exception
Throw New Exception("DataRow does not contain valid Book data.", ex)
End Try
End Sub
#End Region
...
#Region " Create, Update & Delete "
Function Save() As SqlCommand
If _BookId = Guid.Empty Then
_BookId = Guid.NewGuid()
End If
Dim Command As SqlCommand = New SqlCommand("sp_Book_CreateUpdateDelete")
Command.Parameters.AddWithValue("#BookId", _BookId)
Command.Parameters.AddWithValue("#Title", _Title)
Command.Parameters.AddWithValue("#AuthorId", _AuthorId)
Command.Parameters.AddWithValue("#Price", _Price)
Command.Parameters.AddWithValue("#Deleted", _Deleted)
Return Command
End Function
Shared Function Delete(ByVal BookId As Guid) As SqlCommand
Dim Doomed As Book = FindByBookId(BookId)
Doomed.Deleted = True
Return Doomed.Save()
End Function
...
#End Region
...
#Region " Finders "
Shared Function FindByBookId(ByVal BookId As Guid, Optional ByVal TryDeleted As Boolean = False) As Book
Dim Command As SqlCommand
If TryDeleted Then
Command = New SqlCommand("sp_Book_FindByBookIdTryDeleted")
Else
Command = New SqlCommand("sp_Book_FindByBookId")
End If
Command.Parameters.AddWithValue("#BookId", BookId)
If Database.Find(Command).Rows.Count > 0 Then
Return New Book(Database.Find(Command).Rows(0))
Else
Return Nothing
End If
End Function
Such a system preserves all past versions of each row, but can be a real pain to manage.
PROS:
Total history preserved
Fewer stored procedures
CONS:
relies on non-database application for data integrity
huge amount of code to be written
No foreign keys managed within database (goodbye automatic Linq-to-SQL-style object generation)
I still haven't come up with a good user interface to retrieve all that preserved past versioning.
CONCLUSION:
I wouldn't go to such trouble on a new project without some easy-to-use out-of-the-box ORM solution.
I'm curious if the Microsoft Entity Framework can handle such database designs well.
Jeff and the rest of that Stack Overflow team must have had to deal with similar issues while developing Stack Overflow: Past revisions of edited questions and answers are saved and retrievable.
I believe Jeff has stated that his team used Linq to SQL and MS SQL Server.
I wonder how they handled these issues.

Answer 3

To the best of my knowledge, ORM frameworks are going to want to generate the CRUD code for you, so they would have to be explicitly designed to implement a MVCC option; I don't know of any that do so out of the box.
From an Entity framework standpoint, CSLA doesn't implement persistence for you at all -- it just defines a "Data Adapter" interface that you use to implement whatever persistence you need. So you could set up code generation (CodeSmith, etc.) templates to auto-generate CRUD logic for your CSLA entities that go along with a MVCC database architecture.
This approach would work with any entity framework, most likely, not just CSLA, but it would be a very "clean" implementation in CSLA.

Answer 4

Check out the Envers project - works nice with JPA/Hibernate applications and basically does that for you - keeps track of different versions of each Entity in another table and gives you SVN-like possibilities ("Gimme the version of Person being used 2008-11-05...")
http://www.jboss.org/envers/
/Jens

Answer 5

I always figured you'd use a db trigger on update and delete to push those rows out into a TableName_Audit table.
That'd work with ORMs, give you your history and wouldn't decimate select performance on that table. Is that a good idea or am I missing something?

Answer 6

What we do, is just use a normal ORM ( hibernate ) and handle the MVCC with views + instead of triggers.
So, there is a v_emp view, which just looks like a normal table, you can insert and update into it fine, when you do this though, the triggers handle actually inserting the correct data into the base table.
Not.. I hate this method :) I'd go with a stored procedure API as suggested by Tim.