I have created Backup of my in-memory database using the below code:
public static void backUpDatabase(Connection conn)throws SQLException
{
String backupdirectory ="c:/mybackups/"+JCalendar.getToday();
CallableStatement cs = conn.prepareCall("CALL SYSCS_UTIL.SYSCS_BACKUP_DATABASE(?)");
cs.setString(1, backupdirectory);
cs.execute();
cs.close();
System.out.println("backed up database to "+backupdirectory);
}
Now I want to restore the database by creating an In-memory database from the previous backup. How should I do it?
Thanks.
The fastest way I am aware of:
First, use shutdown=true:
boolean gotSQLExc = false;
try {
DriverManager.getConnection("jdbc:derby:memory:testdb;shutdown=true");
} catch (SQLException se) {
if ( se.getSQLState().equals("08006") )
System.out.println("Database shut down normally");
else
System.out.println("Database did not shut down normally")
}
Then restoreFrom=
Connection connection = DriverManager.getConnection("jdbc:derby:memory:testdb;restoreFrom=path/to/backup/file");
connection.close();
A db containing ~300 tables with some sample data restored in ~150ms on my dev machine this way.
This is really useful for junit integration testing where fast fast db restore is needed. Depending on your setup, you may need to perform some additional work in-between tests, for example if using c3p0 connection pool, you need to get all connections from C3P0Registry and call hardReset() after this restore. On the other hand, nothing needs to be done with some other pools.
As far as I know, the normal restore-from-backup mechanisms will work just fine to restore an in-memory database. So use the restoreFrom clause on your connection URL, as described here: http://db.apache.org/derby/docs/10.9/adminguide/tadminhubbkup44.html
Related
Right now, to make sure that I'm writing to a correct and functional database at all times, I'm calling the following createDB function every time when I'm accessing the database for data insertion (which happens on a 15 second interval):
protected void createDB(String dbName){
// clear the singleton if forcing a switch
try{
mdbHelper = new DaoMaster.DevOpenHelper(mcontext, dbName, null);
database = mdbHelper.getWritableDatabase();
mdaoMaster = new DaoMaster(database);
dbSessionInstance = mdaoMaster.newSession();
}catch (Exception e){
System.out.println(TAG + e.getMessage());
}
}
This will attempt to create a database with the name given to dbName if it doesn't already exist; if it exists, then it'll just use it.
There are a few scenarios I would like to consider:
1) When the database app crashes, do I have to do anything to make sure the database doesn't get messed up?
2) Does it do more good or harm if I regularly and often call createDB?
3) Just in case, how do I safely perform secure database closures in the event that accidents happen?
When an app crashes, there's nothing it can do because it is no longer running.
As long as you keep the database open, it still exists.
Trying to (re)create the DB repeatedly while your app is running makes sense only if you suspect that somebody else is deleting the DB.
The boundary for guaranteed durability are not database connection but transactions.
In SQLite, transactions are designed so that the database is in a consistent state even if the app crashes during a transaction (i.e., uncommitted transactions are rolled back when the DB is opened afterwards).
I have no Idea about JTA,to understand the overall scenarios please follow this link How to maintain acid property of 3 sequential transaction of three diffrenet databases ,However based on suggestions from the post,I have to use Distributed transactions. I am using apache-tomcat server.
But As I said i have no idea about JTA, So my problem is that, I have more than 15 database connection, and based on the some condition, their respective database is connected. So I can't create hibernate.cfg.xml and session factories and entities for each databases .
So My question is that, can i use JTA with plain jdbc? ,and if possible then provide me some links or examples.
Yes . You can use JTA with plain JDBC . The general idea is that instead of using JDBC Connection object to declare the transaction boundary , you use the Transaction Manager object which is provided by the JTA implementation to declare the transaction boundary .
For example , in the case of Bitronix Transaction Manager , declaring a transaction boundary across many database Connection can be done by the following codes:
PoolingDataSource derbyDataSource1 = new PoolingDataSource();
derbyDataSource1.setClassName("org.apache.derby.jdbc.EmbeddedXADataSource");
derbyDataSource1.setUniqueName("derby1");
derbyDataSource1.getDriverProperties().setProperty("databaseName", "database1");
derbyDataSource1.init();
PoolingDataSource derbyDataSource2= new PoolingDataSource();
derbyDataSource2.setClassName("org.apache.derby.jdbc.EmbeddedXADataSource");
derbyDataSource2.setUniqueName("derby2");
derbyDataSource2.getDriverProperties().setProperty("databaseName", "database2");
derbyDataSource2.init();
BitronixTransactionManager btm = TransactionManagerServices.getTransactionManager();
btm.begin();
try {
Connection c1= derbyDataSource1.getConnection();
Connection c2= derbyDataSource2.getConnection();
/***Use c1 and c2 to execute statements again their corresponding DBs as usual**/
btm.commit();
} catch (SQLException ex) {
ex.printStackTrace();
btm.rollback();
}
Suppose you set up a TransactionScope object as illustrated per the Microsoft example here. Now suppose that you need to update a lot of database tables, and you want them all in the scope of the TransactionScope object. Continually nesting SqlConnection and SqlCommand objects 10 deep will create a source code mess. If instead you call other functions which create connections (in your data access layer, for example), will they be within scope of the TransactionScope object?
Example:
' Assume variable "x" is a business object declared and populated with data.
Using scope As New TransactionScope()
Dal.Foo.SaveProducts(x.Products)
Dal.Foo.SaveCustomer(x.Customer)
Dal.Foo.SaveDetails(x.Details)
' more DAL calls ...
Dal.Foo.SaveSomethingElse(x.SomethingElse)
scope.Complete()
End Using
Assume that each DAL function contains its own using statements for connections. Example:
Public Shared Sub SaveProducts(x As Object)
Using conn As New SqlConnection("connection string")
Using cmd As New SqlCommand("stored procedure name", conn)
With cmd
' etc.
End With
End Using
End Using
End Sub
Yes, they will be inside the TransactionScope. What the TransactionScope basically does is to create a Transaction object and set Transaction.Current to that.
In other words, this:
Using scope As New TransactionScope()
... blah blah blah ...
End Using
is basically the same as this:
try
{
// Transaction.Current is a thread-static field
Transaction.Current = new CommittableTransaction();
... blah blah blah ...
}
finally
{
Transaction.Current.Commit(); // or Rollback(), depending on whether the scope was completed
Transaction.Current = null;
}
When a SqlConnection is opened, it checks if Transaction.Current (on this thread) is null or not, and if it is not null then it enlists (unless enlist=false in the connection string). So this means that SqlConnection.Open() doesn't know or care if the TransactionScope was opened in this method or a method that called this one.
(Note that if you wanted the SqlConnection in the child methods to NOT be in a transaction, you can make an inner TransactionScope with TransactionScopeOption.Suppress)
When you create a TransactionScope, all connections you open while the TransactionScope exists join the transaction automatically (they're 'auto enlisted'). So you don't need to pass connection strings around.
You may still want to, when SQL Server sees different transactions (even if they are all contained by one DTC transaction), it doesn't share locks between them. If you open too many connections and do a lot of reading and writing, you're headed for a deadlock.
Why not put the active connection in some global place and use it?
Some more info after some research. Read this: TransactionScope automatically escalating to MSDTC on some machines? .
If you're using SQL Server 2008 (and probably 2012, but not any other database), some magic is done behind the scenes, and if you open two SQL Connections one after the other, they are going to be united into a single SQL transaction, and you're not going to have any locking problem.
However, if you're using a different database, or you may open two connections concurrently, you will get a DTC transaction, which means SQL Server will not manage the locks properly, and you may encounter very unpleasant and unexpected deadlocks.
While it's easy to make sure you're only running on SQL Server 2008, making sure you don't open two connections at the same time is a bit harder. It's very easy to forget it and do something like this:
class MyPersistentObject
{
public void Persist()
{
using(SQLConnection conn=...)
{
conn.Open()
WriteOurStuff()
foreach(var child in this.PersistedChildren)
child.Persist()
WriteLogMessage()
}
}
}
If the child's Persist method opens another connection, your transaction is escalated into a DTC transaction and you're facing potential locking issues.
So I still suggest maintaining the connection in one place and using it through your DAL. It doesn't have to be a simple global static variable, you can create a simple ConnectionManager class with a ConnectionManager.Current property which will hold the current connection. Make ConnectionManager.Current as [ThreadStatic] and you solved most of your potential problems. That's exactly how the TransactionScope works behind the scenes.
I would like one of my table to have row level locking. I am currently using JPA (hibernate) with play framework.
Is this something that needs to be specified at the DB level or can it be done progammatically?
As a work around I ended up setting the transaction to read_uncommited with the code below.
try{
Session session = (Session) Report.em().getDelegate();
Connection connection = session.connection();
connection.setTransactionIsolation(Connection.TRANSACTION_READ_UNCOMMITTED);
} catch(SQLException e){
...
}
This seems to suit my needs but is not exactly what I was looking for.
Furthermore the session.connection() is deprecated and this might not be a long term solution.
I've done some searching and also read the FAQ on the SQLite site, no luck finding an answer to my question.
It could very well be that my database approach is flawed, but at the moment, I would like to store my data in multiple SQLite3 databases, so that means separate files. I am very worried about data corruption due to my application possibly crashing, or a power outage in the middle of changing data in my tables.
In order to ensure data integrity, I basically need to do this:
begin transaction
modify table(s) in database #1
modify table(s) in database #2
commit, or rollback if error
Is this supported by SQLite? Also, I am using sqlite.net, specifically the latest which is based on SQLite 3.6.23.1.
UPDATE
One more question -- is this something people would usually add to their unit tests? I always unit test databases, but have never had a case like this. And if so, how would you do it? It's almost like you have to pass another parameter to the method like bool test_transaction, and if it's true, throw an exception between database accesses. Then test after the call to make sure the first set of data didn't make it into the other database. But maybe this is something that's covered by the SQLite tests, and should not appear in my test cases.
Yes transactions works with different sqlite database and even between sqlite and sqlserver. I have tried it couple of times.
Some links and info
From here - Transaction between different data sources.
Since SQLite ADO.NET 2.0 Provider supports transaction enlistement, not only it is possible to perform a transaction spanning several SQLite datasources, but also spanning other database engines such as SQL Server.
Example:
using (DbConnection cn1 = new SQLiteConnection(" ... ") )
using (DbConnection cn2 = new SQLiteConnection(" ... "))
using (DbConnection cn3 = new System.Data.SqlClient.SqlConnection( " ... ") )
using (TransactionScope ts = new TransactionScope() )
{
cn1.Open(); cn2.Open(); cn3.Open();
DoWork1( cn1 );
DoWork2( cn2 );
DoWork3( cn3 );
ts.Complete();
}
How to attach a new database:
SQLiteConnection cnn = new SQLiteConnection("Data Source=C:\\myfirstdatabase.db");
cnn.Open();
using (DbCommand cmd = cnn.CreateCommand())
{
cmd.CommandText = "ATTACH DATABASE 'c:\\myseconddatabase.db' AS [second]";
cmd.ExecuteNonQuery();
cmd.CommandText = "SELECT COUNT(*) FROM main.myfirsttable INNER JOIN second.mysecondtable ON main.myfirsttable.id = second.mysecondtable.myfirstid";
object o = cmd.ExecuteScalar();
}
Yes, SQLite explicitly supports multi-database transactions (see https://www.sqlite.org/atomiccommit.html#_multi_file_commit for technical details). However, there is a fairly large caveat. If the database file is in WAL mode, then:
Transactions that involve changes against multiple ATTACHed databases
are atomic for each individual database, but are not atomic across all
databases as a set.