Restoring differential backups from SQL Server - sql-server

I have a strange problem. I have a database, called restorelogs.
An autobackup script do a full backup, and after that, in every 3 hours a differential backup.
Next day do a full again, and then the diffs again, and so on.
No i want to write a script, what is restore this files int the restorelogs database.
From php I am using these sql queries through sqlsrv:
USE master;
ALTER DATABASE restorelogs SET MULTI_USER WITH ROLLBACK IMMEDIATE;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307151435.bak' WITH REPLACE, NORECOVERY;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307151745diff.bak' WITH FILE= 1, REPLACE, NORECOVERY;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307152045diff.bak' WITH FILE= 1, REPLACE, NORECOVERY;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307152345diff.bak' WITH FILE= 1, REPLACE, NORECOVERY;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307160245diff.bak' WITH FILE= 1, REPLACE, NORECOVERY;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307160545diff.bak' WITH FILE= 1, REPLACE, NORECOVERY;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307160845diff.bak' WITH FILE= 1, REPLACE, NORECOVERY;
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307161145diff.bak' WITH FILE= 1, REPLACE, RECOVERY;
And now will comes the interesting part of the game.
If I copy these queries into SQL Server Management Studio, and run it, it works fine. There are the tables, no error messages, everything is all right.
BUT. When I try to do it from php, the database leave in restoring mode.
As I read through a lot of pages, I understood, the last query need to include the RECOVERY option to recover the database (I also try to not use the REPLACE, just at the first query).
How can it happens?
Please help me, what do i wrong.
UPDATE: Based on Adam comment: i exactly try to do that. I collect the backup files, and use the first (full), with NORECOVERY, REPLACE, and the last WITH RECOVERY.
This is what i try:
function pushToDatabase($files) {
$dbh = new Database();
$dbh->query("USE master");
pre($dbh->errorInfo(), 0);
$cnt = count($files);
$i = 0;
foreach ($files as $file) {
$withFile = '';
$option = '';
$moves = '';
if ($i == 0) {
//restoreFromDisk($file);
//truncateDatabase();
$option = " NORECOVERY, REPLACE";
} else {
$option = " RECOVERY";
}
if ($i > 0) {
$withFile = " FILE=1, ";
}
if ($i == 0 || $i == count($files) - 1) {
$sql = "RESTORE DATABASE " . DB_NAME . " FROM DISK = '" . $file . "' WITH " . $withFile . $option . ";";
echo $sql . "<br />";
$dbh->exec($sql);
pre($dbh->errorInfo(), 0);
}
$i++;
}
$sql = "RESTORE DATABASE " . DB_NAME . " WITH RECOVERY;";
$dbh->exec($sql);
pre($dbh->errorInfo(), 0);
echo $sql . "<br />";
die("NOW WE ARE DONE!");
}
The first statement is running without any problems:
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307151435.bak' WITH NORECOVERY, REPLACE;
But after this, when i want to restore the last diff file:
RESTORE DATABASE restorelogs FROM DISK = 'D:/Apache/htdocs/logreader/extracts/SRO_VT_LOG201307161145diff.bak' WITH FILE=1, RECOVERY;
i've get back an error message:
[Microsoft][SQL Server Native Client 10.0][SQL Server]A previous restore operation was interrupted and did not complete processing on file 'Log_DB_Data'. Either restore the backup set that was interrupted or restart the restore sequence.'
And the database state leaves in restoring mode.
I've also tried to play with the RESTORE FILELISTONLY, and moving the files into the right place. I could do that, but after that i can not recover the differential backup.
function restoreFromDisk($file) {
$sql = "RESTORE FILELISTONLY FROM DISK = '" . $file . "'";
$res = dbQuery($sql);
$option = ' ,REPLACE, RECOVERY';
while ($row = dbFetchAssoc($res)) {
if ($row["LogicalName"] == "Log_DB_Data") {
$move[] = " MOVE '" . $row["LogicalName"] . "' TO '" . MSSQL_DATA_PATH . "resetlogs.mdf'";
} else {
$move[] = " MOVE '" . $row["LogicalName"] . "' TO '" . MSSQL_DATA_PATH . "resetlogs.ldf'";
}
}
$moves = join(",", $move);
$sql = "RESTORE DATABASE " . DB_NAME . " FROM DISK = '" . $file . "' WITH " . $moves . $option . ";";
dbQuery($sql);
echo $sql . "<br />";
$sql = "RESTORE DATABASE " . DB_NAME . " WITH RECOVERY;";
dbQuery($sql);
echo $sql . "<br />";
}

You should be setting the database to single user, if there is the potential for users to be connected (and if it is okay to kill their sessions). Since you are restoring the database, I would think that it is okay. You will need to specify the replace option on the full backup restoration. Also you only need to restore the latest differential backup (using with recovery), as it will contain all the changes since the last full backup. It should also be noted that this restore will not include transactions that have occurred since the last differential backup.
I would think about using log backups every 3 hours. This will have less impact on your system and will allow the log to truncate (if you are in full recovery). In this restore scenario, you will need the full backup and all the log backups, with the last option specifying with recovery.

A different approach would be to write your SQL Script to a temp file, then use osql to execute it as a script. Of course this assumes that you're running on a machine that osql installed.
osql -E -S <server> -d master -i <sql file>

Related

Pyodbc - SQL Server database restore incomplete

Am trying to restore the database from python 3.7 in Windows using below script.
Drop database functions correctly as expected.
The restore database doesn't work as expected, database always shows "Restoring...." and never completes.
Database files are there in the specified path, but database is not usable.
How to fix this?
import pyodbc
try:
pyconn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=MY-LAPTOP\\SQLEXPRESS;DATABASE=master;UID=sa;PWD=sa123')
cursor = pyconn.cursor()
pyconn.autocommit = True
sql = "IF EXISTS (SELECT 0 FROM sys.databases WHERE name = 'data_test') BEGIN DROP DATABASE data_test END"
pyconn.cursor().execute(sql)
sql = """RESTORE DATABASE data_test FROM DISK='G:\\dbbak\\feb-20-2020\\data_test_backup_2020_02_20_210010_3644975.bak' WITH RECOVERY,
MOVE N'Omnibus_Data' TO N'd:\\db\\data_test.mdf',
MOVE N'Omnibus_Log' TO N'd:\\db\\data_test_1.ldf';"""
print(sql)
pyconn.cursor().execute(sql)
while pyconn.cursor().nextset():
pass
pyconn.cursor().close()
except Exception as e:
print(str(e))
You're not using a single cursor, and so your program is exiting before the restore is complete, aborting it in the middle.
Should be something like:
conn = pyodbc.connect(' . . .')
conn.autocommit = True
cursor = conn.cursor()
cursor.execute(sql)
while cursor.nextset():
pass
cursor.close()
0
After hours I found solution. It must be performed no MASTER, other sessions must be terminated, DB must be set to OFFLINE, then RESTORE and then set to ONLINE again.
def backup_and_restore():
server = 'localhost,1433'
database = 'myDB'
username = 'SA'
password = 'password'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE=MASTER;UID='+username+';PWD='+ password)
cnxn.autocommit = True
def execute(cmd):
cursor = cnxn.cursor()
cursor.execute(cmd)
while cursor.nextset():
pass
cursor.close()
execute("BACKUP DATABASE [myDB] TO DISK = N'/usr/src/app/myDB.bak'")
execute("ALTER DATABASE [myDB] SET SINGLE_USER WITH ROLLBACK IMMEDIATE;")
execute("ALTER DATABASE [myDB] SET OFFLINE;")
execute("RESTORE DATABASE [myDB] FROM DISK = N'/usr/src/app/myDB.bak' WITH REPLACE")
execute("ALTER DATABASE [myDB] SET ONLINE;")
execute("ALTER DATABASE [myDB] SET MULTI_USER;")

How to bring SQL Server database out of "restoring" mode using SMO

I'm bringing databases out of log shipping, trying to use SMO to do it. I'm attempting to mimic the following T-SQL using SMO:
restore database <database name> with recovery
Here's my code:
# select secondary_database from msdb.dbo.log_shipping_secondary_databases
$dsSecLSDB = $secInst.Databases["MSDB"].ExecuteWithResults("select secondary_database from log_shipping_secondary_databases")
$secLSDB = $dsSecLSDB.Tables.Rows
foreach($db in $secLSDB.secondary_database) {
write-host "Restoring database (bringing online)..."
$secRestrObj = New-Object -TypeName Microsoft.SqlServer.Management.Smo.Restore -Property #{
Action = 'Database';
Database = $db;
NoRecovery = $FALSE;
}
$secRestrObj.SqlRestore($secInst);
write-host "Done with restore."
}
The error:
Microsoft.SqlServer.Management.Smo.PropertyNotSetException: To accomplish this action, set property Devices
The available options for DeviceType (from https://msdn.microsoft.com/en-us/library/microsoft.sqlserver.management.smo.devicetype(v=sql.105).aspx) are:
LogicalDevice
Tape
File
Pipe
VirtualDevice
The problem is, I don't know which DeviceType to create. My guess is LogicalDevice but I don't know the value of it. Has anyone done this before?
I'm facing the same trouble.
I guess that the method sqlRestore is limited to a strong database restore, from a backup file/device.
Just bringing "online" a database with the status "restoring" (ie stand by db from log shipping or from a broken AG) seems to be possible only executing the "restore database with recovery" with the invoke-sqlcmd applet.
Or if anybody has an other solution ...

Using SMO to restore a database from a .bak file

I tried a few solutions and below is the most straight forward one but I get the error below
Logical file 'RestoredProcessMananger' is not part of database 'RestoredProcessMananger'. Use RESTORE FILELISTONLY to list the logical file names.
RESTORE DATABASE is terminating abnormally.
What am I doing wrong ? The idea is to create DLL to be used by other programs the allow a baseline database to be reloaded on a server overwriting whatever is there..
ServerConnection connection = new ServerConnection("xxx", "sa", "srv$xxx");
Server svr =new Server(connection);
Restore res = new Restore();
res.Database = "RestoredProcessMananger";
res.Action = RestoreActionType.Database;
res.Devices.AddDevice(#"C:\temp\ProcessManager.bak", DeviceType.File);
res.ReplaceDatabase = true;
res.RelocateFiles.Add(new RelocateFile("RestoredProcessMananger", _
#"c:\ProcessManager2.mdf"));
res.RelocateFiles.Add(new RelocateFile("RestoredProcessMananger_Log", _
#"c:\ProcessManager2_log.ldf"));
res.SqlRestore(svr);
svr.Refresh();
EDIT 1: fixed
public static void RestoreDatabase(string Server, //sqlserver //from CONFIG
string BackupFilePath, //where the bak file I want to restore //from CONFIG
string destinationDatabaseName, //what the restored database will be called //from CONFIG
string DatabaseFolder, //where the data/log files for the destination (break into 2 variables (2 different locations)) (get from GetDatabaseMDFFilePathName)
string DatabaseFileName, //the destination MDF file name (get from GetDatabaseMDFFilePathName)
string DatabaseLogFileName) //the destination LDF file name (get from GetDatabaseMDFFilePathName)
{
Server myServer = GetDatabases("xxx");
Restore myRestore = new Restore();
myRestore.Database = destinationDatabaseName;
Database currentDb = myServer.Databases[destinationDatabaseName];
if (currentDb != null)
myServer.KillAllProcesses(destinationDatabaseName);
myRestore.Devices.AddDevice(BackupFilePath, DeviceType.File);
string DataFileLocation = DatabaseFolder + "\\" + destinationDatabaseName + ".mdf";
string LogFileLocation = DatabaseFolder + "\\" + destinationDatabaseName + "_log.ldf";
myRestore.RelocateFiles.Add(new RelocateFile(DatabaseFileName, DataFileLocation));
myRestore.RelocateFiles.Add(new RelocateFile(DatabaseLogFileName, LogFileLocation));
myRestore.ReplaceDatabase = true;
myRestore.PercentCompleteNotification = 10;
myRestore.SqlRestore(myServer);
currentDb = myServer.Databases[destinationDatabaseName];
currentDb.SetOnline();
}
The error indicates that RestoredProcessMananger is not the name of a logical file contained in your backup file.
Have you used RESTORE FILELISTONLY ... to list the files contained in the .BAK file?
For instance, if your backup file is named C:\MyBackupFile.bak you would need to run the following query from SQL Server Management Studio (or SQLCMD, etc):
RESTORE FILELISTONLY FROM DISK='C:\MyBackupFile.bak';
This will provide a list of the logical files contained in the backup file. You will then need to pass the name of one of these files into your RelocateFile code.

Batch file SQLCMD Restore Loop - Files Disappear

So I've searched and found a lot of info in similar areas, but nothing that quite hits the issue I'm having.
I'm working in a test environment and need to restore from the same SQL database backup (.bak) to many named instances of SQL server. All the sql instances are pre-installed and running.
I played around with different ideas, but a batch file using sqlcmd seems to be the best suited for the job.
So I created a batch file (.bat) that asks for the starting and stopping instance numbers and then should restore from the backup to each SQL named instance incrementing the instance number along the way.
When it runs the sqlcmd appears to work fine. At the end it prints out
RESTORE DATABASE successfully processed X pages in Y seconds
Also the files (.mdf, .ndf, .ldf) are in the directory as expected and then it moves on to the next.
The problem is that when it moves on to the next, the files that were just restored disappear from the directory.
If anyone has any ideas it would certainly be appreciated.
Here's the batch...
ECHO OFF
ECHO Enter starting instance number for restore db
SET /P _IntStart=
ECHO Enter number of last instance for restore db
SET /P _IntStop=
SET /a _IntStop=_IntStop+1
:RestoreDb
If %_IntStart% GEQ %_IntStop% goto:EOF
ECHO Display Instance Number... IntStart = %_IntStart%
sqlcmd -e -s localhost\instance%_IntStart% -d master -U user -P password -Q "Use [master]; RESTORE DATABASE DBName1 FROM DISK = 'C:\DBName1.bak'WITH REPLACE, MOVE 'DBName1' TO 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE%_IntStart%\MSSQL\DATA\DBName1.mdf', MOVE 'DBName1_log' TO 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE%_IntStart%\MSSQL\DATA\DBName1_log.LDF', MOVE 'ftrow_DBName1Catalog' TO 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE%_IntStart%\MSSQL\DATA\DBName1_1.ndf';"
SET /a _IntStart=_IntStart+1
GOTO:RestoreDb
PAUSE
EXIT
=========================================
From SQL Mgmt. Studio I've also tried the below. It works if I comment out the loop and run it each time manually bumping the instance number. It will create separate copies of the db and files. The problem here is SQLCMD doesn't appear to like concatenation in Mgmt. Studio so I can't increment the instance number in the :CONNECT. It ends up trying to connect to localhost\instance$(SCintnum).
Declare #intnum int
Set #intnum = 1
Declare #intstr NVARCHAR(255)
Set #intstr = #intnum
Declare #PathName1 NVARCHAR(255)
Declare #PathName2 NVARCHAR(255)
Declare #PathName3 NVARCHAR(255)
Set #PathName1 = 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE' + #intstr + '\MSSQL\DATA\DBName1.mdf'
Set #PathName2 = 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE' + #intstr + '\MSSQL\DATA\DBName1_log.LDF'
Set #PathName3 = 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE' + #intstr + '\MSSQL\DATA\DBName1_1.ndf'
While #intnum < 51
:SETVAR SCintnum 1
:CONNECT localhost\instance$(SCintnum) -U user -P password
Use [master];
RESTORE DATABASE DBName1 FROM DISK = 'C:\DBName1.bak'
WITH REPLACE,
MOVE 'DBName1' TO #PathName1,
MOVE 'DBName1_log' TO #PathName2,
MOVE 'ftrow_DBName1Catalog' TO #PathName3;
:SETVAR SCintnum $(SCintum)+1
Set #intnum = #intnum+1
Set #intstr = #intnum
Set #PathName1 = 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE' + #intstr + '\MSSQL\DATA\DBName1.mdf'
Set #PathName2 = 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE' + #intstr + '\MSSQL\DATA\DBName1_log.LDF'
Set #PathName3 = 'E:\Microsoft SQL Server\MSSQL10_50.INSTANCE' + #intstr + '\MSSQL\DATA\DBName1_1.ndf'
===================================================================
This is an example of what I ended up using in c#.....
///Set SQL Connection
SqlConnection myConnection = new SqlConnection("user id=sa;" +
"password="+ sapassword+";server="+servername+"\\instance"+currentinstancenum+";" +
"Trusted_Connection=yes;" +
"database=master; " +
"connection LifeTime=0; connection Timeout=30");
///Set SQL Command
string thesqlcommand = "USE [master]; RESTORE DATABASE " + dbname + " FROM DISK = '" + backuplocation + "' WITH REPLACE, MOVE '" + dbname + "' TO " + #PathName1 + ", MOVE '" + dbname + "_log' TO " + #PathName2 + ", MOVE 'ftrow_" + dbname + "Catalog' TO " + #PathName3 + ";";
SqlCommand myCommand = new SqlCommand(thesqlcommand, myConnection);
///Set SQL Command TimeOut, open connection, execute command, close command
myCommand.CommandTimeout = 180;
myCommand.Connection.Open();
myCommand.ExecuteNonQuery();
myConnection.Close();
It is doing what you asked...
MOVE 'DBName1'
and
MOVE 'DBName1_log'
Ended up creating a little utility in C# to do this. Wish I had started there as it was far simpler. I added an example to the bottom of the original post.

How to copy SQL Server database from Access

We have an Access UI to Sql Server database. The user connects to many databases (containing the same tables with different data), can choose between them. We use this for versioning. We want to make him able to copy and delete databases right from Access UI. He should be able to copy at least to the same server, and ideally also to other server.
A Backup and restore is probably going to be your best bet. There is another way as well. But there will be some restrictions.
You can detach the DB you want to copy, make a copy of the files attach the old one, and attach the new one as a new DB. Your problem will be because you are using Access to connect to the DB, you will not be able to detach it because there is a connection to it, and all connections must be dropped before you can detach it.
Dropping the DB (delete it) will have the same problem. It won't drop unless you there is no connections on the DB.
This is my final solution:
Dim conn As New ADODB.Connection
conn.ConnectionString = "Provider=SQLOLEDB;Data Source=" dbServer & ";" _
& "User ID=" & user & ";Password=" & password
conn.Open
' backup
conn.Execute "BACKUP DATABASE [" & sourceDb & "] TO [backup device] WITH NOFORMAT, NOINIT, NAME = N'backup', SKIP, NOREWIND, NOUNLOAD, STATS = 10"
' restore
conn.Execute "RESTORE DATABASE [" & targetDb & "] FROM [abcosting temporary backup] WITH FILE = 1, NOUNLOAD, REPLACE, STATS = 10, " & mdf_move & ", " & ldf_move

Resources