Given the following (hypothetical) scenario, how would one best backup/restore the database.
Daily doing full backups # 12 am.
Hourly doing differentials 1 am, 2am etc
Transaction log backups on the half hours, 130am, 230am etc
I am also storing the active .ldf file on drive X and the .mdf on drive Y.
Also important the master db is on Y.
Lets say hypothetically the Y drive fails at 245am.
I have the full, diffs and transaction logs up until 230am. BUT I also have the .ldf.
In theory I would have to probably reinstall SQL Server. Then I would want to recover that database up until 245am.
I have heard of doing a tail-log backup on a restore operation BUT I don't have the .mdf anymore. So, I would need to create a new database from my full/diff/log backups. After that I'm not sure how to proceed to get that last 15 minutes of transactions.
I hope this is making sense...
Thanks!
Steve.
You are asking,how to take TailLog Backup when you don't have access to MDF files..
This works only if your database is not in BulkLoggedRecovery model or your log doesn't have Bulk logged transactions..This has been covered in depth here: Disaster recovery 101: backing up the tail of the log
Here are the steps in order
Create a dummy database with same names
Delete all files of this dummy database,by bringing it offline
Copy the original database LDF
Bring this database online which will fail..
Now you can take TailLog Backup using below command..
BACKUP LOG dummydb
TO DISK = N'D:\SQLskills\DemoBackups\DBMaint_Log_Tail.bck' WITH INIT, NO_TRUNCATE;
GO
Now since you have all the backups,you can restore to point in time of Failure
Related
I'm having an issue that only happens sometimes where my SQL Server differential backup job fails with an error message similar to
Msg 3035, Sev 16, State 1, Line 1 : Cannot perform a differential backup for database "MyDatabaseName", because a current database backup does not exist. Perform a full database backup by reissuing BACKUP DATABASE, omitting the WITH DIFFERENTIAL option. [SQLSTATE 42000]
Msg 3013, Sev 16, State 1, Line 1 : BACKUP DATABASE is terminating abnormally. [SQLSTATE 42000]
I'm currently using Ola Hallengren's SQL Server Mantenance Solution script for backup's, integrity check's and index maintenance. I've scheduled the backup job's as such:
Full Backup of system databases every day # 1:30 AM
Full Backup of all user databases every week on Monday, Wednesday & Friday # 2:30 AM
Differential Backup of all user databases every week on Sunday, Tuesday, Thursday & Saturday # 2:30 AM
Transaction Log backup or all user databases every 30 minutes
I've also set the cleanup time to 168 hours...which is 7 days.
I know that usually when this error message comes up, it is due to a full backup not being present, or possible the recovery mode on a database being changed. I have checked both of these and neither seems to be the case. I can confirm that my Friday full backup was successful, however my Saturday and Sunday Differentials failed. There have also been no changes to the recovery model, and no manual backup taken through SQL Server.
It's worth noting that this only happens sometimes. Sometimes the differential backups work without a problem and other times they fail.
This server is a VM, and we are using VMWare vSphere/vCenter 6.5. I've talked to my server admin and asked how his backup is running an he told me that we are using Quest AppAssure which leverage VMWare snapshot technology, and that he is backuping the drives every x number of minutes, so it is possible that the time of his backup changes and eventually overlaps with mine.
We looked back at the time that his backups ran on the weekend and they happened within a few minutes before mine started. I'm wondering if this is causing my backup issues? if so is there a way to prevent this, or do we simply need to plan the backups at different times that don't overlap?
Thanks
We had another call with Quest today and figured out a solution to the problem.
It would seem that when configuring the backup through Quest Rapid Recovery, you have the option to do Machine level backups or Volume level backup. When it's configured to do volume level backups you can choose for it to either do a block level backup which isn't SQL Server aware, or for it to do a backup that is SQL Server aware, which ends up using the volume shadow copy service and these backups show up as Full (Copy Only) backups in the SQL backup history...even though you can't restore them from SQL Server.
Rapid Recovery can only do backups on a schedule, plus has the option to Truncate Logs after the backup complete to avoid filling the log files, but can't do transaction log backups, so you loose the ability to do a more granular recovery down to the second like with the native SQL Timeline Recovery...Which is why we chose to use native SQL Server Backups.
So to fix the issue, you need to either do a machine level backup which has not SQL Server Awareness/integration. Or you can do a volume level backup, but disable the SQL Server Writer extension, and the truncate logs options to remove the integration.
We've run a bunch of tests, and from the point/time that this change was done, we only see SQL Server backups and no longer the Rapid Recovery backups in the SQL Server Backup history.
So now i am in talks with my server/backup admin see if we can do a machine level backup once a day, so that we can do a machine level restore in case of disaster, and also add a volume level backup of my Backup drive, so that he captures my backups more frequently during the day. I think once this is complete, we'll have the best of both backup solutions.
Ability to do machine level restores (Rapid Recovery)
Retention of SQL Server Backups (Rapid Recovery)
Flexible point in time recovery (SQL Server)
Just wanted to post a update to this issue
We scheduled a call with Quest yesterday, and they assured me that their backup is only taking volume snapshots and will have no impact on my SQL backups. They said that the reason that i was seeing these errors is likely do to Rapid Recovery(I guess AppAssure was renamed to Rapid Recovery) and my SQL backup jobs both trying to use the volume shadow copy service at the same time and so we simply needed to stagger the backup jobs. I end up finding out that this wasn't completely true because the Rapid Recovery backups were configured to truncate my SQL Logs. I also told the guy from rapid recovery that when i queried the msdb backupset table that i was seeing backup jobs listed that lined up with the times of the rapid recovery backup. Still he assured me that it would have zero impact on my backups.
I was still concerned that the Rapid Recovery backups might be impacting the chain of my backup files, so in our test environment i right clicked on one of our databases and clicked tasks > restore >database just to see the recovery history. I see a database backup listed as type Full (Copy Only) which coincides with the rapid recovery backup, then a number of my transaction log backups.
It would seem to me that Rapid Recovery is definitely impacting my SQL Backups.
One other thing to note that i just tried in out test environment. I did a few tests backups using Full's , Transaction Log, Differentials and Full (Copy Only) just to see how things showed up in the restore window in SQL Server Management Studio.
So i realized that in the default restore screen, it tries to recover to the nearest point in time using the least combination of various backup files. To go past the last full backup, I have to use the Timeline option.
I could see the Full backup as the first item, and then the transaction logs backups.
Once i do the Differential backup, i see the Full plus the differential, but no more transaction log backups.... which makes sense since it's trying to get the the closest time possible for the recovery.
Next if i do another Transaction Log backup, I see the Full, Differential and the Transaction Log back
One thing that surprised me however is if i do a Full (Copy Only) followed by a transaction log backup, that i see these two item in the list of files for the recovery, yet if i do a differential after a Full (Copy Only) that it show me the last Full (non copy only), plus the differential. I expected that the backups would always be based off of the last full, both for transaction log and differential backups. I thought that the Copy Only backups would be ignored in the backup chain.
Next I decided to use the timeline restore feature and select a point in time during my tests where the Rapid Recovery backup wasn't part of the listed backups, and do a verify backup. As expected it was successful. After this i tried a restore to another point in time where the Rapid Recovery Full (Copy Only) backup was listed and the verify failed on the Full (Copy Only) backup file from rapid recovery, since it doesn't exist on the sql server.
Any recommendations on how to fix this? The point of the Rapid Recovery backup is supposed to be to backup the machine, in case we ever loose the server and have to restore the entire server, plus for it to pick up my sql server backups to keep for retention, since i only keep 7 days worth of backup on the server itself.
You can check the error log file.
Your job may be the victim of deadlock transactions.
Backup file location may not have enough permission.
So, you should check all the possible error log files.
I am attempting to do a poor mans log shipping, manually. I need to move a rather large database from one host, to a VM in Azure. The database is currently 16Gb, and we need to do the switch over within an hour.
Note that Replication, Mirroring and Log Shipping are not options due to issues we have run into, and this manual process is what I need to try and achieve.
What I am attempting is to do a full backup the database a few weeks before, using striping of the bak files (So that if a copy fail happens, it's not the full 16Gb we need to restart the copy of), and copy those files of the full backup over a period of a few days.
For my full backup, I am trying to span the backup file over 15 files:
BACKUP DATABASE MyProductionDatabase
TO
DISK='E:\BackupTrial\V5_DEV_FULL_01.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_02.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_03.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_04.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_05.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_06.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_07.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_08.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_09.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_10.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_11.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_12.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_13.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_14.bak',
DISK='E:\BackupTrial\V5_DEV_FULL_15.bak'
WITH FORMAT,
MEDIANAME = 'V5_DEV_FullBackup',
MEDIADESCRIPTION = 'Striped media set for V5_DEV database';
GO
Then restore that database on the new VM in Azure (Virtual Machine, with SQL Server on it, as I am not sure I can go direct BAK file to AzureSQL).
And then daily or weekly (The database has low usage), do log file backups and copy those backups to the new host and restore. So basically, manually trying to keep the databases in sync.
On go-live day, do a final log file backup on the current host, copy that final bak file to the new host, and then both DBs should be in sync, with only small daily copies.
Then switch the site to use the new DB.
But I am stuck:
The MS example is:
BACKUP LOG AdventureWorks2012
TO MyAdvWorks_FullRM_log1;
GO
And it says:
This example creates a transaction log backup for the
AdventureWorks2012 database to the previously created named backup
device, MyAdvWorks_FullRM_log1.
I'm unsure what they mean by "previously created backup device". I have numerous filenames, because of the striping. How do I do a log file backup, of my full backup?
There may be other backup processes that go on on this database. Will this cause me issues with the log file backups (Another full backup of the database might clear my transaction log before I get my backup done?) How should my daily log file backup look?
I'm unsure what they mean by "previously created backup device".
This means that in tis example they previously created virtual backup device.
So now instead of using syntax backup...**to disk** =... they use
backup ...to <device_name>
You can completely ignore this and backup log to differen files as all the world do:
BACKUP LOG AdventureWorks2012
TO disk = 'V:\backups\log\AdventureWorks2012_20190918.trn;
I have numerous filenames, because of the striping. How do I do a log file backup, of my full backup?
Log backup is not done "of full backup".
First you create a "base" by taking full backup, and it doesn't matter if you stripe it into many files or not, then you take log backup (log backups generally are pretty small and you don't need to strip)
There may be other backup processes that go on on this database. Will
this cause me issues with the log file backups (Another full backup of
the database might clear my transaction log before I get my backup
done?)
Full backups are not a problem for you as they just create other "bases" from which you can start your restore chain. You will have problems if others take log backups, as it will interrupt log backup chain (it's not full backup to clear the log, log truncation is done only by log backups).
You should talk to those who take these log backups if they can take them with copy_only: Copy-Only Backups (SQL Server)
or if they can leave these backups for you on some disk.
I have Windows scheduler calling a program that does a full database backup every day at 3:00am.
I would also like to do a transaction log backup every ten minutes.
What is the best way to sync these?
I understand that transaction logs are independent of full back-ups, but is it a problem if the two different tasks both do a backup at 3:00am? i.e. the database is asked to produce a full back-up and a transaction log back-up at exactly the same time.
Perhaps I should have one task and query SQL Server to see if the last full backup was more than 24 hours ago. If not, then do a transaction log backup.
As long as you're running SQL 2005 or higher, you can run log backups at the same time as a full backup without issue. Glossing over a bit of detail, log backups are based on the last completed full backup at the time that the log backup started. The only time it really matters is when you'd need to do a recovery and even then, it's only ever "contentious" (i.e. "which full backup should I restore given that I want to restore to this point in time?") for log backups that are taken while a full backup is in progress.
If ever you need to do a point-in-time restore for such a time, you'd work backwards. That is, first find the log backup that contains said point in time. Then, to find the full backup that it's based on, look at the database backup lsn for that log backup in either msdb.dbo.backupset or by running restore headeronly on the log backup. From there, find the full backup that has that value as the checkpoint lsn. Now, you just need to restore that full backup (using the norecovery option) and every log backup based on it up to and including the log backup you identified in the first step (again, all using norecovery). Then run restore database [yourDB] with recovery to run crash recovery and you're back in business.
I have a SQL Server 2005 .BAK file (created with a maintenance plan) that I am trying to restore into a different instance of SQL Server 2005.
Regardless of whether I use the UI or RESTORE DATABASE, the restore process seems to halt at 99%. If I use RESTORE VERIFYONLY FROM DISK='mybackup.bak' it reports that my backup is valid.
Interestingly, immediately after I start the restore process, a 43 GB transaction log file appears in my MSSQL\DATA directory. The .BAK file is 60 MB. The system has more than enough free space for this recovery.
Any suggestions on other restore strategies I should try? I'm going to let the restore run overnight and see what happens.
Thanks
If you have a 43Gb transaction log, depending on the speed of your disks, the restore will take a while. What that means is that when you backed up your db, it had a 43Gb log file and it was empty. So when you restore, the SQL Server has to go and allocate 43Gb for the log file, and it has to physically go and zero out 43Gb, and this takes a while.
What I suggest is to wait a while for restore to complete, couple hours maybe, or do a transaction log backup on production, then shrink the trans log file to say 100Mb and then do a backup, and restore from that.
IT is very important to understand recovery models, and backup models used. Dont just slap a db on production and let it run, if you dont manage it, you will end up with big problems.
I'm using SQL Server 2005.
I need to move a database to a new server with minimum downtime.
Currently my option is to take everything offline, perform a backup, copy the backup to the new server, restore the backup and bring everything back online.
The problem here is that the backup file is about 10Gb, therefore, it takes about 5-10 mins to perform this whole operation.
I've heard of people performing a full backup with everything online and then doing a differential backup so there is less to copy over.
I've taken the full backup, copied that to the new server and restored it. I've then taken a differential backup and copied it to the new server.
The problem is, when I try and restore this backup I get the message "The log or differential backup cannot be restored because no files are ready to rollforward."
I have also tried selecting both the full and differential backup but then I get the error "The volume on device 'D:\FullBackup.bak' is not part of a multiple family media set. BACKUP WITH FORMAT can be used to form a new media set."
Does anyone know what I'm doing wrong and what the easiest way of doing this is?
Thanks
It seems you did not set the NORECOVERY option on the full RESTORE.
You need this option to leave all transactions intact so that you can apply the differential backup later.
If you don't set it, the uncommited transactions get rolled back while restoring the full backup, and the last LSN of the database becomes greater than the first LSN of the differential backup.
Another quick way of doing this is to Detach the database, and then copy the underlying .mdf and .ldf files for your DB straight to the DB, then reattach. Time taken to copy a 10GB file will be much less than the Backup, and it acheives the same thing.