I have a SQL Server 2005 database that is backed up nightly. There backup consists of:
FULL backup of the database.
backup of the transaction log.
These are currently two separate jobs.
The log is huge and I'd like to set things up so that:
the database is backed up in full nightly
the log is set such that I can recover the database from any point between one backup and the next.
How can I set this up so that the log files are manageable? I suspect that the log has never been shrunk, as the log is huge.
You are currently implementing the FULL Recovery Model from the sound of things. This will allow you to restore to a point in time provided that you have a transaction log backup that covers the desired point in time (post full backup).
In order to reduce the size of your required transaction log file, you should look to increase the frequency of your transaction log backups. I would suggest hourly. Once you have gauged the actual usage of your log file, you can then look to shrink it to a more suitable size. The key point to note here is that once a transaction log backup has been completed, the inactive portion of the log file becomes available for use once again. The reason why a transaction log file grows continuously is if the transaction log backups are either, not being taken at all or their frequency is not sufficient.
I would also suggest that you consider performing a mix of DIFFERENTIAL and FULL Backups in order to reduce the collective size of your backed up data. An example schedule would be a weekly FULL Backup, say every Sunday, with daily DIFFERENTIAL backups.
I hope what I have detailed makes sense. Please feel free to contact me directly and I will happily assist you in deploying an appropriate backup strategy for your environment.
Essential References:
How to stop the transaction log
file from growing enexpectedly
Backup and Restoring Databases in
SQL Server
One of the things I find with backups is that people typically don't run them frequently enough - especially when it comes to log file backups. And it sounds like you're correct, that the log file isn't being truncated regularly (which means you're likely wasting premium disk space [1]). More importantly though, that's leaving you completely exposed from a recoverability standpoint.)
Happily though, getting things up and running as you need them isn't so hard. In fact, I'd recommend the following three videos as they should give you the background info you need, and then the step-by-step instructions you'll want to follow to get everything working correctly:
http://www.sqlservervideos.com/video/logging-essentials
http://www.sqlservervideos.com/video/sql2528-log-files
http://www.sqlservervideos.com/video/sqlbackup-best-practices
1 Maximize Storage Performance: http://www.sqlmag.com/Article/ArticleID/100893/sql_server_100893.html
What you are doing is effectively a SIMPLE mode backup with bonus disadvantage of not shrinking the log. There is no point to back up both at the same time. If you're doing a full backup, you can just truncate the log.
If you're going to be able to restore to any point of time, you will have to do a full backup once a day (say) and back up the log few times during the day. See http://msdn.microsoft.com/en-us/library/ms191429(SQL.90).aspx
Related
I have a database with an LDF file of around 14GB and an MDF file of 12GB.
No one has ever performed a Transaction log backup for this DB.
Though the team always just perform Full DB Backup every morning.
Resulting in the log file size is just growing and growing.
Now, If I take its transaction log backup and truncate log file.
Am I going to lose any data?
I am curious because taking log backup reduces log file size, I just want to make sure that their won't be any impact of my MDF file or Data that we have in DB.
And after taking t-log backup whats the good strategy?
Should we restore it to check if it works fine?
If I restore a t-log backup, is it mandatory to apply Full Backup restore first on that Testing server?
[SQL Server 2012]
The short answer is "you wouldn't lose any more than what you normally would otherwise". However, there are several circumstances that might affect your final decision.
The first question your team should ask itself is, of course, "if we never take log backups, why the database is in the full recovery mode"? For DEV databases, log backups are unnecessary in most cases, so switch it to SIMPLE recovery, issue a checkpoint and then you can truncate the log. Oh, and it shouldn't grow much after that, unless someone would run a large modification batch in a single transaction.
Just make sure you understand the difference between the truncation modes. Most probably, you will need to utilise both NOTRUNCATE and TRUNCATEONLY.
Whether you should start taking log backups regularly depends on your workflow and the importance of the data. The main benefits of the full recovery mode are:
In case of a disaster, you don't lose modifications made after the last full / diff backup, only after the last transaction log backup (which are usually short if taken regularly, and as such put less strain on a server). However, if your transaction log file survived the crash and you were able to perform a tail-log backup after the database became unusable, you lose nothing.
You can perform a point-in-time restore of your database, which sometimes is a crucial capability - for example, when you investigate a data corruption / loss, or for any other investigation of that kind. However, as I've said, development databases are rarely that precious.
So weigh these options, consider your specifics, and the answer will come. If it doesn't, I recommend turning your attention to dba.stackexchange.com, as the question in its current form isn't really about development.
your database recovery model is full that means sql don't over write into ldf data pages until you get log backup , and then over write them
after that your log don't get larger unless you have a huge transaction
so , you may declare some log backup job or change recovery model to simple
and of course the full recovery model is better but remember you should keep every single log backup that you take after last full backup
I've recently taken on the database administration of a few SQL servers varying from SQL Server 2005 to 2014, where many of the DB's are in Full recovery mode, however no good ongoing backup maintenance plans were ever setup.
It seems to me that the previous DBA would only deal with transaction log files when they got out of control and fill up the hard drive. So i'd like to change this and fix the issue once and for all. I've been doing some reading and think I have a decent understanding of what need to be done, so i'd like to validate my understanding as well as ask a few question to clarify a few points that I still don't fully get.
So based on my understanding to date i would need to create a maintenance plan which starts with a Full Backup. I still need to talk to management to figure out things like RTO, acceptable data loss, etc so let's assume for this example that we'll do Full Backup's on Sunday.
Next I would add to this maintenance plan Differential backups every night... so Monday to Saturday. I realize that this could also be Full backup's or run the differentials more frequently, but again this is just as an example to make sure i'm understanding things correctly.
Now as for the transaction log backups. I get that i would need to back these up and truncate the log file to prevent it from continually growing and getting out of control. I don't know if there are any specific recommendation for how often to back this up, but i've seen 15 minutes suggested. I guess this would more so fall under the acceptable data loss window. Is that correct?
So the other thing that i've discovered is that when you backup the transaction log file with truncation, that if the log file has already grown out of control that it doesn't shrink the file. I've also read that it isn't good to shrink these files, at least on regular basis because once you shrink it, it would need to grow again and this will cause fragmentation and performance issues.
Now since I am currently in a situation that the files have already grown out of control, i'd assume that i should in fact shrink the log files this one time once i've but my maintenance in place. Is that assumption correct?
Also once i Shrink the transaction log file this one time, are there any maintenance task that i should be running to avoid performance issue due to shrinking the log files?
One other question that i was wondering about would be with respect to point in time recovery. So let's say I take a full backup at 5:00 AM and i also take a transaction log backup every 15 minutes. I get alerted that at 6:18 AM something has gone wrong (let's say a table was deleted). So i know i can restore by Full backup that happened at 5:00 AM and leave it in NO RECOVERY mode and restore all of the Transaction log backup from 5:15 AM to 6:15 AM, but here is what i'm interested in...since i have my DB in full recovery mode, is it possible to somehow use my existing transaction log file (not the backups) to roll forward all transaction between 6:15 and 6:17 just before the table was deleted? If so how would you do this? I guess this obviously wouldn't work in the case of you loosing the hard drive with your transaction log files, or your server exploding...but in a case like i've outlined is it do able?
Thanks
I would recommend doing a full backup after everyone stopped working, e. g. at 10 p.m. (if that is the case), not in the morning shortly before people start working. Just in order to give it enough time to run.
Personally, I prefer doing daily full backups instead of incremental backups if the database is not too big to save backups for, say, 14 days. I feel better to rely on less files. If database and full backups are too big, incremental backups might be the better choice.
As you said: How many transaction log backups you create during the day depends on the acceptable data loss window. In an environment > 5 people working on the system (just as a gut feeling) I would configure them to run all 15 minutes, on very big systems maybe even more.
After the first transaction log backup, you might want to shrink the LOG file ONCE.
I think it's not necessary to run any optimizations after a log shrink.
As far as I know it's not possible to restore transactions between 06:15 and 06:17.
When activating the transaction log backups, keep in mind that the first transaction log backup will be quite big (around the size of the current, large log). Ensure to have enough space on disk until you shrink the log file and delete the first transaction log (usually done automatically within the maintenance plan, e. g. after 14 days.).
Most of the time users will hit the database to read news. There are very few number of queries executed under transactions. 95% of the database hits would be for read-only purposes.
My database log files size is growing 1 GB per day. Even if I shrink the database, the log file size is not decreasing. What could be the reason for growing the log file size more and more? How can I control this? As per my knowledge log file does not increase when we read data from tables.
Any suggestions on how to deal with the log file growing? How can it be kept a manageable or reasonable size? Does this effect performance in any way?
There are couple of things to consider. What type of backups you do, and what type of backups you need. If you will have the answer to this question you can either switch Recovery Mode to simple or leave it full but then you need to make incremental backups on a daily basis (or whatever makes you happy with log size).
To set your database logging to simple (but only if you do Full Backups of your database!).
Right click on your database
Choose Properties
Choose Options
Set Recovery mode to simple
This will work and is best if your backup schedule is Full Backup every day. Because in such scenario your log won't be trimmed and it will skyrocket (just like in your case).
If you would be using Grandfather&Father&Son backup technique, which means Monthly Full backup, Weekly Full backup, and then every day incremental backup. Then for that you need Full Recovery Mode. If 1GB of log per day is still too much you can enable incremental backup per hour or per 15 minutes. This should fix the problem of log growing more and more.
If you run Full Backup every day you can switch it to simple recovery mode and you should be fine without risking your data (if you can leave with possible 1 day of data being lost). If you plan to use incremental then leave it at Recovery Mode Full.
Full backups will not help, you must regularly backup the transaction log (as well as the regular database full and differential backups) for it to be emptied. If you are not backing up the log and you are not in simple recovery mode, then your transaction log has every transaction in it since the database was set up. If you have enough action that you are growing by a gig a day, then you may also have large imports or updates affecting many records at once. It is possible you need to be in simple recovery mode where transactions are not recorded individually. Do NOT do that however if you have a mix of data from imports and users. In that case you need to back up the transaction log frequently to be able to keep the size manageable and to a point in time. We backup our transaction log every 15 minutes.
Read about transaction log backups in BOL to see how to fix the mess you have right now. Then get your backups set up and running properly. You need to read and understand this stuff thoroughly before attempting to fix. Right now, you would probably be in a world of hurt if your server failed and you had to recover the database. Transaction log backups are critical to being able to recover properly from a failure.
Do you backup your database frequently? You need to perform full- and/or transaction log- backups in order for SQL Server to consider shrinking your log file.
The major reason for the log file large size is due to bulk transaction in DB. To reduce the log file size best option to take the transaction log backup after certain interval of time.
I have a nightly job that does a bunch of inserts. Since I have a full recovery model, this increases my transaction log size.
Currently I have my log file big enough to accommodate these transactions, but the issue is that the transaction log is mostly empty throughout the day.
Is it an issue (besides disk space) to have a huge (mostly empty) transaction log?
I'm thinking about switching the database to simple recovery before the job, running the job and then switching it back to full recovery. I can have the transaction logs just not be backed up until our nightly differential backup comes around and then i can start the transaction log backup again.
Suggestions?
I'd do nothing. Or have SIMPLE permanently.
Changing the recovery model back to full will require a full backup anyway to preserve integrity later on. You'll have a gap in your LSN chains otherwise.
You've mentioned differential backup, so I assume your full is not each night.
So, putting this together means you'll use more disk space for your full backup than for the LDF file.
I'd keep it FULL and, if push comes to shove, force log backups during the nightly job to keep the log size small. Also, is OK to shrink the log if you really have to, ie. it grew once due to a one time operation and it won't grow back. The fragmentation concerns are more valid for data files, log files have a totally different structure and allocation pattern. Just don't get into the habit of constant shrinking.
I'm saying this because if you do have a backup restore strategy in place already it seems silly to me to increase the window of data loss from 'last log backup' to 'last differential backup'. We're talking a change from 10-30 minutes (typical log backup frequency) to 3-24 hours (typical differential frequency). You won't be able to do differentials as often as log backups because differentials are growing in size (starting with second differential after full each differential is at least as big as previous differential). Log backups only backup the log since last backup, so they stay relative constant in size. Also with a SIMPLE mode you won't be able to do a log tail backup attempt and recover all data in case of crash.
It just seems you're trading off a lot more that you gain with reducing the log file size.
This question might be kind of elementary, but here goes:
I have a SQL Server database with a 4 GB log file. The DB is 16GB and is backed up nightly.
Can I truncate the log regularly because the entire DB+Log is backed up each night?
you can something like this to you maintenance schedule to run every night before the backup. This will try to shrink/truncate your log file to 1 meg
BACKUP LOG DBNAME
TO disk = 'D:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Backup\DBNAME.log'
DBCC SHRINKFILE('DBNAME_Log', 1)
Are you sure the log is backed up nightly and not just the database?
If so, then what does this database do? Are you deleting and refreshing whole tables? If so, your log might be the right size for the amount of transactions you have. You want the log to be large enough to handle your normal transaction load without having to grow. A small log can be a detriment to performance.
If this database is not transactional in nature (i.e., the tables are populated by full refreshes rather than one record ata time), the change the recovery mode to simple. Do not do that though if you have transactional tables that you will need to be able to recover from the log rahter than simply re-importing the data.
If you can run log backups during the day (depending on load, etc. this may or may not be possible for you) you can keep the log file under control by doing so. This will prevent the log file itself from growing quite so large, and also provide the side benefit of giving you the ability to restore closer to the point of failure in the event of a problem.
You'll still need to shrink the log file once using DBCC SHRINKFILE, but if it's backed up regularly after that point it shouldn't stabilize at a smaller size.