I recently had to root out the cause of intermittent ODBC errors in a 12 year old SQL Based application. The culprit turned out to be this statement at the end of one of the stored procedures:
DUMP TRAN NotTheRealDBName WITH NO_LOG
I wasn't familiar with this command, so I did some research and found that it makes a backup of the transaction log. I believe this would only be useful if you at some point restored the transaction log, which didn't seem to be happening. The stored procedure in question was a simple sequence of inserts, updates, selects, and deletes on various tables.
I'm trying to imagine why one would want to backup and restore the transaction log in the middle of business logic, but I can't get my mind around it. Can anyone give me an example of an efficacious use of DUMP TRAN?
They key part is that the program also adds the WITH NO_LOG clause. So basically it truncates the log. Probably 12 years ago the author had run into log growth issues, 'thoroughly' researched the issue and concluded that the best avenue is to truncate the log whenever he was doing some lengthy updates. Nevermind that in the process he breaks the backup chain of any maintenance plan...
This command is used when you want to shrink your database; I never saw it used in business logic and I believe it to be actually dangerous when used that way.
DUMP is now BACKUP and I'm pretty sure the NO_LOG option has been discontinued in SQL Server 2008 (and with good cause).
Basically, if you configured your database improperly - if you used the "Full" recovery model but didn't actually do proper backups - then this was a way to shrink the log.
I can think of very few good reasons why you'd want to truncate the log without backing it up, and I also can't think of many reasons why you'd want to back up the log in the middle of business logic, but there is a very good reason to backup the transaction log in general, and that is to meet RPO targets.
Generally you're only going to be doing one database backup per day, but if you have a business requirement to be able to recover half a day's worth of transactions if the server blows up or some rogue DBA manages to hose your production database and all mirrors, then you'll likely be making hourly (or more frequent) transaction log backups. That way, you can restore the backup from yesterday night and use the transaction log backups to restore up to an hour ago (or whatever the RPO is).
Related
I have a database with an LDF file of around 14GB and an MDF file of 12GB.
No one has ever performed a Transaction log backup for this DB.
Though the team always just perform Full DB Backup every morning.
Resulting in the log file size is just growing and growing.
Now, If I take its transaction log backup and truncate log file.
Am I going to lose any data?
I am curious because taking log backup reduces log file size, I just want to make sure that their won't be any impact of my MDF file or Data that we have in DB.
And after taking t-log backup whats the good strategy?
Should we restore it to check if it works fine?
If I restore a t-log backup, is it mandatory to apply Full Backup restore first on that Testing server?
[SQL Server 2012]
The short answer is "you wouldn't lose any more than what you normally would otherwise". However, there are several circumstances that might affect your final decision.
The first question your team should ask itself is, of course, "if we never take log backups, why the database is in the full recovery mode"? For DEV databases, log backups are unnecessary in most cases, so switch it to SIMPLE recovery, issue a checkpoint and then you can truncate the log. Oh, and it shouldn't grow much after that, unless someone would run a large modification batch in a single transaction.
Just make sure you understand the difference between the truncation modes. Most probably, you will need to utilise both NOTRUNCATE and TRUNCATEONLY.
Whether you should start taking log backups regularly depends on your workflow and the importance of the data. The main benefits of the full recovery mode are:
In case of a disaster, you don't lose modifications made after the last full / diff backup, only after the last transaction log backup (which are usually short if taken regularly, and as such put less strain on a server). However, if your transaction log file survived the crash and you were able to perform a tail-log backup after the database became unusable, you lose nothing.
You can perform a point-in-time restore of your database, which sometimes is a crucial capability - for example, when you investigate a data corruption / loss, or for any other investigation of that kind. However, as I've said, development databases are rarely that precious.
So weigh these options, consider your specifics, and the answer will come. If it doesn't, I recommend turning your attention to dba.stackexchange.com, as the question in its current form isn't really about development.
your database recovery model is full that means sql don't over write into ldf data pages until you get log backup , and then over write them
after that your log don't get larger unless you have a huge transaction
so , you may declare some log backup job or change recovery model to simple
and of course the full recovery model is better but remember you should keep every single log backup that you take after last full backup
I've recently taken on the database administration of a few SQL servers varying from SQL Server 2005 to 2014, where many of the DB's are in Full recovery mode, however no good ongoing backup maintenance plans were ever setup.
It seems to me that the previous DBA would only deal with transaction log files when they got out of control and fill up the hard drive. So i'd like to change this and fix the issue once and for all. I've been doing some reading and think I have a decent understanding of what need to be done, so i'd like to validate my understanding as well as ask a few question to clarify a few points that I still don't fully get.
So based on my understanding to date i would need to create a maintenance plan which starts with a Full Backup. I still need to talk to management to figure out things like RTO, acceptable data loss, etc so let's assume for this example that we'll do Full Backup's on Sunday.
Next I would add to this maintenance plan Differential backups every night... so Monday to Saturday. I realize that this could also be Full backup's or run the differentials more frequently, but again this is just as an example to make sure i'm understanding things correctly.
Now as for the transaction log backups. I get that i would need to back these up and truncate the log file to prevent it from continually growing and getting out of control. I don't know if there are any specific recommendation for how often to back this up, but i've seen 15 minutes suggested. I guess this would more so fall under the acceptable data loss window. Is that correct?
So the other thing that i've discovered is that when you backup the transaction log file with truncation, that if the log file has already grown out of control that it doesn't shrink the file. I've also read that it isn't good to shrink these files, at least on regular basis because once you shrink it, it would need to grow again and this will cause fragmentation and performance issues.
Now since I am currently in a situation that the files have already grown out of control, i'd assume that i should in fact shrink the log files this one time once i've but my maintenance in place. Is that assumption correct?
Also once i Shrink the transaction log file this one time, are there any maintenance task that i should be running to avoid performance issue due to shrinking the log files?
One other question that i was wondering about would be with respect to point in time recovery. So let's say I take a full backup at 5:00 AM and i also take a transaction log backup every 15 minutes. I get alerted that at 6:18 AM something has gone wrong (let's say a table was deleted). So i know i can restore by Full backup that happened at 5:00 AM and leave it in NO RECOVERY mode and restore all of the Transaction log backup from 5:15 AM to 6:15 AM, but here is what i'm interested in...since i have my DB in full recovery mode, is it possible to somehow use my existing transaction log file (not the backups) to roll forward all transaction between 6:15 and 6:17 just before the table was deleted? If so how would you do this? I guess this obviously wouldn't work in the case of you loosing the hard drive with your transaction log files, or your server exploding...but in a case like i've outlined is it do able?
Thanks
I would recommend doing a full backup after everyone stopped working, e. g. at 10 p.m. (if that is the case), not in the morning shortly before people start working. Just in order to give it enough time to run.
Personally, I prefer doing daily full backups instead of incremental backups if the database is not too big to save backups for, say, 14 days. I feel better to rely on less files. If database and full backups are too big, incremental backups might be the better choice.
As you said: How many transaction log backups you create during the day depends on the acceptable data loss window. In an environment > 5 people working on the system (just as a gut feeling) I would configure them to run all 15 minutes, on very big systems maybe even more.
After the first transaction log backup, you might want to shrink the LOG file ONCE.
I think it's not necessary to run any optimizations after a log shrink.
As far as I know it's not possible to restore transactions between 06:15 and 06:17.
When activating the transaction log backups, keep in mind that the first transaction log backup will be quite big (around the size of the current, large log). Ensure to have enough space on disk until you shrink the log file and delete the first transaction log (usually done automatically within the maintenance plan, e. g. after 14 days.).
I have several SSIS packages jobs running, and some months ago my disk got full because the size of the SSISDB database.
I noticed that the cleanup_server_retention_window was set to 365 days, and I changed it to one day. (It is a development server, and at this point I really dont care about the history).
The (big) problem, obviously, is now that the transaction log grows a lot and fast.
To prevent this, I start performing a full backup every week and a transaction log backup every day, and the size of the database is now controlled.
However, some more experienced guys are telling me that this is not the best aproach to this issue, but I cant see any problem with it..
I would like to know if there is a better solution for this.
I tried just about everything including changing retention window; it was deleted the transactions but not reducing the log size. For me the allocated log file size grew to 75 GB. Nothing seemed to help.
The main issue has to do with the recovery model
of the SSIS DB that was set to 'Full'. Once I set this to 'Simple' and changed the initial log file size, all was fixed!
I have been monitoring this for the last couple of days just to make sure all is well and it looks fine to me so this operation is safe.
The current log file size is 512KBMB as opposed to 75GB!
The (big) problem, obviously, is now that the transaction log grows a lot and fast.
You will not see this everyday..The Cause of transaction log growth was changing cleanup_server_retention_window ..when you changed the value from 365 to 1,internally it has to do a lot of deletes
I start performing a full backup every week and a transaction log backup every day, and the size of the database is now controlled
I don't see an issue with backing up SSISD.In our instance ,we changed the recovery model to simple and do daily full backups
I fixed this in 3 ways:
adding some missing indexes on the SSISDB database; they should be there after installation of CU4
changed the parameter #delete_batch_size from 1000 to 25 in the stored procedure internal.cleanup_server_retention_window
changed the recoevry model from Full to Simple
Now, when running the SSISDB maintenance job, the transaction log no longer fills up beyond repair causing a database 'crash' / rollback
I am looking into using log shipping in a SQL Server 2005 environment. The idea was to set up frequent log shipping to a secondary server. The intent: Use the secondary server to serve report queries, thereby offloading the primary db server.
I came across this on a sqlservercentral forum thread:
When you create the log shipping you have 2 choices. You can configure restore log operation to be done with norecovery or with standby option. If you use the norecovery option, you can not issue select statements on it. If instead of norecovery you use the standby option, you can run select queries on the database.
Bear in mind with the standby option when log file restores occur users will be kicked out without warning by the restore process. Acutely when you configure the log shipping with standby option, you can also select between 2 choices – kill all processes in the secondary database and perform log restore or don’t perform log restore if the database is being used. Of course if you select the second option, the restore operation might never run if someone opens a connection to the database and doesn’t close it, so it is better to use the first option.
So my questions are:
Is the above true? Can you really not use log shipping in the way I intend?
If it is true, could someone explain why you cannot execute SELECT statements to a database while the transaction log is being restored?
EDIT:
First question is duplicate of this serverfault question. But I still would like the second question answered: Why is it not possible to execute SELECT statements while the transaction log is being restored?
could someone explain why you cannot
execute SELECT statements to a
database while the transaction log is
being restored?
Short answer is that RESTORE statement takes an exclusive lock on the database being restored.
For writes, I hope there is no need for me to explain why they are incompatible with a restore. Why does it not allow reads either? First of all, there is no way to know if a session that has a lock on a database is going to do a read or a write. But even if it would be possible, restore (log or backup) is an operation that updates directly the data pages in the database. Since these updates go straight to the physical location (the page) and do not follow the logical hierarchy (metadata-partition-page-row), they would not honor possible intent locks from other data readers, and thus have the possibility to change structures as they are read. A SELECT table scan following the page next-prev pointers would be thrown into disarray, resulting in a corrupted read.
Well yes and no.
You can do exactly what you wish to do, in that you may offload reporting workloads to a secondary server by configuring Log Shipping to a read only copy of a database. I have set this type of architecture up on a number of occasions previously and it works very well indeed.
The caveat is that in order to perform a restore of a Transaction Log Backup file there must be no other connections to the database in question. Hence the two choices being, when the restore process runs it will either fail, thereby prioritising user connections, or it will succeed by disconnecting all user connection in order to perform the restore.
Dependent on your restore frequency this is not necessarily a problem. You simply educate your users to the fact that, say every hour at 10 past the hour, there is a possibility that your report may fail. If this happens simply re-run the report.
EDIT: You may also want to evaluate alternative architeciture solutions to your business need. For example, Transactional Replication or Database Mirroring with a Database Snapshot
If you have enterprise version, you can use database mirroring + snapshot to create read-only copy of the database, available for reporting, etc. Mirroring uses "continuous" log shipping "under the hood". It is frequently used in scenario you have described.
Yes it's true.
I think the following happens:
While the transaction log is being restored, the database is locked, as large portions of it are being updated.
This is for performance reasons more then anything else.
I can see two options:
Use database mirroring.
Schedule the log shipping to only occur when the reporting system is not in use.
Slight confusion in that, the norecovery flag on the restore means your database is not going to be brought out of a recovery state and into an online state - that is why the select statements will not work - the database is offline. The no-recovery flag is there to allow you to restore multiple log files in a row (in a DR type scenario) without bringing the database back online.
If you did not want to log ship / have the disadvantages you could swap to a one way transactional replication, but the overhead / set-up will be more complex overall.
Would peer-to-peer replication work. Then you can run queries on one instance and so save the load on the original instance.
I have a SQL Server 2005 database that is backed up nightly. There backup consists of:
FULL backup of the database.
backup of the transaction log.
These are currently two separate jobs.
The log is huge and I'd like to set things up so that:
the database is backed up in full nightly
the log is set such that I can recover the database from any point between one backup and the next.
How can I set this up so that the log files are manageable? I suspect that the log has never been shrunk, as the log is huge.
You are currently implementing the FULL Recovery Model from the sound of things. This will allow you to restore to a point in time provided that you have a transaction log backup that covers the desired point in time (post full backup).
In order to reduce the size of your required transaction log file, you should look to increase the frequency of your transaction log backups. I would suggest hourly. Once you have gauged the actual usage of your log file, you can then look to shrink it to a more suitable size. The key point to note here is that once a transaction log backup has been completed, the inactive portion of the log file becomes available for use once again. The reason why a transaction log file grows continuously is if the transaction log backups are either, not being taken at all or their frequency is not sufficient.
I would also suggest that you consider performing a mix of DIFFERENTIAL and FULL Backups in order to reduce the collective size of your backed up data. An example schedule would be a weekly FULL Backup, say every Sunday, with daily DIFFERENTIAL backups.
I hope what I have detailed makes sense. Please feel free to contact me directly and I will happily assist you in deploying an appropriate backup strategy for your environment.
Essential References:
How to stop the transaction log
file from growing enexpectedly
Backup and Restoring Databases in
SQL Server
One of the things I find with backups is that people typically don't run them frequently enough - especially when it comes to log file backups. And it sounds like you're correct, that the log file isn't being truncated regularly (which means you're likely wasting premium disk space [1]). More importantly though, that's leaving you completely exposed from a recoverability standpoint.)
Happily though, getting things up and running as you need them isn't so hard. In fact, I'd recommend the following three videos as they should give you the background info you need, and then the step-by-step instructions you'll want to follow to get everything working correctly:
http://www.sqlservervideos.com/video/logging-essentials
http://www.sqlservervideos.com/video/sql2528-log-files
http://www.sqlservervideos.com/video/sqlbackup-best-practices
1 Maximize Storage Performance: http://www.sqlmag.com/Article/ArticleID/100893/sql_server_100893.html
What you are doing is effectively a SIMPLE mode backup with bonus disadvantage of not shrinking the log. There is no point to back up both at the same time. If you're doing a full backup, you can just truncate the log.
If you're going to be able to restore to any point of time, you will have to do a full backup once a day (say) and back up the log few times during the day. See http://msdn.microsoft.com/en-us/library/ms191429(SQL.90).aspx