I understand that the transaction logs keep a record of historical transactions in order to facilitate a restore if needed. However do I need to keep creating transaction log backups for inactive databases that are hanging around on the server? No DDL statements are run against them and they are just used for reference.
I am just a bit worried that I might run out of log space if I get this wrong.
Have you considered changing the recovery model of your databases to the SIMPLE recovery model? Doing so would negate the need to backup the transaction log as it would be automatically re-used in the "unlikely" event that you need it to be.
I would still advise that regular FULL database backups be taken.
Also, if these database are indeed true read only databases then why not consider setting them to be so. This action would have the advantage of immediately highlighting any queries/users that are "still" issuing DML operations when you believe there to be none.
Other options for identifying queries that are performing more than just READ operations include running a Profiler Trace of activity on your database server and also an aggressive option would be to revoke all data modification rights from the relevant database Users.
Transaction logs are actually truncated when they're backed up. So, if these databases are actually inactive, you shouldn't be backing up any transaction logs for them since the logs would be empty.
Also, common practice for "inactive" databases would be to make them READ ONLY with a SIMPLE recovery model.
Related
I have a sql database that I dump data into every 15 minutes using SSIS. The transaction log get's huge and I back it up and shrink it a few times a week. But I know I'm doing something wrong. What is the best practice for me to maintain it? Should it stay ~1GB and I should be backing it up hourly? Since it's a datawarehouse, should I be backing it up at all? Show I be doing something different in SSIS? The datawarehouse recovery model is Bulk Logged.
You should also look to make sure SSIS is setup correct to do minimally logged operations as you may not have it set correctly. Then once that is working correctly, evaluate if you really need bulk logged recovery model: http://msdn.microsoft.com/en-us/library/ms175987(v=sql.105).aspx is good link on subject. If you can quickly (your definition of quick) redo the data that is lost and don't want point in time recovery of non-bulk operations, move to simple recovery, IMHO.
You have a few options with logs and they are related to how you are backing up your database and how much data loss you can stand. If you are doing full backups nightly for example...change you database to simple mode. This truncates your log on commit (meaning once logged operations complete the log cleans itself up) ... there are a lot of articles on this topic ... Google it up and if you need more help post back.
I am looking into using log shipping in a SQL Server 2005 environment. The idea was to set up frequent log shipping to a secondary server. The intent: Use the secondary server to serve report queries, thereby offloading the primary db server.
I came across this on a sqlservercentral forum thread:
When you create the log shipping you have 2 choices. You can configure restore log operation to be done with norecovery or with standby option. If you use the norecovery option, you can not issue select statements on it. If instead of norecovery you use the standby option, you can run select queries on the database.
Bear in mind with the standby option when log file restores occur users will be kicked out without warning by the restore process. Acutely when you configure the log shipping with standby option, you can also select between 2 choices – kill all processes in the secondary database and perform log restore or don’t perform log restore if the database is being used. Of course if you select the second option, the restore operation might never run if someone opens a connection to the database and doesn’t close it, so it is better to use the first option.
So my questions are:
Is the above true? Can you really not use log shipping in the way I intend?
If it is true, could someone explain why you cannot execute SELECT statements to a database while the transaction log is being restored?
EDIT:
First question is duplicate of this serverfault question. But I still would like the second question answered: Why is it not possible to execute SELECT statements while the transaction log is being restored?
could someone explain why you cannot
execute SELECT statements to a
database while the transaction log is
being restored?
Short answer is that RESTORE statement takes an exclusive lock on the database being restored.
For writes, I hope there is no need for me to explain why they are incompatible with a restore. Why does it not allow reads either? First of all, there is no way to know if a session that has a lock on a database is going to do a read or a write. But even if it would be possible, restore (log or backup) is an operation that updates directly the data pages in the database. Since these updates go straight to the physical location (the page) and do not follow the logical hierarchy (metadata-partition-page-row), they would not honor possible intent locks from other data readers, and thus have the possibility to change structures as they are read. A SELECT table scan following the page next-prev pointers would be thrown into disarray, resulting in a corrupted read.
Well yes and no.
You can do exactly what you wish to do, in that you may offload reporting workloads to a secondary server by configuring Log Shipping to a read only copy of a database. I have set this type of architecture up on a number of occasions previously and it works very well indeed.
The caveat is that in order to perform a restore of a Transaction Log Backup file there must be no other connections to the database in question. Hence the two choices being, when the restore process runs it will either fail, thereby prioritising user connections, or it will succeed by disconnecting all user connection in order to perform the restore.
Dependent on your restore frequency this is not necessarily a problem. You simply educate your users to the fact that, say every hour at 10 past the hour, there is a possibility that your report may fail. If this happens simply re-run the report.
EDIT: You may also want to evaluate alternative architeciture solutions to your business need. For example, Transactional Replication or Database Mirroring with a Database Snapshot
If you have enterprise version, you can use database mirroring + snapshot to create read-only copy of the database, available for reporting, etc. Mirroring uses "continuous" log shipping "under the hood". It is frequently used in scenario you have described.
Yes it's true.
I think the following happens:
While the transaction log is being restored, the database is locked, as large portions of it are being updated.
This is for performance reasons more then anything else.
I can see two options:
Use database mirroring.
Schedule the log shipping to only occur when the reporting system is not in use.
Slight confusion in that, the norecovery flag on the restore means your database is not going to be brought out of a recovery state and into an online state - that is why the select statements will not work - the database is offline. The no-recovery flag is there to allow you to restore multiple log files in a row (in a DR type scenario) without bringing the database back online.
If you did not want to log ship / have the disadvantages you could swap to a one way transactional replication, but the overhead / set-up will be more complex overall.
Would peer-to-peer replication work. Then you can run queries on one instance and so save the load on the original instance.
Oracle has SQL commands that one can issue so that a transaction does not get logged. Is there something similar for SQL Server 2008?
My scenario: We need Tx logs on servers (Dev, QA, Prod), but maybe we can do without them on developer machines.
You can't do without transaction logs in SQL Server, under any circumstances. The engine simply won't function.
You CAN set your recovery model to SIMPLE on your dev machines - that will prevent transaction log bloating when tran log backups aren't done.
ALTER DATABASE MyDB SET RECOVERY SIMPLE;
There is a third recovery mode not mentioned above. The recovery mode ultimately determines how large the LDF files become and how ofter they are written to. In cases where you are going to be doing any type of bulk inserts, you should set the DB to be in "BULK/LOGGED". This makes bulk inserts move speedily along and can be changed on the fly.
To do so,
USE master ;
ALTER DATABASE model SET RECOVERY BULK_LOGGED ;
To change it back:
USE master ;
ALTER DATABASE model SET RECOVERY FULL ;
In the spirit of adding to the conversation about why someone would not want an LDF, I add this: We do multi-dimensional modelling. Essentially we use the DB as a large store of variables that are processed in bulk using external programs. We do not EVER require rollbacks. If we could get a performance boost by turning of ALL logging, we'd take it in a heart beat.
SQL Server requires a transaction log in order to function.
That said there are two modes of operation for the transaction log:
Simple
Full
In Full mode the transaction log keeps growing until you back up the database. In Simple mode: space in the transaction log is 'recycled' every Checkpoint.
Very few people have a need to run their databases in the Full recovery model. The only point in using the Full model is if you want to backup the database multiple times per day, and backing up the whole database takes too long - so you just backup the transaction log.
The transaction log keeps growing all day, and you keep backing just it up. That night you do your full backup, and SQL Server then truncates the transaction log, begins to reuse the space allocated in the transaction log file.
If you only ever do full database backups, you don't want the Full recovery mode.
What's your problem with Tx logs? They grow? Then just set truncate on checkpoint option.
From Microsoft documentation:
In SQL Server 2000 or in SQL Server
2005, the "Simple" recovery model is
equivalent to "truncate log on
checkpoint" in earlier versions of SQL
Server. If the transaction log is
truncated every time a checkpoint is
performed on the server, this prevents
you from using the log for database
recovery. You can only use full
database backups to restore your data.
Backups of the transaction log are
disabled when the "Simple" recovery
model is used.
If this is only for dev machines in order to save space then just go with simple recovery mode and you’ll be doing fine.
On production machines though I’d strongly recommend that you keep the databases in full recovery mode. This will ensure you can do point in time recovery if needed.
Also – having databases in full recovery mode can help you to undo accidental updates and deletes by reading transaction log. See below or more details.
How can I rollback an UPDATE query in SQL server 2005?
Read the log file (*.LDF) in sql server 2008
If space is an issue on production machines then just create frequent transaction log backups.
I read some Books Online about recovery/backup, one stupid question, if I use full database backup/full recovery model, for backup operation itself, will it generate any additional transaction log to source database server? Will full recovery operation generate additional transaction log to destination database?
A more useful view of this might be to say that Full Recovery prevents the contents of the transaction log from being overwritten without some other action allowing them to be overwritten
SQL Server will log most transactions (e.g. bulk load and a few others aside) and when running in simple recovery mode, effectively discard the newly created log contents at the end of the transaction associated with the creation of the same. When running in Full Recovery mode the contents of the trans log are retained until marked as available to be overwritten. To mark them as available to be overwritten one normally performs a backup (either Full or Trans Log).
If there is no space in the trans log and no logs contents marked as available to be overwritten then SQL Server will attempt to increase the size of the logs.
In practical terms Full Recovery requires you to manage your transaction logs, generally by performing a trans log backup every so often (every 1 hour is probably a good rule of thumb if you have no SLA to work to or other driver to determine how often to do this)
I'm not sure I completely understand your question, but here goes. Keeping your DB in Full Recovery mode can make your transaction logs grow to be very large. The trade off is that you can restore to the point of recovery.
The reason that the transaction logs are larger than normal is ALL transactions are fully logged. This can include bulk-logged operations, index creation, etc.
If drive space is not a concern (and with drives being so inexpensive, it shouldn't be), this is the recommended backup approach.
So our SQL Server 2000 is giving me the error, "The log file for database is full. Back up the transaction log for the database to free up some log space."
How do I go about fixing this without deleting the log like some other sites have mentioned?
Additional Info: Enable AutoGrowth is enabled growing by 10% and is restricted to 40MB.
To just empty it:
backup log <dbname> with truncate_only
To save it somewhere:
backup log <dbname> to disk='c:\somefile.bak'
If you dont really need transactional history, try setting the database recovery mode to simple.
Scott, as you guessed: truncating the log is a bad move if you care about your data.
The following, free, videos will help you see exactly what's going on and will show you how to fix the problem without truncating the logs. (These videos also explain why that's such a dangerous hack and why you are right to look for another solution.)
SQL Server Backups Demystified
SQL Server Logging Essentials
Understanding Backup Options
Together these videos will help you understand exactly what's going on and will show you whether you want to switch to SIMPLE recovery, or look into actually changing your backup routines. There are also some additional 'how-to' videos that will show you exactly how to set up your backups to ensure availability while managing log file sizing and growth.
ether backup your database logs regularly if you need to recover up to the minute or do other fun stuff like log shipping in the future, or set the database to simple mode and shrink the data file.
DO NOT copy, rename, or delete the .ldf file this will break your database and after you recover from this you may have data in an inconsistent state making it invalid.
I don't think renaming or moving the log file will work while the database is online.
Easiest thing to do, IMO, is to open the properties for the database and switch it to Simple Recovery Model. then shrink the database and then go back and set the DB to Full Recoery Model (or whatever model you need).
Changing the logging mode forces SQL Server to set a checkpoint in the database, after which shrinking the database will free up the excess space.
My friend who faced this error in the past recommends:
Try
Backing up the DB. The maintenance plan includes truncation of these files.
Also try changing the 'recovery mode' for the DB to Simple (instead of Full for instance)
Cause:
The transaction log swells up due to events being logged (Maybe you have a number of transactions failing and being rolled back.. or a sudden peaking in transactions on the server )
You may want to check related SO question:
How do you clear the transaction log in a SQL Server 2005 database?
Well you could take a copy of the transaction log, then truncate the log file, which is what the error message suggests.
If disk space is full and you can't copy the log to another machine over the network, then connect a drive via USB and copy it off that way.
You have the answer in your question: Backup the log, then it will be shrunk.
Make a maintenance plan to regularly backup the database and don't forget to select "Backup the transaction log". That way you'll keep it small.
If it's a non production environment use
dump tran <db_name> with no_log;
Once this has completed shrink the log file to free up disk space. Finally switch database recovery mode to simple.
As soon as you take a full backup of the database, and the database is not using the Simple recovery model, SQL Server keeps a complete record of all transactions ever performed on the database. It does this so that in the event of a catastrophic failure where you lose the data file, you can restore to the point of failure by backing up the log and, once you have restored an old data backup, restore the log to replay the lost transactions.
To prevent this building up, you must back up the transaction log. Or, you can break the chain at the current point using the TRUNCATE_ONLY or NO_LOG options of BACKUP LOG.
If you don't need this feature, set the recovery model to Simple.
My dear friend it is vey important for a DBA to check his log file quite frequently. Because if you don't give much attention towards it some day it is going to give this error.
For this purpose you have to periodically take back up so that the logs file would not faced such error.
Other then this the above given suggestion are quite right.
Rename it it. eg:
old-log-16-09-08.log
Then the SQL server can use a new empty one.