SonarQube database using too much space (60+ GB) - sql-server

I'm currently scanning 70 projects with a total of about 2 million lines of code. All has been going well until a few weeks ago I was notified that a few projects failed because we ran out of hard drive space on the SonarQube server. I was sure we had more than enough space according to the HW/SW requirements. I read that restarting the Sonarqube server service does clear up temp files, but after doing that several times, something is still eating up more space. The culprit is coming from SQL server:
...\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\sonarqube_log.ldf
The size of this file is currently at 66.8 GB. Does anyone know if I can just truncate the stuff in there, know of any best practices for reducing this file size, and keeping the size down during future scans?

OK, if your database recovery is set to Full and you're not backing up the transaction logs, that's the problem.
If you want help understanding the differences between recovery models, then start here. The short version is that Full recovery allows point-in-time (aka, point-of-failure) recovery, meaning you can restore the DB to the exact moment before the problem occurred. The drawback of Full recovery is that you must backup your transaction logs or this exact issue happens: the logs grow endlessly. Simple recovery eliminates the need for transaction log backups (indeed, I don't think you can do a transaction log backup on a Simple database) but limits you to restoring the DB to what it was when you last ran a database backup. Note that simple recovery still uses transaction logs! The system just periodically flushes them.
So, you will need to do one of two things: Use Full recovery and backup your transaction logs periodically (I've seen systems that do it every hour or even every 15 minutes for high traffic systems), or switch to Simple recovery. Those are your only real options.
Whichever one you do, switching to Simple or backing up the logs will flush the transaction logs. You can verify that with DBCC SQLPERF(logspace). However, you'll notice that sonarqube_log.ldf will not change size at all. It will still be 66.8 GB. This is by design. In a properly managed system, the transaction logs will reach the size that they need to operate, and then the backups and simple flushes will keep the size constant. The log file will be the proper size to run the system, so it will never need to grow (which is expensive) and will never run out of space (which would cause all transactions to fail).
"So, how do I get my disk space back?" you ask. "I've fixed the problem so the log file is going to be 95% wasted space now."
What you'll need to do is shrink the log file. This is something that you will often see written that you should never do. And, I agree, in a properly managed system, you should never need to do this. However, your system was not running properly. The log files were on runaway. In general, though, you shouldn't ever need to do this.
First, take a full database backup. I repeat: Take a full backup of your database. This shouldn't cause any problems, but you don't want to be doing stuff like this without a fresh backup.
Next, you'll need to find the file id for the log file for the database in question. You can do that like this:
select d.name, mf.file_id
from sys.databases d
join sys.master_files mf
on d.database_id = mf.database_id
where mf.type = 1
and d.name = 'SonarQube'
The mf.type = 1 returns only transaction log files. Use the name of your database if it's not SonarQube.
--Switch to the SonarQube database. If you're not in the right context, you'll shrink the wrong file.
use [SonarQube];
--Do a checkpoint if you're on Simple recovery
checkpoint;
--Do a log backup if you're on Full recovery
backup log ....
--Shrink logfile to 1 GB. Replace #file_id with the id you got above.
dbcc shrinkfile ( #file_id, 1024 );
The size there is the size in MB. You'll have to make a guess based on how large your DB is and how many changes you make to your system. Something between 50% the size of the DB and 100% the size of the DB is pretty safe for most systems. In any case, I would not shrink the logs further than 1 GB. You'll need to monitor the log space and whether or not the log continues to grow. The Disk Usage report in SSMS is a good way to do that.
The first time you run this, you may not see much gain in disk space at all. It might go to 66.0 GB or 62 GB. The reason for that is because the current tail of the log file is still at the end of the transaction log file. What you should do is, if the system is under activity, wait a few hours and then run the above again. That will give the system time to cycle the log back to the beginning of the log file, and you'll be able to shrink it down. Here is a good article covering more about how shrinking the log file actually works, if you're interested.

Related

Sql Server Setting max file size lead to faile transactions

I have a database with log file size of 700 MB. Now I have fixed its max file size to 1 GB.
When it reaches 1 GB,transaction failed the reason for the same is that "The transaction log for database is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases"
This is same case if I uncheck Autogrowth for log file.
When I checked log_reuse_wait_desc column in sys.databases it says "Active_Transaction".
I am not able to understand why Sql server is not maintaining the max file size limit.Why it cannot delete old logs or something like that to maintain the max file size.
How does it work.
What I want is to limit the log file size that not to exceed 1 GB in any case.
There are a few things you need to consider here, especially if you want to restrict the log file size to 1GB.
as already mentioned, you need to understand the difference
between the three recovery models. Taking log backups is a key task
when using the full recovery model. However this is only part of the
problem; log backups only truncate the inactive part of the log,
therefore a transaction could fill the log file with 1GB+ of data,
and then you are in the same position you are in now... even if you are in simple recovery model (a log backup will not help you here!).
In an ideal situation, you would not restrict the log file in such a way, because of this problem. if possible you want to allow it to auto-grow so, in theory, it could fill the disk.
Transaction log management is a science in itself. Kimberly Tripp has some very good advice on how to manage transaction log throughput here
understanding VLF's will allow you to better manage your transaction log, and could help towards better proportioning your log file for large transactions.
If, after everything you have read, you are still required to limit the transaction log growth, then you will need to consider batch updating large result sets. this will allow you to update, say, 1000 rows at a time, meaning that only 1000 records are written to the log. SQL Server uses write-ahead logging, so in order to complete a transaction, you first need to have enough space in the transaction log to write all the details. If using a simple recovery model, this write-ahead logging is automatically truncated, meaning you don't need to backup the log. Therefore, writing 1000 records at a time (for example) will causing less of a problem than a huge 1,000,000 record insert (say)
Redgate provide a free e-book to help you on your way!
EDIT:
P.s. I've just read your comment above... If you are in full recovery model you MUST do log backups, otherwise sql server WILL NOT recover the space from the log, and will continue to write to the log causing it to expand! However note, you MUST have a full backup for transaction log backups to take effect. SQL Server cannot backup the log if it doesn't have an initial restore point (i.e. the full backup).

SQL Server backup strategy questions for DB in Full recovery mode

I've recently taken on the database administration of a few SQL servers varying from SQL Server 2005 to 2014, where many of the DB's are in Full recovery mode, however no good ongoing backup maintenance plans were ever setup.
It seems to me that the previous DBA would only deal with transaction log files when they got out of control and fill up the hard drive. So i'd like to change this and fix the issue once and for all. I've been doing some reading and think I have a decent understanding of what need to be done, so i'd like to validate my understanding as well as ask a few question to clarify a few points that I still don't fully get.
So based on my understanding to date i would need to create a maintenance plan which starts with a Full Backup. I still need to talk to management to figure out things like RTO, acceptable data loss, etc so let's assume for this example that we'll do Full Backup's on Sunday.
Next I would add to this maintenance plan Differential backups every night... so Monday to Saturday. I realize that this could also be Full backup's or run the differentials more frequently, but again this is just as an example to make sure i'm understanding things correctly.
Now as for the transaction log backups. I get that i would need to back these up and truncate the log file to prevent it from continually growing and getting out of control. I don't know if there are any specific recommendation for how often to back this up, but i've seen 15 minutes suggested. I guess this would more so fall under the acceptable data loss window. Is that correct?
So the other thing that i've discovered is that when you backup the transaction log file with truncation, that if the log file has already grown out of control that it doesn't shrink the file. I've also read that it isn't good to shrink these files, at least on regular basis because once you shrink it, it would need to grow again and this will cause fragmentation and performance issues.
Now since I am currently in a situation that the files have already grown out of control, i'd assume that i should in fact shrink the log files this one time once i've but my maintenance in place. Is that assumption correct?
Also once i Shrink the transaction log file this one time, are there any maintenance task that i should be running to avoid performance issue due to shrinking the log files?
One other question that i was wondering about would be with respect to point in time recovery. So let's say I take a full backup at 5:00 AM and i also take a transaction log backup every 15 minutes. I get alerted that at 6:18 AM something has gone wrong (let's say a table was deleted). So i know i can restore by Full backup that happened at 5:00 AM and leave it in NO RECOVERY mode and restore all of the Transaction log backup from 5:15 AM to 6:15 AM, but here is what i'm interested in...since i have my DB in full recovery mode, is it possible to somehow use my existing transaction log file (not the backups) to roll forward all transaction between 6:15 and 6:17 just before the table was deleted? If so how would you do this? I guess this obviously wouldn't work in the case of you loosing the hard drive with your transaction log files, or your server exploding...but in a case like i've outlined is it do able?
Thanks
I would recommend doing a full backup after everyone stopped working, e. g. at 10 p.m. (if that is the case), not in the morning shortly before people start working. Just in order to give it enough time to run.
Personally, I prefer doing daily full backups instead of incremental backups if the database is not too big to save backups for, say, 14 days. I feel better to rely on less files. If database and full backups are too big, incremental backups might be the better choice.
As you said: How many transaction log backups you create during the day depends on the acceptable data loss window. In an environment > 5 people working on the system (just as a gut feeling) I would configure them to run all 15 minutes, on very big systems maybe even more.
After the first transaction log backup, you might want to shrink the LOG file ONCE.
I think it's not necessary to run any optimizations after a log shrink.
As far as I know it's not possible to restore transactions between 06:15 and 06:17.
When activating the transaction log backups, keep in mind that the first transaction log backup will be quite big (around the size of the current, large log). Ensure to have enough space on disk until you shrink the log file and delete the first transaction log (usually done automatically within the maintenance plan, e. g. after 14 days.).

Monitoring SQL Log File

After deploying a project on the client's machine, the sql db log file has grown to up to 450G, although the db size is less than 100MB, The logging mode is set to Simple mode, and the transactions are send from a windows service that send insertion and updating transaction every 30 seconds.
my question is, how to know the reason of db log file growth?
I would like to know how to monitor the log file to know what is the exact transaction that causes the problem.
should i debug the front end ? or there is away that expose the transactions that cause db log file growth.
Thank you.
Note that a simple recovery model does not allow for log backups since it keeps the least amount of information and relies on CHECKPOINT, so if this is a critical database, consider protecting the client by use of a FULL RECOVERY plan. Yes, you have to use more space, but disk space is cheap and you can have greater control over the point in time recovery and managing your log files. Trying to be concise:
A) Your database in Simple Mode will only truncate transactions in your transaction log as when a CHECKPOINT is created.
B) Unfortunately, large/lots of uncommitted transactions, including BACKUP, creation of SNAPSHOT, and LOG SCANs, among other things will stop your database from creating those checkpoints and your database will be left unprotected until those transactions are completed.
Your current system relies on having the right edition of your .bak file, which depending on the size may mean hours of potential loss.
In other words, it is that ridiculous size because your database is not able to create a CHECKPOINT to truncate these transactions often enough....
a little note on log files
Foremost, Log files are not automatically truncated every time a transaction is committed (otherwise, you would only have the last committed transaction to go back to). Taking frequent log backups will ensure pertinent changes are kept (point in time) and SHRINKFILE will squeeze the log file to the smallest size available/size specified.
Use DBCC SQLPERF(logspace) to see how much of your log file is in use and how large it is. Once you perform a full backup, the log file will be truncated to the remaining uncommitted/active transactions. (Do not confuse this with shrinking the size)
Some suggestions on researching your transactions:
You can use the system tables to see the most expensive cache, frequent, and active plans.
You can search the log file using an undocumented extended stored procedure, fn_dblog.
Pinal has great info on this topic that you can read at this webpage and link:
Beginning Reading Transaction Log
A Log File is text, and depending on your log levels and how many errors and messages you receive these files can grow very quickly.
You need to rotate your logs with something like logrotate although from your question it sounds like you're using Windows so not sure what the solution for that would be.
The basics of log rotation are taking daily/weekly versions of the logs, and compressing them with gzip or similar and trashing the uncompressed version.
As it is text with a lot of repetition this will make the files very very small in comparison, and should solve your storage issues.
log file space won't be reused ,if there is open transaction..You can verify the reason for log space reuse using below DMV..
select log_reuse_wait_desc,database_id from sys.databases
In your case,your database is set to simple and database is 100 MB..but the log has grown upto 450 GB..which is very huge..
My theory is that ,there may be some open transactions ,which prevented log space reuse..log file won't shrink back,once it grew..
As of know you can run above DMV and see ,what is preventing log space reuse at this point,you can't go back in time to know what prevented log space reuse

SQL Server 2008 log file size is large and growing quickly

Most of the time users will hit the database to read news. There are very few number of queries executed under transactions. 95% of the database hits would be for read-only purposes.
My database log files size is growing 1 GB per day. Even if I shrink the database, the log file size is not decreasing. What could be the reason for growing the log file size more and more? How can I control this? As per my knowledge log file does not increase when we read data from tables.
Any suggestions on how to deal with the log file growing? How can it be kept a manageable or reasonable size? Does this effect performance in any way?
There are couple of things to consider. What type of backups you do, and what type of backups you need. If you will have the answer to this question you can either switch Recovery Mode to simple or leave it full but then you need to make incremental backups on a daily basis (or whatever makes you happy with log size).
To set your database logging to simple (but only if you do Full Backups of your database!).
Right click on your database
Choose Properties
Choose Options
Set Recovery mode to simple
This will work and is best if your backup schedule is Full Backup every day. Because in such scenario your log won't be trimmed and it will skyrocket (just like in your case).
If you would be using Grandfather&Father&Son backup technique, which means Monthly Full backup, Weekly Full backup, and then every day incremental backup. Then for that you need Full Recovery Mode. If 1GB of log per day is still too much you can enable incremental backup per hour or per 15 minutes. This should fix the problem of log growing more and more.
If you run Full Backup every day you can switch it to simple recovery mode and you should be fine without risking your data (if you can leave with possible 1 day of data being lost). If you plan to use incremental then leave it at Recovery Mode Full.
Full backups will not help, you must regularly backup the transaction log (as well as the regular database full and differential backups) for it to be emptied. If you are not backing up the log and you are not in simple recovery mode, then your transaction log has every transaction in it since the database was set up. If you have enough action that you are growing by a gig a day, then you may also have large imports or updates affecting many records at once. It is possible you need to be in simple recovery mode where transactions are not recorded individually. Do NOT do that however if you have a mix of data from imports and users. In that case you need to back up the transaction log frequently to be able to keep the size manageable and to a point in time. We backup our transaction log every 15 minutes.
Read about transaction log backups in BOL to see how to fix the mess you have right now. Then get your backups set up and running properly. You need to read and understand this stuff thoroughly before attempting to fix. Right now, you would probably be in a world of hurt if your server failed and you had to recover the database. Transaction log backups are critical to being able to recover properly from a failure.
Do you backup your database frequently? You need to perform full- and/or transaction log- backups in order for SQL Server to consider shrinking your log file.
The major reason for the log file large size is due to bulk transaction in DB. To reduce the log file size best option to take the transaction log backup after certain interval of time.

How do you clear the SQL Server transaction log?

I'm not a SQL expert, and I'm reminded of the fact every time I need to do something beyond the basics. I have a test database that is not large in size, but the transaction log definitely is. How do I clear out the transaction log?
Making a log file smaller should really be reserved for scenarios where it encountered unexpected growth which you do not expect to happen again. If the log file will grow to the same size again, not very much is accomplished by shrinking it temporarily. Now, depending on the recovery goals of your database, these are the actions you should take.
First, take a full backup
Never make any changes to your database without ensuring you can restore it should something go wrong.
If you care about point-in-time recovery
(And by point-in-time recovery, I mean you care about being able to restore to anything other than a full or differential backup.)
Presumably your database is in FULL recovery mode. If not, then make sure it is:
ALTER DATABASE testdb SET RECOVERY FULL;
Even if you are taking regular full backups, the log file will grow and grow until you perform a log backup - this is for your protection, not to needlessly eat away at your disk space. You should be performing these log backups quite frequently, according to your recovery objectives. For example, if you have a business rule that states you can afford to lose no more than 15 minutes of data in the event of a disaster, you should have a job that backs up the log every 15 minutes. Here is a script that will generate timestamped file names based on the current time (but you can also do this with maintenance plans etc., just don't choose any of the shrink options in maintenance plans, they're awful).
DECLARE #path NVARCHAR(255) = N'\\backup_share\log\testdb_'
+ CONVERT(CHAR(8), GETDATE(), 112) + '_'
+ REPLACE(CONVERT(CHAR(8), GETDATE(), 108),':','')
+ '.trn';
BACKUP LOG foo TO DISK = #path WITH INIT, COMPRESSION;
Note that \\backup_share\ should be on a different machine that represents a different underlying storage device. Backing these up to the same machine (or to a different machine that uses the same underlying disks, or a different VM that's on the same physical host) does not really help you, since if the machine blows up, you've lost your database and its backups. Depending on your network infrastructure it may make more sense to backup locally and then transfer them to a different location behind the scenes; in either case, you want to get them off the primary database machine as quickly as possible.
Now, once you have regular log backups running, it should be reasonable to shrink the log file to something more reasonable than whatever it's blown up to now. This does not mean running SHRINKFILE over and over again until the log file is 1 MB - even if you are backing up the log frequently, it still needs to accommodate the sum of any concurrent transactions that can occur. Log file autogrow events are expensive, since SQL Server has to zero out the files (unlike data files when instant file initialization is enabled), and user transactions have to wait while this happens. You want to do this grow-shrink-grow-shrink routine as little as possible, and you certainly don't want to make your users pay for it.
Note that you may need to back up the log twice before a shrink is possible (thanks Robert).
So, you need to come up with a practical size for your log file. Nobody here can tell you what that is without knowing a lot more about your system, but if you've been frequently shrinking the log file and it has been growing again, a good watermark is probably 10-50% higher than the largest it's been. Let's say that comes to 200 MB, and you want any subsequent autogrowth events to be 50 MB, then you can adjust the log file size this way:
USE [master];
GO
ALTER DATABASE Test1
MODIFY FILE
(NAME = yourdb_log, SIZE = 200MB, FILEGROWTH = 50MB);
GO
Note that if the log file is currently > 200 MB, you may need to run this first:
USE yourdb;
GO
DBCC SHRINKFILE(yourdb_log, 200);
GO
If you don't care about point-in-time recovery
If this is a test database, and you don't care about point-in-time recovery, then you should make sure that your database is in SIMPLE recovery mode.
ALTER DATABASE testdb SET RECOVERY SIMPLE;
Putting the database in SIMPLE recovery mode will make sure that SQL Server re-uses portions of the log file (essentially phasing out inactive transactions) instead of growing to keep a record of all transactions (like FULL recovery does until you back up the log). CHECKPOINT events will help control the log and make sure that it doesn't need to grow unless you generate a lot of t-log activity between CHECKPOINTs.
Next, you should make absolute sure that this log growth was truly due to an abnormal event (say, an annual spring cleaning or rebuilding your biggest indexes), and not due to normal, everyday usage. If you shrink the log file to a ridiculously small size, and SQL Server just has to grow it again to accommodate your normal activity, what did you gain? Were you able to make use of that disk space you freed up only temporarily? If you need an immediate fix, then you can run the following:
USE yourdb;
GO
CHECKPOINT;
GO
CHECKPOINT; -- run twice to ensure file wrap-around
GO
DBCC SHRINKFILE(yourdb_log, 200); -- unit is set in MBs
GO
Otherwise, set an appropriate size and growth rate. As per the example in the point-in-time recovery case, you can use the same code and logic to determine what file size is appropriate and set reasonable autogrowth parameters.
Some things you don't want to do
Back up the log with TRUNCATE_ONLY option and then SHRINKFILE. For one, this TRUNCATE_ONLY option has been deprecated and is no longer available in current versions of SQL Server. Second, if you are in FULL recovery model, this will destroy your log chain and require a new, full backup.
Detach the database, delete the log file, and re-attach. I can't emphasize how dangerous this can be. Your database may not come back up, it may come up as suspect, you may have to revert to a backup (if you have one), etc. etc.
Use the "shrink database" option. DBCC SHRINKDATABASE and the maintenance plan option to do the same are bad ideas, especially if you really only need to resolve a log problem issue. Target the file you want to adjust and adjust it independently, using DBCC SHRINKFILE or ALTER DATABASE ... MODIFY FILE (examples above).
Shrink the log file to 1 MB. This looks tempting because, hey, SQL Server will let me do it in certain scenarios, and look at all the space it frees! Unless your database is read only (and it is, you should mark it as such using ALTER DATABASE), this will absolutely just lead to many unnecessary growth events, as the log has to accommodate current transactions regardless of the recovery model. What is the point of freeing up that space temporarily, just so SQL Server can take it back slowly and painfully?
Create a second log file. This will provide temporarily relief for the drive that has filled your disk, but this is like trying to fix a punctured lung with a band-aid. You should deal with the problematic log file directly instead of just adding another potential problem. Other than redirecting some transaction log activity to a different drive, a second log file really does nothing for you (unlike a second data file), since only one of the files can ever be used at a time. Paul Randal also explains why multiple log files can bite you later.
Be proactive
Instead of shrinking your log file to some small amount and letting it constantly autogrow at a small rate on its own, set it to some reasonably large size (one that will accommodate the sum of your largest set of concurrent transactions) and set a reasonable autogrow setting as a fallback, so that it doesn't have to grow multiple times to satisfy single transactions and so that it will be relatively rare for it to ever have to grow during normal business operations.
The worst possible settings here are 1 MB growth or 10% growth. Funny enough, these are the defaults for SQL Server (which I've complained about and asked for changes to no avail) - 1 MB for data files, and 10% for log files. The former is much too small in this day and age, and the latter leads to longer and longer events every time (say, your log file is 500 MB, first growth is 50 MB, next growth is 55 MB, next growth is 60.5 MB, etc. etc. - and on slow I/O, believe me, you will really notice this curve).
Further reading
Please don't stop here; while much of the advice you see out there about shrinking log files is inherently bad and even potentially disastrous, there are some people who care more about data integrity than freeing up disk space.
A blog post I wrote in 2009, when I saw a few "here's how to shrink the log file" posts spring up.
A blog post Brent Ozar wrote four years ago, pointing to multiple resources, in response to a SQL Server Magazine article that should not have been published.
A blog post by Paul Randal explaining why t-log maintenance is important and why you shouldn't shrink your data files, either.
Mike Walsh has a great answer covering some of these aspects too, including reasons why you might not be able to shrink your log file immediately.
-- DON'T FORGET TO BACKUP THE DB :D (Check [here][1])
USE AdventureWorks2008R2;
GO
-- Truncate the log by changing the database recovery model to SIMPLE.
ALTER DATABASE AdventureWorks2008R2
SET RECOVERY SIMPLE;
GO
-- Shrink the truncated log file to 1 MB.
DBCC SHRINKFILE (AdventureWorks2008R2_Log, 1);
GO
-- Reset the database recovery model.
ALTER DATABASE AdventureWorks2008R2
SET RECOVERY FULL;
GO
From: DBCC SHRINKFILE (Transact-SQL)
You may want to backup first.
DISCLAIMER: Please read comments below this answer carefully before attempting it, and be sure to check the accepted answer. As I said nearly 5 years ago:
if anyone has any comments to add for situations when this is NOT an
adequate or optimal solution then please comment below
Turns out there were :-)
Original Answer:
Right click on the database name.
Select Tasks → Shrink → Database
Then click OK!
I usually open the Windows Explorer directory containing the database files, so I can immediately see the effect.
I was actually quite surprised this worked! Normally I've used DBCC before, but I just tried that and it didn't shrink anything, so I tried the GUI (2005) and it worked great - freeing up 17 GB in 10 seconds
In Full recovery mode this might not work, so you have to either back up the log first, or change to Simple recovery, then shrink the file. [thanks #onupdatecascade for this]
--
PS: I appreciate what some have commented regarding the dangers of this, but in my environment I didn't have any issues doing this myself especially since I always do a full backup first. So please take into consideration what your environment is, and how this affects your backup strategy and job security before continuing. All I was doing was pointing people to a feature provided by Microsoft!
Below is a script to shrink the transaction log, but I’d definitely recommend backing up the transaction log before shrinking it.
If you just shrink the file you are going to lose a ton of data that may come as a life saver in case of disaster. The transaction log contains a lot of useful data that can be read using a third-party transaction log reader (it can be read manually but with extreme effort though).
The transaction log is also a must when it comes to point in time recovery, so don’t just throw it away, but make sure you back it up beforehand.
Here are several posts where people used data stored in the transaction log to accomplish recovery:
How to view transaction logs in SQL Server 2008
Read the log file (*.LDF) in SQL Server 2008
USE DATABASE_NAME;
GO
ALTER DATABASE DATABASE_NAME
SET RECOVERY SIMPLE;
GO
--First parameter is log file name and second is size in MB
DBCC SHRINKFILE (DATABASE_NAME_Log, 1);
ALTER DATABASE DATABASE_NAME
SET RECOVERY FULL;
GO
You may get an error that looks like this when the executing commands above
“Cannot shrink log file (log file name) because the logical
log file located at the end of the file is in use“
This means that TLOG is in use. In this case try executing this several times in a row or find a way to reduce database activities.
Here is a simple and very inelegant & potentially dangerous way.
Backup DB
Detach DB
Rename Log file
Attach DB
New log file will be recreated
Delete Renamed Log file.
I'm guessing that you are not doing log backups. (Which truncate the log). My advice is to change recovery model from full to simple. This will prevent log bloat.
If you do not use the transaction logs for restores (i.e. You only ever do full backups), you can set Recovery Mode to "Simple", and the transaction log will very shortly shrink and never fill up again.
If you are using SQL 7 or 2000, you can enable "truncate log on checkpoint" in the database options tab. This has the same effect.
This is not recomended in production environments obviously, since you will not be able to restore to a point in time.
This technique that John recommends is not recommended as there is no guarantee that the database will attach without the log file. Change the database from full to simple, force a checkpoint and wait a few minutes. The SQL Server will clear the log, which you can then shrink using DBCC SHRINKFILE.
Most answers here so far are assuming you do not actually need the Transaction Log file, however if your database is using the FULL recovery model, and you want to keep your backups in case you need to restore the database, then do not truncate or delete the log file the way many of these answers suggest.
Eliminating the log file (through truncating it, discarding it, erasing it, etc) will break your backup chain, and will prevent you from restoring to any point in time since your last full, differential, or transaction log backup, until the next full or differential backup is made.
From the Microsoft article onBACKUP
We recommend that you never use NO_LOG or TRUNCATE_ONLY to manually
truncate the transaction log, because this breaks the log chain. Until
the next full or differential database backup, the database is not
protected from media failure. Use manual log truncation in only very
special circumstances, and create backups of the data immediately.
To avoid that, backup your log file to disk before shrinking it. The syntax would look something like this:
BACKUP LOG MyDatabaseName
TO DISK='C:\DatabaseBackups\MyDatabaseName_backup_2013_01_31_095212_8797154.trn'
DBCC SHRINKFILE (N'MyDatabaseName_Log', 200)
The SQL Server transaction log needs to be properly maintained in order to prevent its unwanted growth. This means running transaction log backups often enough. By not doing that, you risk the transaction log to become full and start to grow.
Besides the answers for this question I recommend reading and understanding the transaction log common myths. These readings may help understanding the transaction log and deciding what techniques to use to "clear" it:
From 10 most important SQL Server transaction log myths:
Myth: My SQL Server is too busy. I don’t want to make SQL Server transaction log backups
One of the biggest performance intensive operations in SQL Server is an auto-grow event of the online transaction log file. By not making transaction log backups often enough, the online transaction log will become full and will have to grow. The default growth size is 10%. The busier the database is, the quicker the online transaction log will grow if transaction log backups are not created
Creating a SQL Server transaction log backup doesn’t block the online transaction log, but an auto-growth event does. It can block all activity in the online transaction log
From Transaction log myths:
Myth: Regular log shrinking is a good maintenance practice
FALSE. Log growth is very expensive because the new chunk must be zeroed-out. All write activity stops on that database until zeroing is finished, and if your disk write is slow or autogrowth size is big, that pause can be huge and users will notice. That’s one reason why you want to avoid growth. If you shrink the log, it will grow again and you are just wasting disk operation on needless shrink-and-grow-again game
Use the DBCC ShrinkFile ({logicalLogName}, TRUNCATEONLY) command. If this is a test database and you are trying to save/reclaim space, this will help.
Remember though that TX logs do have a sort of minimum/steady state size that they will grow up to. Depending upon your recovery model you may not be able to shrink the log - if in FULL and you aren't issuing TX log backups the log can't be shrunk - it will grow forever. If you don't need TX log backups, switch your recovery model to Simple.
And remember, never ever under any circumstances delete the log (LDF) file! You will pretty much have instant database corruption. Cooked! Done! Lost data! If left "unrepaired" the main MDF file could become corrupt permanently.
Never ever delete the transaction log - you will lose data! Part of your data is in the TX Log (regardless of recovery model)... if you detach and "rename" the TX log file that effectively deletes part of your database.
For those that have deleted the TX Log you may want to run a few checkdb commands and fix the corruption before you lose more data.
Check out Paul Randal's blog posts on this very topic, bad advice.
Also in general do not use shrinkfile on the MDF files as it can severely fragment your data. Check out his Bad Advice section for more info ("Why you should not shrink your data files")
Check out Paul's website - he covers these very questions. Last month he walked through many of these issues in his Myth A Day series.
To Truncate the log file:
Backup the database
Detach the database, either by using Enterprise Manager or by executing : Sp_DetachDB [DBName]
Delete the transaction log file. (or rename the file, just in case)
Re-attach the database again using: Sp_AttachDB [DBName]
When the database is attached, a new transaction log file is created.
To Shrink the log file:
Backup log [DBName] with No_Log
Shrink the database by either:
Using Enterprise manager :-
Right click on the database, All tasks, Shrink database, Files, Select log file, OK.
Using T-SQL :-
Dbcc Shrinkfile ([Log_Logical_Name])
You can find the logical name of the log file by running sp_helpdb or by looking in the properties of the database in Enterprise Manager.
First check the database recovery model. By default, SQL Server Express Edition creates a database for the simple recovery
model (if I am not mistaken).
Backup log DatabaseName With Truncate_Only:
DBCC ShrinkFile(yourLogical_LogFileName, 50)
SP_helpfile will give you the logical log file name.
Refer to:
Recover from a full transaction log in a SQL Server database
If your database is in Full Recovery Model and if you are not taking TL backup, then change it to SIMPLE.
It happened with me where the database log file was of 28 GBs.
What can you do to reduce this?
Actually, log files are those file data which the SQL server keeps when an transaction has taken place. For a transaction to process SQL server allocates pages for the same. But after the completion of the transaction, these are not released suddenly hoping that there may be a transaction coming like the same one. This holds up the space.
Step 1:
First Run this command in the database query explored
checkpoint
Step 2:
Right click on the database
Task> Back up
Select back up type as Transaction Log
Add a destination address and file name to keep the backup data (.bak)
Repeat this step again and at this time give another file name
Step 3:
Now go to the database
Right-click on the database
Tasks> Shrinks> Files
Choose File type as Log
Shrink action as release unused space
Step 4:
Check your log file
normally in SQL 2014 this can be found at
C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQL2014EXPRESS\MSSQL\DATA
In my case, its reduced from 28 GB to 1 MB
To my experience on most SQL Servers there is no backup of the transaction log.
Full backups or differential backups are common practice, but transaction log backups are really seldom.
So the transaction log file grows forever (until the disk is full).
In this case the recovery model should be set to "simple".
Don't forget to modify the system databases "model" and "tempdb", too.
A backup of the database "tempdb" makes no sense, so the recovery model of this db should always be "simple".
Take a backup of the MDB file.
Stop SQL services
Rename the log file
Start the service
(The system will create a new log file.)
Delete or move the renamed log file.
Database → right click Properties → file → add another log file with a different name and set the path the same as the old log file with a different file name.
The database automatically picks up the newly created log file.
Try this:
USE DatabaseName
GO
DBCC SHRINKFILE( TransactionLogName, 1)
BACKUP LOG DatabaseName WITH TRUNCATE_ONLY
DBCC SHRINKFILE( TransactionLogName, 1)
GO
Slightly updated answer, for MSSQL 2017, and using the SQL server management studio.
I went mostly from these instructions https://www.sqlshack.com/sql-server-transaction-log-backup-truncate-and-shrink-operations/
I had a recent db backup, so I backed up the transaction log. Then I backed it up again for good measure.
Finally I shrank the log file, and went from 20G to 7MB, much more in line with the size of my data.
I don't think the transaction logs had ever been backed up since this was installed 2 years ago.. so putting that task on the housekeeping calendar.
Backup DB
Detach DB
Rename Log file
Attach DB (while attaching remove renamed .ldf (log file).Select it and remove by pressing Remove button)
New log file will be recreated
Delete Renamed Log file.
This will work but it is suggested to take backup of your database first.
Some of the other answers did not work for me: It was not possible to create the checkpoint while the db was online, because the transaction log was full (how ironic). However, after setting the database to emergency mode, I was able to shrink the log file:
alter database <database_name> set emergency;
use <database_name>;
checkpoint;
checkpoint;
alter database <database_name> set online;
dbcc shrinkfile(<database_name>_log, 200);
DB Transaction Log Shrink to min size:
Backup: Transaction log
Shrink files: Transaction log
Backup: Transaction log
Shrink files: Transaction log
I made tests on several number of DBs: this sequence works.
It usually shrinks to 2MB.
OR by a script:
DECLARE #DB_Name nvarchar(255);
DECLARE #DB_LogFileName nvarchar(255);
SET #DB_Name = '<Database Name>'; --Input Variable
SET #DB_LogFileName = '<LogFileEntryName>'; --Input Variable
EXEC
(
'USE ['+#DB_Name+']; '+
'BACKUP LOG ['+#DB_Name+'] WITH TRUNCATE_ONLY ' +
'DBCC SHRINKFILE( '''+#DB_LogFileName+''', 2) ' +
'BACKUP LOG ['+#DB_Name+'] WITH TRUNCATE_ONLY ' +
'DBCC SHRINKFILE( '''+#DB_LogFileName+''', 2)'
)
GO

Resources