I have SQL database and I have sent the same database to other place for further development (there may be or may not changes in Schema). I created a a backup file and restored it on another SQL Server.
I found the log file size is very huge (14GB) even though I only have 4 tables with 1000 rows and will not grow two much. Currently I run query and found (Size is in MB).
data_size data_used_size log_size log_used_size
801.00 2.06 14220.75 55.63
What I did:
I have shrunk the log files using SQL Management Studio and also kept SIMPLE mode recovery, as we have only few updates to this database that can be done again if transaction fails at any point. I created a backup and restored same and found size of log file has decreased considerably and here is here under.
total_size data_size data_used_size log_size log_used_size
802.00 801.00 2.06 1.00 0.46
Question 1: Since the database size very less should we decrease the initial size of database
Question 2: Is this ok now to send this .bak file for restoring database at another location
Answer to Q1:
It is always a good idea to estimate your data growth and set an initial size for your database. The reason is simply to avoid the SQL Data file from performing Auto Growth operations which are very expensive. If you are not expecting any data growth, then whether or not you set Initial Size, it does not matter.
Answer to Q2:
You can send the Backup file to any location as long as your SQL Server version on which you restore the file is of similar or higher version. Only point to note is the data in the backup file - consider encryption if you have sensitive data.
Related
I am a newbie in SQL Server, I have a task to move the whole SQL Server to another.
I am trying to estimate how much space I need in the new SQL Server.
I ran EXEC sp_spaceused
And the following came up:
When I look into the output, it seems that the Database is using ~122GB (reserved), but when looking in the total database size (mdf + ldf) it is ~1.8 TB.
Does that mean when I copy the Database from the existing SQL Server to a new one I will need ~1.8 TBs into the new?
I am thinking about creating a back-up and copy the back-up to the new Server. How does the back-up takes into consideration the unallocated space? Does the back gets closer to the reserved or the database_size? I understand that this is without taking into consideration the uncompressed in the back-up, which will improve the file size.
Thx for the help.
The backup file will be much smaller than 1.8TB, since unallocated pages are not backed up. But the log and data files themselves will be restored to an identical size, so you will need 1.8TB on the target in order to restore the database in its current state.
Did you check to see if your log file is large due to some uncontrolled growth that happened at some point and is maybe no longer necessary? If this is where all your size is, it's quite possible you can fix this before the move. Make sure you have a full backup and take at least one log backup, then use DBCC SHRINKFILE to bring the log file back into the right stratosphere, especially if it was caused by either a one-time abnormal event or a prolonged log backup neglect that has now been addressed.
I really don't recommend copying/moving the mdf/ldf files (background) or using the SSMS UI to "do a copy" since you can have much greater control over what you're actually doing by using proper BACKUP DATABASE and RESTORE DATABASE commands.
How do I verify how much of the log data is being used?
If you're taking regular log backups (or are in simple recovery), it should usually be a very small % of the file. DBCC SQLPERF(LogSpace); will tell you % in use for all log files.
To minimize the size that the log will require in the backup file itself, then:
if full recovery, back up the log first.
if simple recovery, run a couple of CHECKPOINT; commands first.
Our production server has a large .ldf file (300 gb) associated with a .mdf file. On our DEV server, we restore the DEV database using the production backup from time to time. As our DEV server should not have many transactions associated with it, but the .ldf file is 300 gb also. I shrank the .ldf file to 50 gigs in size. Will the DEV .ldf file grow again after a restore from a production backup?
When you restore a database backup, the size of the database and log files will be whatever file size it was from the original database. So if the file size in prod was 300 GB, it will be 300 GB when you restore it to another database/server. It's common to do two things after restoring a prod database backup to a dev environment - shrink the log file size, and set recovery model to SIMPLE (whether you do either will depend on your needs/requirements).
Since you reduced the file size to 50 GB in size, it can grow to exceed 50 GB for many reasons, but most notably:
You create a transaction that generates an enormous amount of log
You have FULL recovery model, and you generate a lot of log, and never do log backups
You enable some kind of replication feature (replication, CDC, etc.) and never run logreader agent.
Will the DEV .ldf file grow again after a restore from a production
backup?
Yes, in case if your production database log file still has such size - 300 GB your database on dev will again have such log file.
We do not know what is the baseline of your live database.
However, if your production database has no regular heavy DML operations that result in long-running transactions, consider reducing the size of the log file on production to smaller size, for instance to 50 GB.
Also, this reduction will significantly reduce restore time, because LDF files are to be zeroed internally before RESTORE starts writing actual data. It means that SQL Server firstly has to create 300 GB file and write zeroes in it. In contrast with log files, data files can benefit from "instant file initialization" and such zeroing can be skipped if SQL Server instance service account correctly configured to have enough permissions.
Otherwise, every time when you do a restore to a dev environment such maintenance task to be done:
USE yourDev
ALTER DATABASE yourDev SET RECOVERY SIMPLE
-- assumption: only one ldf in db
DBCC SHRINKFILE(2, 1024)
You probably have either a background task that is running and running or a lot of open (long running) transactions that are preventing the log being backed up and shrunk. I bet that the same will happen on your new machine. You need to figure out what is preventing the transaction log from shrinking. You should not have to shrink it yourself.
How to reduce log file size of MS SQL database. The requirement is to reduce log file of the version control server's DB, Shrinking is not an option in our case.
How to reduce log file size of MS SQL database
Shrinking is the only option here, but you need to clear out the log by backing it up first. Otherwise, there won't be any free space to shrink. After you shrink it once, you can keep it from getting blown out by either
Backing up your transaction logs more regularly. How often depends on your RPO
Setting the database to Simple Mode. This is only suggested if your RPO is large enough to where your FULL BACKUPs and DIFFERENTIALS would cover it.
I'm trying to run a restore query from a bak file stored in S3 bucket to an RDS SQL Server Web edition, and kept getting this error:
[2017-09-13 20:30:22.227] Aborted the task because of a task failure or a concurrent RESTORE_DB request. [2017-09-13 20:30:22.287] There is not enough space on the disk to perform restore database operaton.
The bak file is 77 GB and the DB has 2TB, how come this is still not enough?
This is the query from AWS docs:
exec msdb.dbo.rds_restore_database
#restore_db_name='database_name',
#s3_arn_to_restore_from='arn:aws:s3:::bucket_name/file_name_and_extension';
Source: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html#SQLServer.Procedural.Importing.Native.Using
There is not enough space on the disk to perform restore database
operaton.
...
The bak file is 77 GB and the DB has 2TB, how come this is still not
enough?
You need 2Tb of space to be able to restore this backup.
The fact is that restore operation will reconstruct your original database that is 2Tb.
Backup is backing up only data, not empty space. If your backup is only 77Gb and is not compressed this means that your original database has only 77Gb of data (or even less, because backup contains a certain amout of log as well).
Any database consists of data file(s) and log file(s), and if your db is about 2Tb with only 77Gb of data, it means it has enormous log file. I think it's in full recovery model and someone does not take regular log backups (or even did not take any log backup at all!!)
So you should take a look at your original db, change recovery model to simple if you don't need point in time recovery and take no log backups, or, if you really need full recovery model, you should backup log more frequently.
Taking regular log backups or changing recovery model to simple will permit you to shrink the log to reasonable size, from that moment you will not need 2Gb of space to restore it anymore
I have a small SQL Server 2005 database. I take daily backup (automated) and the size of the .bak file typically comes out to be 400MB, growing by 5MB every day (which is inline with its usages).
Last night the size of the backup file jumped to 1GB. Suspecting that someone was trying to fill the database with garbage data, I ran a report (Reports -> Standard Reports -> Disk User By Top Tables) and the total size came out to be around 400MB.
Then thinking maybe something was wrong with the automated backup process, I immediately took backup again but the .bak file came out to be over 1GB. Before this automated backup yesterday, an automated task that defragments indexes also ran. However, all these months, a backup after this index optimization used to actually reduce the size of the .bak file.
I am trying to find an explanation for this big jump in size overnight and also why the .bak file is more than double the size of the the actual database disk usages?
UPDATE: I ran
DBCC SHRINKDATABASE(mydb)
to remove transaction logs. Then took a backup again. The size of the .bak file came out even bigger than last time.
This is the query I ran:
DBCC SHRINKFILE(mydb_log, 1)
BACKUP LOG mydbWITH TRUNCATE_ONLY
DBCC SHRINKFILE(mydb_log, 1)
1 - How are you backing up the data? Maintenance plans?
2 - If you are appending to the same database backup file, the backup will grow!
Check out the contents of the file.
RESTORE FILELISTONLY FROM AdventureWorksBackups WITH FILE=1;
http://technet.microsoft.com/en-us/library/ms173778.aspx
This assumes you have a dump device named AdventureWorksBackups. You can also change it to a DISK='AdventureWorks.bak'.
3 - Also, the maintenance plans do not do a good job with determining when to re-organize/update stats versus rebuild an index.
4 - Check out the ola hallengren scripts. They are way better!
http://ola.hallengren.com/
First, they create a directory structure for you.
c:\backup\<server name>\<database name>\full
c:\backup\<server name>\<database name>\diff
c:\backup\<server name>\<database name>\log
Second, each backup has a date time stamp. No appending to backup files.
Third, they clean up after themselves by passing the number of hours to keep on-line.
Fourth, they handle index fragmentation better - 5-30 = re-organize or 30+ = rebuild.
I usually set up the following for my databases.
1 - system databases - full backup every night, log backups hourly.
2 - user databases - full backup 1 x week, diff backup x 6 days, log backups hourly
Last but not least, SQL Server 2005 does not have native support for compressed backups. This does not mean you can not run a batch file to zip them up afterwards.
Third party tools like QUEST (DELL) and RED GATE used to support their own backup utility. The main reason was to fill this gap. Since SQL Server 2008, compressed backups were available. I think many of the vendors are getting rid of this utility since it is now standard.