We need to implement a strategy for improving our SQL Server database performance and I am investigating the topic of Data (mdf) and Transaction Log (ldf) files. There is quite a lot of information if I google but some slight differences in guidance. Can I ask the following clarification questions
If I have both data files and log files on same drive
a) Performance is affected due to the different write president on log files over data
files?
b) If drive is totally corrupted then everything since last
backup is lost completely?
If I have data file and log file on different drives
a) Does this have to be just different drives in virtual server, a different physical
drive on the same host server that is hosting numerous virtual servers or in
a completely different host server machine?
b) If data file drive becomes corrupted then database becomes Read-only and transactions
from time of last backup and time of corruption can be restored from
log file using a Tail-Log Backup?
c) If logical file drive becomes corrupted then database becomes Read-only and transactions
since last backup are lost completely?
Related
For huge tables I am thinking of using the concept of partitioning using file groups and have each file group on a separate disk. My question is that -
If one of the file group disk crashes, then should this incident be treated as database crash? Will it cause the database to stop working?
Will the restore operation (assuming full backup was taken) automatically create the file groups as configured before the crash?
If one of the file group disk crashes, then should this incident be
treated as database crash? Will it cause the database to stop working?
If all your data from that disk is in memory, you'll not even notice this crash.
Until a checkpoint attempts to write on that disk, or you need to read from it a new portion of data(if it's not in memory) you'll be able to work without any error.
Will the restore operation (assuming full backup was taken)
automatically create the file groups as configured before the crash?
Your question is not clear.
You can restore certain filegroups from full backup, but if these filegroups are not readonly you'll not be able to reconcile them with the rest of your database. It will be
only possible if your db is in full recovery model and you take and restore tail of the log backup after your full backup (and may be other log backups in between).
Here you can read more in details Piecemeal Restores (SQL Server)
Background:
I have a medium-sized database (900GB) that needs to be copied onto another server (driven via code, not scheduled). Currently we take a backup (to .bak), move it to a staging server, and restore it to the target server. The target server does not have enough space to hold the backup file, and the restored instance simultaneously, thus the staging server. These transfers (backup to staging, restore from staging) happen over SMB2. The staging server needs to go away due to business requirements, however. It is worth mentioning the target server will be taken offline (and used offline) after the transfer, so I'm not sure the mirroring or replication options are valid.
I have identified two options -- one is to backup the database to the primary server, and open up firewall rules/smb to serve the backup file to the target server over SMB. ("RESTORE FROM \x.x.x.x\blah\db.bak"). Security isn't a fan, though.
The ideal solution (and one that could easily be implemented in every other database I've worked with), is to quiesce the database and transfer the datafiles (in the case of ms-sql, mdf and ldf files). However, upon research I see there is no such functionality available out of the box. I know I can take the database offline to copy the mdf/ldf safely, but that's not an acceptable solution (database must remain online).
I have read LOTS of posts and Microsoft documentation regarding VSS / shadow copy, but I have also read lots of conflicting information about the reliability of using VSS/sqlwriter to copy the mdf/ldf file to the target server, and simply re-attaching the database.
I am looking for documentation or advice (or even backup software that can be programmatically driven via an API) to accomplish this goal of transferring the database without requiring a secondary holding place. Currently I'm researching how to drive this copying process with Powershell, using VSS(vssadmin/vshadow from sdk), but I'm not confident in what I'm reading, and it's not even clear to me if VSS/sqlwriter is a supportable method to copying online LDF/MDF files. Any advice is appreciated.
Thanks,
I have ran into an error today in regards to unavailable space on the drive where my SQL tables and logs are stored which has caused me to be unable to update any of the databases.
I have looked at the database server and deleted a database of approximatedly 1.5GB to allow me to continue. On looking on the server drive I can see backups for the database that I have deleted in the location:
E:\Program Files\Microsoft SQL Server\MCSSQL12.SQL2014\MSSQL\Backup
Inside this folder there are 5 backup copies for the last 5 days which I would like to delete to clear up the space. However when I delete the .BAK files from this folder it does not reallocate the free space. Am I missing a step here somewhere?
Quick wins:
If this is on a SAN, get the drive expanded by 20% just to give you some breathing room while you sort everything else.
Dropping databases does not delete their backups. You have to delete those too (and make sure they don't just land in the Recycle Bin). But not until you've verified that you either A) really don't need them anymore or B) have copies of those backups somewhere more durable.
Get your backups moved to a different drive - both now and for your scheduled backups. Your backups should not be on the same drive as your data. File this under "putting all of your eggs into one basket". Consider this: If your E drive fails, it's going to take both your data files and your backups with it. What do you have left?
Review your backup retention. Do you really need all 5 of those backups? On my instances, we do daily Full backups and transaction log backups every 15 minutes. We keep at most 2 Fulls on local storage - everything else is on tape. And once you have a Full, the transaction logs between that Full and the previous one aren't really needed unless you need to a Point In Time Restore to somewhere in between them. All managed by Agent jobs executing Ola Hallengren's backup scripts so we're hands-off and just monitoring.
Are you using backup compression? If not, this may help you get more backups on your volume if you can't change your retention period or backup location (but really, put your backups on another volume, in the interest of safety).
Less-quick wins:
Consider moving your logs to a separate drive. Transaction logs and data files have different I/O profiles/requirements, so by moving them to a different drive you can tune each appropriately - for both performance and space requirements.
Review your indexes and eliminate any that you don't really need (or consolidate them). Indexes take up space! Dropping unused indexes won't give free space back to the volume, but it will free space in your MDF files that can be used for other data instead of growing right away.
Do not shrink your database unless absolutely necessary and you're confident that it won't re-grow significantly in the very near future.
What are your data and log file growth increments? If you have free space on the drives where the log and data files live, and your are allowing for automatic file growth, sql server will grow the files when space is needed.
You may also need to review the information here MSDN regarding dbcc command shrinkfile.
According to this article https://msdn.microsoft.com/en-us/library/bb402876.aspx
having the log file and data file on separate drives will improve performance as I/O activity occurs at the same time for both the data and log files. Based on this information I have moved the non system databases.
Accordingly to this article http://www.sqlservercentral.com/Forums/Topic1263294-1550-1.aspx the only system log file that should be moved to a separate drive is the tempDB, the rest won't make a difference in performance as master and model as they are hardly updated.
I want to know from the Stackoverflow community, will it be beneficial to move the System Database log files on another storage drive to improve performance on SQL Server 2012 (master, model, msdb)?
I have a database that has lost one of its .ndf files and have been unable to get at the data. The .ndf file in question was added last Thursday and placed in a temporary storage location by a colleague (d'oh!). There is no backup available from this database since prior to this .ndf being created. I have seen numerous solutions to similar problems when the .ndf in question is its own filegroup, but in this case it actually is in a filegroup with an additional file which I want to try and get data from. I am pretty sure what I am trying to do is not possible but there is always a chance right?
The database setup
PRIMARY: Data.mdf -200mb
Data Filegroup 1: Data_1.ndf - 2.9gb
Data_2.ndf - 64gb (newly added file that is now lost - I believe it is just preallocated space)
LOG: Log.ldf - 128mb
When we logged into the VM this morning (hosted in Azure), we were presented with an unexpected shutdown notification from Windows (it seems there was a powerloss/shutdown at 1am) and our application was not reaching the database. Looking in SQL Server Management Studio I could see that it was Recovery Pending status. Trying to bring the db online lead me to an error about Data_2.ndf not being found (located at D:\SQL\Data\Data_2.ndf).
When I accessed the D drive (temporary storage drive) I was presented with a wonderful blank Windows Explorer window - completely blank drive.
I was able to set the Data_2.ndf file offline and bring the database itself online, however I am not able to query any of the data (as all tables were in Data Filegroup 1) due to the filegroup being offline. The other 3 files (mdf, ndf, ldf) are all online.
Is there any way out of this? Any way to perhaps recover any remaining data from Data_1.ndf or is it completely toast?
(This was a hastily stood up development server and there was no backup/recovery strategy for it, as "Azure never crashes" :)).
(Edit:formatting)
You are hosed. Its a miracle you can bring up your database. Are you sure you can retrieve data - have you tried doing selects? You probably will receive more extensive answers on the Database Administrators group.