I have a Azure VM with the following:
Windows DataCenter 2019
SQL Server 2017 Developer
Virtual drive of 6TB (built up of 12 512GB Premium SSD disks)
112 GB of RAM
16 VCPUS
I have a db that has a data file of approx ~5TB (2TB empty) and log file of approx ~1TB (99% empty).
I have backed this up to Azure blob storage (64 block blobs).
When I restore to my SQL Server with Instant File Initialisation enabled, it takes ~ 40 hours.
I can see the network and disk Throughput really low.
When I disable Instant File Initialisation, it takes ~3hours to zero out the files and then I get good performance on the restore ~1 1/2 hours (on top of the ~3 hours to zero out the files)
Does anyone know why this could be.
My code to restore
restore database [<db_name>] from
url = 'https://.....url_1.bak',
...
url = 'https://.....url_64.bak',
move 'db_log' to 'new log location' -- i am only moving the log file, as the data file's location doesn't change
stats = 1, norecovery;
Related
In my database I have 2 nodes RAC 10g database ,
I want to reclaim some space from flash recovery (ASMFLASH) to my production database (ASMPROD)
I tried the following command :
ALTER SYSTEM SET DB_RECOVERY_FILE_DEST_SIZE = 50G SCOPE=BOTH SID='*';
ASMFLASH size before 65 GB and after execute this command the size changed to 50GB
but still the released space not appear on my production database
How to show and use these 15GB in my production database why its not show ?
Sqlserver.exe showing memory greater than max memory limit lock pages also enabled its confusing
AS stated here
SQL Servers max memory setting defines the limits for buffer pool usage only. There will be variable but significant allocations required over and above that limit.
Jonathan Kehayias's, Christian Bolton and John Samson have level 300/400 posts on the topic. Brent Ozar has an easier to read article that might be a better place to start.
Also related: SQL Server 2008 R2 “Ghost Memory”
Min & Max Server Memory
Microsoft SQL Server Management Studio → Right Click the Server → Properties →Memory → Server Memory Options → Minimum server memory (In MB): = 0 and Maximum server memory (In MB): = 2147483647
Configure this memory allocation Based on the RAM installed in the DB Server.
For Ex:
IF DB server is installed with 6 GB of RAM, then maintain the 20% breadth space for the OS installed in the server.
For 6 GB of RAM, Maximum server memory (In MB) will be = 4915 for the SQL server.
Right Click the Server → Properties →Security → Login Auditing → Enable the Failed logins only. This option will avoid the log write and memory log space will be saved.
I have to recover a dropped table from a SQL Server database which runs on SQL Server 2016 Standard edition. Database is in full recovery mode.
After the dropped the table, I took a full database backup and then transaction log backup twice. Now can I recover my dropped table with or without using expensive third party tools?
I tried this link and getting error in the last command.
Result of the STOPBEFOREMARK query:
Processed 2104752 pages for database 'databasecopy', file 'databasefilename' on file 1.
Processed 6 pages for database 'datebasecopy', file 'database_log' on file 1.
RESTORE DATABASE successfully processed 2104758 pages in 123.259 seconds (133.405 MB/sec).
Msg 4335, Level 16, State 2, Line 15
The specified STOPAT time is too early. All or part of the database is already rolled forward beyond that point.
Msg 3013, Level 16, State 1, Line 15
RESTORE LOG is terminating abnormally.
RESTORE DATABASE successfully processed 0 pages in 0.544 seconds (0.000 MB/sec).
When I click the Restore option of the original database and click on Timeline option I get the below screen
Table dropped timing from fun_dblog:
Ok, to avoid getting into a long chat about this here is what I suggest:
Restore the full backup you took 4 months ago, NOT the one you took today:
RESTORE DATABASE [databasecopy] FROM DISK = N'OLD_BACKUP.bak' WITH MOVE N'database' TO N'C:\SQLskills\database2.mdf', MOVE N'database_log' TO N'C:\SQLskills\database2_log.ldf', REPLACE, NORECOVERY; GO
Then run the RESTORE up to that LSN:
RESTORE LOG [databasecopy] FROM DISK = N'D:\SQLskills\database_Log2.bak' WITH STOPBEFOREMARK = 'lsn:3420000002597000001', NORECOVERY;
GO
RESTORE DATABASE [databasecopy] WITH RECOVERY; GO
It will not work if you use your current Full Back up because everything in the log at that point has already been committed and you are trying to go back in time. Restore only goes forward in time. That is the reason for your error.
I am using a daily scheduled job to have a backup of my on-premises sql server DB on Azure blob.
The command never changed and worked for many months but since 3 days ago started failing.
This is the text of the command:
DECLARE #MyFileName varchar(1000)
SELECT #MyFileName = (SELECT 'https://myurl.blob.core.windows.net/full/MyDB_backup_' + convert(varchar(500),GetDate(),112) + '.bak')
BACKUP DATABASE [MyDb] TO URL = #MyFileName WITH CREDENTIAL = N'AzureCredential' , COPY_ONLY, NOFORMAT, NOINIT, NAME = N'MyDb-Full Database Backup', NOSKIP, NOREWIND, NOUNLOAD, STATS = 1
And this is the output:
1 percent processed.
2 percent processed.
3 percent processed.
Msg 121, Level 20, State 0, Line 24
A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The semaphore timeout period has expired.)
This creates a 1TB file (while it normally is 6Gb) and when I try to delete it I get this error: There is currently a lease on the blob and no lease ID was specified in the request.
I believe there is a problem with their service as the code I run and database didn't change. However I don't know how to report a bug.
I would have raised a ticket for this directly with Microsoft but apparently I would need a "Support Plan". So I posted here as it's the resource they list for their support.
Did you experience this problem before?
Where do you think is the most appropriate place I should post this to get an answer?
It seems that the issue has been fixed by Azure today as it started to work again.
It started on the 28th of October so it lasted for 5 days.
However, this is not reported in any page about the status of Azure.
If it happens to any of you again apparently the only way is to wait for days until it works.
I will update this thread if the problem happens again.
When using backup to URL on SQL Server 2012 or 2014, SQL allocates a 1 TB page blob so it can store the maximum backup size supported. After the backup process completes the lease on the blob will be removed and the blob will be re-sized to the size of the backup.
If the backup process fails to complete, the allocated file size may potentially remain at the 1TB size, however you will only be billed for the storage actually used by the blob (see the empty disk FAQ here). Additionally, if error occurs during the backup process an active lease may still be held on the page blob. To deal with deleting blobs with active leases please see this document.
The solution is an ASP.NET MVC application using E/F hosted in IIS on a Windows Server 2012 R2 Standard VM hosted in a Hyper-V environment. The same VM is running SQL Server 2012.
The hosting environment is hosting 30 other solutions and there is plenty of free disk space and no known disk problems with hosting environment or VM (chkdsk and sfc has been run on VM and did not report any problems).
The problem is that the solution/server stops working for short periods of 5-1o minutes and every time we see event ID 508/533 from ESENT and a message about writing to "C:\Windows\system32\LogFiles\Sum".
A similar message has been seen with sqlsvr but this was solved by giving everyone all rights to C:\Windows\system32\LogFiles\Sum.
When the problem persists, it affects the whole VM and sometimes it is no even possible to connect via remote desktop.
We have seen a high number of open SQL Server connections when the problems occurs and prior to introducing caching for a specific Web API method we were actually able to empty the SQL Server connection pool. Just in case we have changed the connection pool from 100 to 200 connections even though we have not seen this particular problem since we introduced the cache.
All DbContext instances are disposed by "using", an ApiController.Dispose override or a Controller.Dispose override and only one SqlConnection are used (for the logging system).
I suspect the problem to be outside the solution and that the high number of SQL Server connections are related to the fact that SQL Server is unable to write to the disk.
Below is some Windows Event Log excerpts for three recent "break downs" with some additional info about the number of web request prior to the problem and after the server has automatically recovered.
Any suggestions?
web requests during the 10 minutes right before the problem: 1399
web requests during the first 10 minutes after the server has recovered: 1630
18-03-2015 20:07:20 833 MSSQLSERVER
SQL Server has encountered 1 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\Xxx.mdf] in database [Xxx] (5). The OS file handle is 0x0000000000000A7C. The offset of the latest long I/O is: 0x000003e104e000
18-03-2015 20:07:40 833 MSSQLSERVER
SQL Server has encountered 1 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\Xxx_log.ldf] in database [Xxx] (5). The OS file handle is 0x0000000000000A8C. The offset of the latest long I/O is: 0x0000007f203000
18-03-2015 20:08:16 533 ESENT
svchost (1740) A request to write to the file "C:\Windows\system32\LogFiles\Sum\Svc.log" at offset 1806336 (0x00000000001b9000) for 4096 (0x00001000) bytes has not completed for 36 second(s). This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
18-03-2015 20:17:14 508 ESENT
svchost (1740) A request to write to the file "C:\Windows\system32\LogFiles\Sum\Svc.log" at offset 1806336 (0x00000000001b9000) for 4096 (0x00001000) bytes succeeded, but took an abnormally long time (36 seconds) to be serviced by the OS. This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
web requests during the 10 minutes right before the problem: 696
web requests during the first 10 minutes after the server has recovered: 614
19-03-2015 01:17:19 533 ESENT
svchost (1740) A request to write to the file "C:\Windows\system32\LogFiles\Sum\Svc.log" at offset 3067904 (0x00000000002ed000) for 4096 (0x00001000) bytes has not completed for 36 second(s). This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
19-03-2015 01:33:02 508 ESENT
svchost (1740) A request to write to the file "C:\Windows\system32\LogFiles\Sum\Svc.log" at offset 3067904 (0x00000000002ed000) for 4096 (0x00001000) bytes succeeded, but took an abnormally long time (983 seconds) to be serviced by the OS. This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
19-03-2015 01:33:03 833 MSSQLSERVER
SQL Server has encountered 5 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\Xxx_log.ldf] in database [Xxx] (5). The OS file handle is 0x0000000000000A8C. The offset of the latest long I/O is: 0x000000a389d000
web requests during the 10 minutes right before the problem: 555
web requests during the first 10 minutes after the server has recovered: 784
19-03-2015 03:33:51 833 MSSQLSERVER
SQL Server has encountered 1 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\Xxx_log.ldf] in database [Xxx] (5). The OS file handle is 0x0000000000000A8C. The offset of the latest long I/O is: 0x000000aa95f000
19-03-2015 03:40:48 533 ESENT
svchost (1740) A request to write to the file "C:\Windows\system32\LogFiles\Sum\Svc.log" at offset 3846144 (0x00000000003ab000) for 4096 (0x00001000) bytes has not completed for 36 second(s). This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
19-03-2015 03:40:48 833 MSSQLSERVER
SQL Server has encountered 1 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\MSDBLog.ldf] in database [msdb] (4). The OS file handle is 0x0000000000000A90. The offset of the latest long I/O is: 0x00000000108000
19-03-2015 03:40:49 508 ESENT
svchost (1740) A request to write to the file "C:\Windows\system32\LogFiles\Sum\Svc.log" at offset 3846144 (0x00000000003ab000) for 4096 (0x00001000) bytes succeeded, but took an abnormally long time (36 seconds) to be serviced by the OS. This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.
19-03-2015 03:40:49 17894 MSSQLSERVER
Dispatcher (0x1a88) from dispatcher pool 'XE Engine main dispatcher pool' Worker 0x00000000F03B8160 appears to be non-yielding on Node 0. Approx CPU Used: kernel 0 ms, user 0 ms, Interval: 336140.
Disk I/O problems was my initial thought but the "funny" thing is that it actually never has happened during peak hours and that the server during peak hours is not stressed on CPU or disk I/O.
I cannot find any VM disk errors. I have no access to the hosting environment but I am told that there are no disk problems. The hosting environment is performing VM backups and if this is the problem, there is nothing to do about it, as it is required. I might try to have the VM moved to another disk but I do not know if this is possible.
Currently we have set up some detailed disk I/O monitoring on the VM and hopefully this will give us some information about the problem but I rather doubt it.
Maybe the VM is just "sick" and the next step might be to create a new one from scratch…
It sounds like your disk is just plain overloaded, since I/Os are taking so long. Ideally they should take around 10 milliseconds. Instead, they're taking over 1000x that long.
Since you're running in a VM, though, tracking down the problem can be a bit more tricky. Is it due to the I/O load in the virtual machine, or on the host? Your VM disk may be shared with other I/O load of the host.
Can you move the database to a different volume in the VM, hosted on a different physical spindle of the host?
Another possibility is that the underlying storage is going bad, and the I/Os are being retried by the underlying hardware.
-martin