We have TFS2015.3. And we have scheduled DB backup in TFS using TFS admin console. But we are finding that TFS DB back up failing frequently with below error:
Exception Message: Write on "\XASDC\public\DASC\IT\BACKUPSQL\XXXXX1234\TFS\Tfs_CollectionA_9859957732898727191F.bak" failed: 1359(An internal error occurred.)
BACKUP DATABASE is terminating abnormally. (type SqlException)
SQL Exception Class: 16
SQL Exception Number: 3202
SQL Exception Procedure:
SQL Exception Line Number: 1
SQL Exception Server: XXXXX1234
SQL Exception State: 1
SQL Error(s):
SQL Error[1]: System.Data.SqlClient.SqlError: BACKUP DATABASE is terminating abnormally.
Class: 16
Number: 3013
Server: XXXXX1234
Source: .Net SqlClient Data Provider
State: 1
Procedure:
Line Number: 1
Exception Data Dictionary:
HelpLink.ProdName = Microsoft SQL Server
HelpLink.EvtSrc = MSSQLServer
HelpLink.EvtID = 3202
HelpLink.BaseHelpUrl = http://go.microsoft.com/fwlink
HelpLink.LinkId = 20476
Exception Stack Trace: at Microsoft.VisualStudio.Services.Configuration.SqlDatabaseBackupRestoreComponent.Execute(String sqlStatement)at Microsoft.VisualStudio.Services.Configuration.SqlDatabaseBackup.Backup(ISqlConnectionInfo connectionInfo)
at Microsoft.TeamFoundation.Admin.Jobs.DatabaseBackupJob.TakeFullBackupOfSingleDatabase(IVssRequestContext requestContext, String dataSource, String databaseName, String fullFileName, ITFLogger logger)
at Microsoft.TeamFoundation.Admin.Jobs.DatabaseBackupJob.TakeFullBackups(IVssRequestContext requestContext, BackupSettings settings, List`1 databaseConnectionInfoList, ITFLogger logger)
It is failing only for a particular Collection. We have 11 collections and Back-up is fine for 10 Collections. But for this particular collection, the scheduled Back-up failed. On rerunning the Backup job manually it succeeds. It failed twice in last 4 days.
Kindly requesting your help to get the issue resolved.
Seems it could be a connection issue with high workload, or an issue like "data cannot be read".
Just try below items to narrow down the issue :
Detach the specific collection from TFS admin console, then re-attach
it.
Change the backup location to local machine instead of UNC path (There is a possibility that caused by the network.).
Recreate the BackupSets file:
Go to the directory where your back ups are stored
Find something like "BackupSets.xml"
Rename it to "BackupSets_BACKUP.xml" (or anything else of your liking)
Schedule the backup in TFS Administration Console
Related
I have a database running on an azure vm with sql server. The db is in full recovery mode. The backup is configured through the web interface. Database and log backups have been working flawlessly for years. But recently the log backup was interrupted halfway through and the log backup process somehow got stuck. The following event has been logged every 5 minutes since then (reading log with managed_backup.sp_get_backup_diagnostics):
[SSMBackup2WAAdminXevent] Database Name = DB, Database ID = 777, Stage =
VerifyJobOutcome, Error Code = 0, Error Message = Warning, Additional Info = A
progress update hasn't been received from SQL Server in more than 30 minutes
for log backup. SSMBackup2WA will continue to wait.
SSMBackup2WA seem to be stuck waiting for a progress update never being received. This has resulted in no log backups being taken. The database backup have continued running without problem.
I have trouble finding the job/task used by SSMBackup2WA. I understand its not in the usual batch of SQL Server Agent jobs but somehow hidden.
My idea is to somehow cancel the existing job that is stuck in waiting loop but I have not figured out how.
I have tried to "reset" the backup process by turning off the backup and then turning it on again but that did not help.
I have no possibility to restart the sql server (and I don't know if that would help).
So since no one seemed to have an answer to this one I resorted to restarting the SQL-server. And after the restart the transaction log backup started working again!
What is interesting is the following log that appeared in the application event log during the restart. It does seem like there was a thread hanging indefinitely, waiting for an status update that never arrived. The restart seems to have taken care of it by killing this status thread and not restarting it again in the erroneous state it had ended up in.
Log Name: Application
Source: Microsoft SQL Server Automated Backup
Date: 1/15/2022 11:16:20 AM
Event ID: 57007
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: wn-sqlserver1
Description:
[Warning] AutomatedBackupStatusMonitorError:
System.Exception:
Error in auto-backup status monitor thread --->
Microsoft.SqlServer.Management.IaaSAgentSqlQuery.Contract.IaaSAgentSqlQueryException:
A network-related or instance-specific error occurred while
establishing a connection to SQL Server. The server was not
found or was not accessible. Verify that the instance name
is correct and that SQL Server is configured to allow remote
connections. (provider: Named Pipes Provider, error: 40 - Could
not open a connection to SQL Server) --->
This is the scenario:
I am testing redundancy and fail over in my dev machine by creating two SQL Servers.
I have created two SQL Servers. One with an extra instance:
-SQL1 : primary server
-SQL2 : mirror server
-SQL2\wtn : witness instance
First I made a full backup of the database and transaction log in my primary server, then I restored the database and transaction log file.
I used option "Restore with no recovery".
In the database node "restoring" is being shown. I believe this is normal when you would like to keep pulling data.
Then in the primary server, I tried to create mirroring on the database. After the wizard completed, I clicked on start Mirroring, but now I am getting the following error:
An error occurred while starting mirroring.
------------------------------
ADDITIONAL INFORMATION:
Alter failed for Database 'WSS_Content_1'. (Microsoft.SqlServer.Smo)
An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)
------------------------------
Database 'wss_content_1' cannot be opened. It is in the middle of a restore. (Microsoft SQL Server, Error: 927)
I see there is another post here on stack overflow, but that does not have much information. I have been waiting more than 10 minutes.
Update
I am using SQL Enterprise 2016 by the way. Perhaps there is a difference in 2016 version.
I was watching this youtube video and my steps were exactly as this guy's.
Update 2:
I followed this as well but did not help.
How to: Prepare a Mirror Database for Mirroring (Transact-SQL)
I have two databases (at the same server) in Microsoft SQL server. One of them can be successfully accessed remotely. However, the other not. It returns the following message in the error log:
Login failed for user 'adminUsr'. Reason: Failed to open the explicitly specified database 'alg_test.alg_test'. [CLIENT: ]
Error: 18456, Severity: 14, State: 38.
Then I go to Microsoft SQL server management and check the status of the database with:
SELECT databasepropertyex('alg_test.alg_test', 'STATUS')
and got this:
RECOVERING
It seems that the database is constantly recovering. How can I fix this? and finally geet access to the database remotely.
Check the SQL Server error log for related messages to see why the database is recovering. Common causes include:
The database was restored with the NORECOVERY option from full, differential, and log backups but RECOVERY was not specified on the last restore. The solution in this case is simply execute RESTORE <your database> WITH RECOVERY; to rollback uncommitted transactions and bring the database online.
The transaction log filled due to a large data modification operation and SQL Server is rolling the transactions(s) back to recover the database, which can take quite a bit of time. The error log will include recovery progress messages. It that's longer than you want to wait, it may be more expeditious to restore the database from backup(s). Be aware that if SQL Server is restarted during the database recovery process, recovery will restart from the beginning at service startup.
Well you could try this script
RESTORE DATABASE mydatabase WITH RECOVERY
SO this will finish the recovery process with no backup files. If you are getting an error message 'database in use'. I think you shoudl try to stop the service, delete the databse and then Restore it with 'Recovery'
Try this one :
Use [master]
GO
RESTORE DATABASE [DataBaseName] WITH RECOVERY
I'm hoping to configure and use SQL Server 2014's Managed Backup to Azure feature. I've tested a manual backup to URL T-SQL query, and it worked. But, the backups created by the managed backup to Azure feature are failing to finish.
The managed backup process leaves a bunch of 0 byte backup files on Azure storage, and they're all locked (active leases) because of the failed backup process. I'm trying to figure out what the issue is. My SQL Server 2014 edition is the Web edition, which apparently does not support compression. The SQL Server Management Studio dialog for configuring the managed backup to Azure feature does not provide any options for compression, though. So, perhaps it's trying to compress the backup file and failing? I don't know how to see what's actually being tried by SQL Server while it does its automated backup to Azure, though. Anybody have any ideas there?
I'm seeing the following errors in the event log ...
Date 10/1/2014 3:10:39 PM
Log SQL Server (Current - 10/1/2014 2:42:00 PM)
Source Backup
Message
BACKUP failed to complete the command BACKUP LOG CaseLaw. Check the backup application log for detailed messages.
Date 10/1/2014 3:12:01 PM
Log SQL Server (Current - 10/1/2014 2:42:00 PM)
Source spid71
Message
Error: 18210, Severity: 16, State: 1.
Date 10/1/2014 3:12:01 PM
Log SQL Server (Current - 10/1/2014 2:42:00 PM)
Source spid71
Message
BackupVirtualDeviceFile::DetermineFileSize: SetPosition(0,EOF) failure on backup device 'https://caselaw.blob.core.windows.net/sqldb01-mssqlserver/CaseLaw_d68e6c91feba4f7f891a66cb86fcce8d_20141001151038-05.log'. Operating system error Backup to URL received an exception from the remote endpoint. Exception Message: The remote server returned an error: (409) Conflict..
I can't read the full 409 error message, so I've got no idea what the actual 409 error is. According to MSDN, there are multiple possible 409 responses.
I have been struggling with this issue for several days. I tried a manual scheduled task that runs my T-SQL query, which works, but I can't have it automatically clean itself up after X days, using that method. Any ideas? Thanks!
This is indeed related to COMPRESSION being set on the backup command that gets called by the Managed Backup process.
Enabling debug messages pointed me to the stored procedure getting called: [msdb].[smart_admin].[sp_do_backup]. If you look inside this stored procedure you will see "COMPRESSION" hard coded into the various backup commands that it issues.
A find and replace for "COMPRESSION" to "NO_COMPRESSION" should fix your issue.
Note this only need apply to SQL 2014 Web Edition, as other versions support compression, and this shouldn't even come up.
Getting errors restoring TFS databases
I've scheduled the backups of our team foundation server with the Team Foundation Server Administration Console. It takes every night a full backup, and it looks like it really does.
Now disaster occured, our tfs crashed, and i'm setting up a new windows server with tfs 2012. Luckely i've got backups... But.....
Restoring the backups seems not so easy.
When i restore, I get the following error messages:
TF401001: The restore operation for database ... failed. Please refer to the logs for more information
I get this for every database (ReportServer, ReportServerTempDB, Tfs_warehouse, tfs_defaultcollection, TFS_configuration).
Looking at the log file:
[Info #09:07:33.420] RESTORE DATABASE [Tfs_Configuration] FROM DISK = N'\\172.16.46.234\Backup_TeamFS.crashed\Tfs_Configuration_9858508765097608601L.trn' WITH RECOVERY, NOUNLOAD, STATS = 10, STOPATMARK = N'TfsMarkTfpt'
[Error #09:07:33.514] TF401001: The restore operation for database Tfs_Configuration failed. Please refer to the logs for more information
[Error #09:07:33.514]
Exception Message: The database "Tfs_Configuration" does not exist. RESTORE can only create a database when restoring either a full backup or a file backup of the primary file.
RESTORE DATABASE is terminating abnormally. (type SqlException)
SQL Exception Class: 16
SQL Exception Number: 3118
SQL Exception Procedure:
SQL Exception Line Number: 1
SQL Exception Server: TEAMFS
SQL Exception State: 1
SQL Error(s):
SQL Error[1]: System.Data.SqlClient.SqlError: RESTORE DATABASE is terminating abnormally.
Class: 16
Number: 3013
Server: TEAMFS
Source: .Net SqlClient Data Provider
State: 1
Procedure:
Line Number: 1
Exception Data Dictionary:
HelpLink.ProdName = Microsoft SQL Server
HelpLink.EvtSrc = MSSQLServer
HelpLink.EvtID = 3118
HelpLink.BaseHelpUrl = http://go.microsoft.com/fwlink
HelpLink.LinkId = 20476
Exception Stack Trace: at Microsoft.TeamFoundation.Admin.SqlDatabaseBackupRestoreComponent.Execute(String sqlStatement)
at Microsoft.TeamFoundation.Admin.SqlDatabaseRestore.Restore(String connectionString)
at Microsoft.TeamFoundation.Admin.RestoreFactory.RestoreLogs(ActivityContext context, String serverName, String databaseName, String path, List`1 backupChain, Boolean useMarkAtEnd, Int32 minPercent)
at Microsoft.TeamFoundation.Admin.RestoreDatabases.Run(ActivityContext context)
But, i'm fully sure i do have a full backup. Looking at log files from the backup:
[9/1/2013 10:24:44 PM] [Info] starting full backups
[9/1/2013 10:24:44 PM] [Info] Taking Full Backup server:TEAMFS database:Tfs_Configuration filename:\\172.16.46.234\Backup_TeamFS\Tfs_Configuration_9858508747697505276F.bak
[9/1/2013 10:24:44 PM] [Info] BACKUP DATABASE [Tfs_Configuration] TO DISK = N'\\172.16.46.234\Backup_TeamFS\Tfs_Configuration_9858508747697505276F.bak' WITH NOFORMAT, NOINIT, NOSKIP, REWIND, NOUNLOAD, STATS = 10
[9/1/2013 10:32:17 PM] [Info] Backup Complete for database Tfs_Configuration
[9/1/2013 10:32:17 PM] [Info] Finished Full Backup server:TEAMFS database:Tfs_Configuration filename:\\172.16.46.234\Backup_TeamFS\Tfs_Configuration_9858508747697505276F.bak
update
I've only scheduled the backups to make a full backup, not incrementals. Maybe something to do with that?
we have managed to do this so I shall share my experience..
So you have got backups..
Also, you have got a fresh new OS and installation files for TFS (TFS 2013 in our case)
Double click installation file.
Once installation completes, ideally MSDN tells you to close the windows and then in the TFS CONSOLSE you have to go to scheduled backups to restore which FAILS so below method will help you resolve the problem:
a. Restore the config and team collections databases via the SQL Management console
b. Open the TFS management console
c. Click on Application Tier and Configure this feature
d. Run CMD as administrator and run following command:
TFSConfig Accounts /add /AccountType:ApplicationTier /Account:"NT Authority\Network Service" SQLInstance:ServerName /DatabaseName:TFS_Configuration
e. Then just follow the next button in the tool from step c.
Cheers
Here is how I just walk around this issue. Use your SQL management tool to restore the backup file. They are just like regular Database backup files, so you just treat them like ones.
After you finish, you can see your collections in your TFS console.
You might bump into some errors during the process.
The one I got was missing one collection. You need to restore everything before going to TFS console.
BTW, after you finish, reboot your TFS service or the whole server, otherwise you might bump into another error like can't find the object reference.
Check out this blog to find some help.
BTW, I assume the reason you did this because you tried to restore TFS to a different server.
You need to reinstall the application tier only from TFS installation. It took me forever to figure it out.