I am attempting to refresh a TFS 2015 test environment on a regular basis. The TFS production environment consists out of 3 servers, and the test environment has the exact same layout (separate App, Build and SQL server - no sharepoint involved).
TFS version : 14.0.23128.0 (Tfs2015)
SQL Server 2012 SP2 CU-1
OS : Windows 2008 R2 Enterprise
Seeing that the refresh will occur regularly and to minimize down time, I want to automate / script the process to be execute at night.
Getting a production back-up from the SQL Server seems no problem, as this could be scheduled from within the TFS Admin Console (Scheduled backups).
At this stage I am trying to figure out how the “scheduled restore” will work. In no time I was bumping my head against attempting to stop all collections before the restore starts on the test environment...
My question: Is such an automated style of TFS backup and restore doable, and is there by any chance a product out there that supports this out of the box?
You can setup automated refresh for every MSSQL database. I would do it using SQLAgent task. Restore command should be preceded with following statement to ensure all connections are terminated otherwise restore will fail:
ALTER DATABASE TFS_POC
SET RESTRICTED_USER
WITH ROLLBACK IMMEDIATE
GO
There is not a way to have a total automated style of TFS restore, no matter you are using TFSBackup.exe, or using Restore Databases in Scheduled Backups, selecting your database backup is required. Also, stopping services is necessary, as stopping services helps protect against data loss or corruption during the restoration process.
To provide a bit of feedback on the issue.
The requirements were fairly simple in our case in the sense that we only wanted the version control databases restored.
This simplified things a bit as restoring functionality such as reporting, builds, sharepoint etc. were left out of scope.
In the end I did succeed in getting the process automated to an extent, by making use of an amalgamation of SQL server stored procedures, Powershell and some standard Tfs commands.
The first step was to get the production Tfs databases backed up.
This was achieved by making use of SQL server stored procures.
Note: You must back up all of the databases to the same time stamp to help ensure against data loss!
https://www.visualstudio.com/en-us/docs/setup-admin/tfs/admin/backup/backup-db-architecture
These database backups are then copied to a shared restore location where the test environment’s Tfs SQL Server can access them during the restore process.
This was also done via some SQL stored procedures.
These steps are scheduled within SQL Server to execute nightly, with the previous day’s backup in the shared location being overwritten each time.
For the orchestration of the restore actions involved, TeamCity was used.
When a restore is requested you can initiate (or schedule) it via TeamCity.
The first step will be to stop all the test environment’s Tfs collections.
This was done by using the TfsServiceControl.exe - https://www.visualstudio.com/en-us/docs/setup-admin/tfs/command-line/tfsservicecontrol-cmd
Next we will execute some stored procedures on the test environment’s SQL Server that will start the DB restore process of the test environment’s Tfs DB’s.
The next steps will prepare the Tfs databases to run in the test environment and among other things, clear away the production values that are embedded in them.
This was done by a PowerShell script that in essence just calls the commands that will be run when a manual restore is done. Most of the commands reside within the TfsConfig command.
https://www.visualstudio.com/en-us/docs/setup-admin/tfs/commandline/tfsconfig-cmd
TfsConfig PrepareClone
TfsConfig ChangeServerID
TfsConfig RemapDBs
TfsConfig Accounts /ResetOwner
TfsConfig Accounts /add
TfsConfig registerDB
TfsConfig ConfigureMail
TfsConfig RebuildWarehouse
Clearing cache
After these steps were completed the test environment’s Tfs collections are started.
This is done by using the TfsServiceControl.exe - https://www.visualstudio.com/en-us/docs/setup-admin/tfs/command-line/tfsservicecontrol-cmd
It is now possible to request or schedule a restore. A bit primitive, but so far it works fine.
Related
We are currently backing up SQL Server databases to an Azure URL as a page blob. SQL Server version is 2012 SP3. At times I need to get the backups to our Dev, Beta, and Test environments to be restored. I'm having an issue with a large backup (~50 GB). There are several smaller backups that are copied and restored okay but the one large backup is the problem. I've tried using PowerShell scripts and AzCopy. When attempting to restore it using the GUI, the backup is not recognized. When attempting to RESTORE VERIFYONLY, I get the error - Msg 3013 (VERIFY DATABASE is terminating abnormally.)
If I do a restore verify from URL, the backup is valid. If I try to restore from URL, it restores but takes forever (almost 6 hours).
Has anybody else had this problem?
The problem is that the backup is not being completely copied because the AzCopy program is interrupted.
After doing a little research, I discovered there are many options to AzCopy. One of them is the capability to resume the copy if it is interrupted. This is handled through a PowerShell script.
AzCopy has great help - AzCopy.exe /?.
Does SharePoint 2013 restore only from Database?
I have a scheduled script in MSSQL Server to run all database backups daily , and my SharePoint site also require a daily differential/weekly full backup usually happen in Central Administration. I am aware that multiple backups running would break log chain in this case.
If I stop doing backup in Central Administration and let DB does the backup only, would I be still able to restore my SharePoint site (Contents and Configurations)?
Does SharePoint 2013 restore only from Database?
The short answer is no. A full fidelity SharePoint farm backup is mostly databases but there is also configuration information and data that is stored outside of the databases. The Central Admin backup facility (as well as the Backup-SPFarm powershell commands) initiate SQL backups as well as backups of all the stuff that isn't in SQL. That is the only point-and-click (or type a single command) solution.
Could you get away with only having some of the databases to recreate your environment? Sure but then you'd have to have a documented and tested (and ideally automated) process for recreating the farm from the databases.
I have a server which is running SQL Server 2008R2 SP1 and there are more than 200 online databases with their specific login and I want to move all of these databases to another machine with the same software SQL Server 2008R2 SP1.
I can afford downtime up to 8 hours so I'm not looking for complex solution like database mirroring and logshipping.
I have tried Backup & Restore, it work fine but trouble me in two ways :
1- I couldn't transfer login to new server so all of my users will have to create their user again which will my users to get angry.
(I have tried to script all of the logins from the security section but since I have realize the hash algorithm between two system maybe different, I cannot transfer my logins in this way)
2 - I have to backup & restore each database one by one which is so time consuming.
I have also tried copy wizard, but I have encountered with following error message at the final stage :
"SQL Server schedule job" job failed the job was invoked by user sa step to run was step 1.
Since mysource server is running based on windows server 2003, I cannot use the powershell V3 solution.
I have done many searches but all I found was a solution to transfer one database to another server.
But since I'm looking for a solution to transfer mass databases, this situation is being hair pulling and so difficult So I wil be very appreciated if you can provide me an easy and practical solution.
for moving logins between servers
https://support.microsoft.com/en-us/kb/918992
[moving logins][1]
then backup and restore all databases in that migraton day using pre built restore script if the db size is small , later restore the differential backups and make it online. if the size is an issue then implement log shipping or mirroring
We currently have a Live and Dev environment within our offices, at regular intervals we need to move the live DB to Dev to make sure data is updated for the dev team to work with.
However the live DB is becoming very difficult to manage as its almost hit 100Gb, we currently run a DB backup, copy the file to the other server and restore it. however this is becoming a major headache that can take upwards of 4 - 5 hours.
does anyone have any good recommendations for how we can move the DB in a more efficient manor?
We are using MS SQL Server 2005 Standard Edition.
The best way to update your dev server from the production is to implement a shipping logs strategy. Perform an incremental backup of your production database daily and place the incremental log in a location where the development server can see it. Then once a week take all the incremental backups (there should be 5) and increment the development database to make it look like the production. The process can be automated with sql server tools out there if you want it to, or you can write a little program that will generate the scripts for you from the file names in the directory where you put the log files. After you do the operation a few times and see the TSQL that SQL server will generate for you each time, you will get a good idea how to write the script generator utility. You can even automate the restore process to your dev box with the same utility, just connect to the dev server and run the scripts that it generates and then automate the running of the utility. Most programmers could whip up this utility in a day or two at best as long as they have a decent understanding of sql server and tsql.
You have other options as well, but this one would probably solve most of your issues
You get incremental backups of your production database in addition to your full backups that you may or may not do.
The utility you write will save time and automate the process all you have to do is check that it was successful or not and you have the utility email you the success/failure. If you are cloud based use an amazon tool for email, or if you azure based, use sendgrid.com.
Your time for producing the utility is not great.
I have live and dev versions of an ASP.NET website, with corresponding live and dev versions of the SQL Server back end. Each is on its own server (so 4 total) on the same local network.
Every so often, we've been manually replacing the dev DB with the live DB, using backup & restore, so that the dev DB is a reasonably recent snapshot of the live DB. What's the best way to automate that process?
I'd be interested in either having it run on a schedule, or making it a pushbutton process where we still control the timing.
(For what it's worth, we use CruiseControl.net; so an option that would let us use the CC.net dashboard to launch this process would be nice.)
1- you can use replication (transactional or snapshot) for synchronizing two databases.
2- write an application that get backup from db and restore it. (use SMO)
3- write an application that delete all data from dev db and copy all data from live db to it. you can use bcp (SQLBulkCopy) for doing this work.
I think scheduling or automating a restore would be relatively simple with the RESTORE command in T-SQL. Schedule your backups to go to a designated location and set whatever schedule / script that refreshes the dev database to look in the same location.
Why not set up a Snapshot replication between your live and dev database? This is the way we handle keep our dev database in synch with our production database.
http://msdn.microsoft.com/en-us/library/ms151198.aspx