OK..so I'm new to Git / Github after being on Visual Source Safe most of my career and have seen the light - love it. So I have migrated all my .NET projects to my Github account and would like to also manage all my SQL Server databases with Git as well. In all my projects I always put my database files in a /Database subdirectory so I have, for example, /Databases/MyDatabase.mdf and /Databases/MyDatabase.ldf in my source tree. I am tracking these files with Git and they show in Github nicely with all the rest of my source.
Here's what I was expecting to happen: I stop SQL Server with a NET STOP MSSQLSERVER and I checkout a branch so Git will pull down the MDF and LDF files for that branch. Restart SQL Server with NET START MSSQLSERVER and do whatever work I need to do on the source and database for that particular branch. Git would track my changes to the MDF file and when I do a stage / commit / push it would send the changes back up to the remote repository.
I tried this by pulling down the database and adding a new column to a table and doing a commit. Git told me that there were no changes to any items which I was not expecting...I changed the MDF file. Is it not possible for Git to track changes in an MDF file? My first guess was that maybe because it was binary and not text base Git might have trouble with it but I believe it's possible to use Git to keep track of image files and other binary items so that doesn't seem like it would be the issue. Any ideas? Is it just not possible? Should I not even be trying to do this? Thanks in advance for your comments.
It is very bad idea to place changing database files into any source control. Try to create/update appropriate scripts and store them in source control - this is the proper way to track schema changes inside the DB
If you're using Visual Studio - this is a good point to start with database and server projects in it. But sometimes it behave in weird manner, so use with caution and gently
OR you can use some commercial/free software to track the changes inside DB schema and data, like RedGate Schema Compare or Redgate Data Compare
Have you considered using our SQL Source Control tool to keep track of development changes. This does all the 'scripting out' for you behind the scenes. It actually uses SQL Compare's engine under the hood.
http://www.red-gate.com/products/sql-development/sql-source-control/
As Oleg correctly points out, it's possible to track schema changes using SQL Compare and SQL Data Compare, but here at Red Gate we wouldn't recommend you do this over maintaining your development environment under source control. Ideally you should do both. Grant Fritchey has written an excellent article describing how SQL Compare's command line can be used with a source control system to track schema changes. He uses SourceGear Vault in his examples, but the principles apply to any source control system.
http://www.simple-talk.com/sql/database-administration/auditing-ddl-changes-in-sql-server-databases/
Related
I have a problem attaching ms sql server database after syncing with git. My steps are:
1. I use Dropbox to keep the bare repository instead of github.
2. The working repository stored in C drive. This repository contains my code and database (ms sql server 2008). So I commit and push changes to bare repository (in dropbox).
3. On the second computer, I clone that project from dropbox (bare repository).
4. At first, the 2nd computer can attach database. Then it edits database, detach database, and push changes to bare repository in dropbox.
5. The problem occurs here, the first computer cannot pull the changes from bare repository. It shows permission deny. I use SmartGit as GUI for git. Beside, when I do these steps again, the other problem is that the first computer can pull the changes but it cannot attach the database any more.
I guess when the second computer edits database, it gave its own permission to access database, that’s why the first one cannot get changes or attach database.
The purpose of the above steps is that I want to keep my database in dropbox using git so that I can work with database from any computer without copying it. Before I used Dropbox to do this, but it cannot sync database, that’s why I think of using git. I do not need git to version control my database, just keeps it in one place for portable working.
If you guys think the above method is not practical, could you suggest me a way of doing this? Thank you so much in advance.
Update:
Problem is solved.
Resolution: 2 ways:
- Generate script based on ms sqlserver using msdeploy and version control that script. (I'm using this method).
- Using data tier app (DAC), but I'm new in this. Will play with it later.
I have Git Source Control Provider setup and running well.
For Visual Studio projects, that is.
The problem is that each such project is tightly tied to a SQL Server database.
I know how to version control a database in its own .git repository but this is neither convenient nor truly robust because ideally I would want the same ADD, COMMIT, TAG and BRANCH commands to operate on both directory trees simultaneously, in a synchronized manner.
Is there a way to Git SQL Server database with Visual Studio's Git Source Control Provider in the manner I described?
You can install the SQL Server Data Tools if you want to, but you don't have to: You can Use the Database Publishing Wizard to script your table data right from Visual Studio into the solution's folder, then Git it just like you do with the other project files in that folder.
You can store your database schema as Visual studio project using SQL Server Data Tools and then version control this project using Git.
Being in the database version control space for 5 years (as director of product management at DBmaestro) and having worked as a DBA for over two decades, I can tell you the simple fact that you cannot treat the database objects as you treat your Java, C# or other files and save the changes in simple DDL scripts.
There are many reasons and I'll name a few:
Files are stored locally on the developer’s PC and the change s/he
makes do not affect other developers. Likewise, the developer is not
affected by changes made by her colleague. In database this is
(usually) not the case and developers share the same database
environment, so any change that were committed to the database affect
others.
Publishing code changes is done using the Check-In / Submit Changes /
etc. (depending on which source control tool you use). At that point,
the code from the local directory of the developer is inserted into
the source control repository. Developer who wants to get the latest
code need to request it from the source control tool. In database the
change already exists and impacts other data even if it was not
checked-in into the repository.
During the file check-in, the source control tool performs a conflict
check to see if the same file was modified and checked-in by another
developer during the time you modified your local copy. Again there
is no check for this in the database. If you alter a procedure from
your local PC and at the same time I modify the same procedure with
code form my local PC then we override each other’s changes.
The build process of code is done by getting the label / latest
version of the code to an empty directory and then perform a build –
compile. The output are binaries in which we copy & replace the
existing. We don't care what was before. In database we cannot
recreate the database as we need to maintain the data! Also the
deployment executes SQL scripts which were generated in the build
process.
When executing the SQL scripts (with the DDL, DCL, DML (for static
content) commands) you assume the current structure of the
environment match the structure when you create the scripts. If not,
then your scripts can fail as you are trying to add new column which
already exists.
Treating SQL scripts as code and manually generating them will cause
syntax errors, database dependencies errors, scripts that are not
reusable which complicate the task of developing, maintaining,
testing those scripts. In addition, those scripts may run on an
environment which is different from the one you though it would run
on.
Sometimes the script in the version control repository does not match
the structure of the object that was tested and then errors will
happen in production!
There are many more, but I think you got the picture.
What I found that works is the following:
Use an enforced version control system that enforces
check-out/check-in operations on the database objects. This will
make sure the version control repository matches the code that was
checked-in as it reads the metadata of the object in the check-in
operation and not as a separated step done manually. This also allow
several developers to work in parallel on the same database while
preventing them to accidently override each other code.
Use an impact analysis that utilize baselines as part of the
comparison to identify conflicts and identify if a difference (when
comparing the object's structure between the source control
repository and the database) is a real change that origin from
development or a difference that was origin from a different path
and then it should be skipped, such as different branch or an
emergency fix.
Use a solution that knows how to perform Impact Analysis for many
schemas at once, using UI or using API in order to eventually
automate the build & deploy process.
An article I wrote on this was published here, you are welcome to read it.
Can we use GIT as the source control for sql management studio?
for Database source control within SSMS
Agent SVN - SCC Subversion Plug-in.
http://www.zeusedit.com/agent/ssms/ms_ssms.html
or
http://www.red-gate.com/products/sql-development/sql-source-control/
I’ve found out that ApexSQL has a tool that natively supports Git as a source control system. It comes as a SSMS add-in, and offers a wizard you can use to map database objects with the source control systems. To do that:
Download and install ApexSQL Source Control
Start SSMS and in Object explorer select the database you want to be linked to a source control
Right-click the database, and form the context menu, select the Link database to source control option, from the ApexSQL Source Control submenu
Select the source control system (in your case it is Git) and choose from 2 database development models - shared or dedicated. Shared model is recommended when you link a database on which multiple developers will work at the same time
Filter objects which you don’t want to track using source control: by schema, type or name by schema, type or name
Provide appropriate login information and repository string. For Git it is:
<protocol>://<hostname>:<portnumber>/<Git server name>/<repository> (see example below):
More detailed step-by-step instructions can be found in this article:
http://knowledgebase.apexsql.com/link-database-source-control-system-2/
You could add Git Bash as an External Tool (Tools | External Tools...):
Name: &Git (use & to specify a hot key)
Command: C:\windows\SysWOW64\cmd.exe (32-bit Command shell)
Arguments: /c ""C:\path\to\Git\bin\sh.exe" --login -i" Finding the path where Git is installed on a Windows system
Initial directory: $(ItemDir)
Try sql-source-control, a free and open source CLI used to get SQL into source control systems like Git.
https://www.npmjs.com/package/sql-source-control
Microsoft has released SQL Operations Studio. It's a free tool that runs on Windows, macOS, and Linux, for managing SQL Server, Azure SQL Database, and Azure SQL Data Warehouse; wherever they're running. It comes with native GIT support.
Since SSMS is (more or less) a custom version of Visual Studio, you might be able to use a solution intended for VS:
Using Git with Visual Studio
Alternatively, manage your DB source code in Visual Studio from the beginning, not in SSMS. That way (at least in VS2010) you have database projects, integrated deployment and unit testing etc. Or continue using SSMS and check in your code from an external tool when you're ready (not so convenient, of course).
But it depends on exactly what you're doing: SSMS is a DBA tool, VS is a developer tool. Either way, you should be using some form of source control, but it's not clear from the question exactly what sort of files you need to version.
Not yet but if you go to http://redgate.uservoice.com/forums/39019-sql-source-control/suggestions/537681-add-git-support and vote for redgate to add support for git it might get added in the next version. Yes I know it's a commercial product but some products are just good enough to pay for!
Red Gate SQL Source Control has been updated to include Git and Mercurial support (as well as Perforce and TFS). Be forewarned that their DVCS integration is not 100% feature complete in relation to their SVN product, as basic features such as viewing history of an object are not supported from within SSMS. This may be a deal-breaker if other Red Gate tools like SQL Compare are part of your workflow.
Our workaround was to install TortoiseGit or GitExtensions and navigate to the repository on disk to drill into the specifics. It works but is a bit clunky.
VersionSQL is an SSMS source control add-in I've designed to be lightweight and easy to use. In the Object Explorer panel, just right-click on a database or object and click Commit. VersionSQL will script it out to Git/SVN in a neatly organized folder structure.
Check it out at https://www.versionsql.com
There have been a number of answers around this question which you might want to look at but in a nutshell ....
The nature of version control is to store the original file and then the deltas, the difference between the original file committed and and subsequent changes (ok i have made that a bit simpler than it is perhaps) and then to manage the version number and give tools to extract any particular revision you need. This also then allows you to compare previous revisions and roll back (etc. etc.)
The RDBMS is made up of the schema and the and data these change and can change frequently in the case of the data so even if you did VCS what would you compare to do a restore and how would that help? Assuming you have a live system then reverting to an earlier revision would lose all the data stored in the interim and although i have never tried it i suspect could destroy the general integrity of the RDBMS.
The better solution is to use a backup application built for that RDBMS, MySQLdump say in the case of MySQL which makes a snapshot of the data and the structure of the data and store that in a safe place.
Dumps can be scheduled regularly and you can do things like master/slave databases (or other strategies) so you can backup live production databases on the fly without impacting on performance
I have been working on a project and gotten it through the first stage. However, the requirments ended up changing and I have to add new tables and redo some of the foriegn key references in the DB.
The problem I have is my lack of knowledge of dealing with doing this kind of change to a staging then production database once I get the development done on dev database.
What are some strategies for migrating database schema changes and maintaining data in the database?
About as far as my knowledge is on doing this is open up Sql Server Management Studio and starting adding tables manually. I know this is probably a bad way to do it so looking for how to do it properly while realizing I probably started out wrong.
For maintaining schema changes you can use ApexSQL Diff, a SQL Server and SQL Azure schema comparison and synchronization tool, and for maintaining data in the database you can use ApexSQL Data Diff, a SQL Server and SQL Azure data comparison and synchronization tool.
Hope this helps
Disclaimer: I work for ApexSQL as a Support Engineer
You have to have something called as a "KIT". Obviously, if you are maintaining some kind of a source control, all the scripts for the changes that you do in the development environments should be maintained in the source control configuration tool.
Once you are done with all the scripts/changes that you deem certified to move to next higher environment. Prepare the kit with having all these scripts in folders (ideally categorized as Procedures, Tables, Functions, Bootstraps) And then have a batch files that could execute these scripts in the kit in a particular order using OSQL command line utility.
Have separate batch files for UAT/ Staging/ production so that you can just double click on the batch file to execute the kit in the appropriate server. Check for OSQL options.
This way all your environments are in sync!
I typically use something like the SQL Server Publishing Wizard to produce SQL scripts of the changes. That is a rather simple and easy approach. The major downside with that tool is that the produced will drop and recreate tables that are not changed but used by procedures that have changed (and I can't understand why), so there is some manual labour involved in going through the script and remove things that don't need to be there.
Note that you don't need to download and install this tool; you can launch it from within Visual Studio. Right-click on a connection in the Server Explorer and select "Publish to Provider" in the context menu.
Red Gate SQL Compare and SQL Data Compare all the way. Since my company bought it, it saved me tons of time staging our databases from DEV to TEST to ACCEPTANCE to PRODUCTION.
And you can have it synchronize with a scripts folder too for easy integration in a source control system.
http://www.red-gate.com
You might want to check out a tool like Liquibase: http://liquibase.org/
You can use visual studio 2015. Go to Tools=> SQL server => New Schema comparison
step 1) Select source and target Database.
Click on Compare option.
step 2) once comparison completed, you can click on icon Generate Script(Shift+alt+G)
this will generate Commit script.
step 3) To generate rollback script for database changes just swap database from step 1
There are some tools available to help you with that.
If you have Visual Studio Team edition, check database projects (aka DataDude aka Visual Studio Team for Database Professionals) See here and here
It allows you to generate a model from the dev/integration database and then (for many, but not all cases) automatically create scripts which update your prod database with the changes you made to dev/integration.
For VS 2008, make sure you get the GDR2 patches.
We have found the best way to push changes is to treat databases changes like code. All changes are in scripts, they are in source control and they are part of a version. Nothing is ever under any circumstances pushed to prod that is not scripted and in source control. That way you don't accidentally push changes that are in dev, but not yet ready to be pushed to prod. Further you can restore prod data to the dev box and rerun all the scripts not yet pushed and you have fresh data and all the dev work preserved. This also works great when you have lookup values to tables that are chaging that you don;t want pushed to prod until other things move as well. Script the insert and put it with the rest of the code for the version.
It's nice to use those tools to do a compare to see if something is missed in the scripts, but I would NEVER rely on them alone. Far too much risk of pushing something "not yet ready for prime time" to prod.
A good database design tool (such as Sybase Powerdesigner) will allow you to create the design changes to the data model, then generate the code to implement those changes. You can then store and run the code as you choose. This tool should also be able to do reverse engineering when you inherit a database you didn't build.
Finding all the changes between development and production is often difficult even in an organized, well-documented environment. Idera has a tool for SQL Server which will detect structural differences between your development and production database and another tool which detects changes in the data. In fact, I often use these to go the other direction and sync development with production to start a new project.
I'm early in development on a web application built in VS2008. I have both a desktop PC (where most of the work gets done) and a laptop (for occasional portability) on which I use AnkhSVN to keep the project code synced. What's the best way to keep my development database (SQL Server Express) synced up as well?
I have a VS database project in SVN containing create scripts which I re-generate when the schema changes. The original idea was to recreate the DB whenever something changed, but it's quickly becoming a pain. Also, I'd lose all the sample rows I entered to make sure data is being displayed properly.
I'm considering putting the .MDF and .LDF files under source control, but I doubt SQL Server Express will handle it gracefully if I do an SVN Update and the files get yanked out from under it, replaced with newer copies. Sticking a couple big binary files into source control doesn't seem like an elegant solution either, even if it is just a throwaway development database. Any suggestions?
There are obviously a number of ways to approach this, so I am going to list a number of links that should provide a better foundation to build on. These are the links that I've referenced in the past when trying to get others on the bandwagon.
Database Projects in Visual Studio .NET
Data Schema - How Changes are to be Implemented
Is Your Database Under Version Control?
Get Your Database Under Version Control
Also look for MSDN Webcast: Visual Studio 2005 Team Edition for Database Professionals (Part 4 of 4): Schema Source and Version Control
However, with all of that said, if you don't think that you are committed enough to implement some type of version control (either manual or semi-automated), then I HIGHLY recommend you check out the following:
Red Gate SQL Compare
Red Gate SQL Data Compare
Holy cow! Talk about making life easy! I had a project get away from me and had multiple people in making schema changes and had to keep multiple environments in sync. It was trivial to point the Red Gate products at two databases and see the differences and then sync them up.
In addition to your database CREATE script, why don't you maintain a default data or sample data script as well?
This is an approach that we've taken for incremental versions of an application we have been maintaining for more than 2 years now, and it works very well. Having a default data script also allows your QA testers to be able to recreate bugs using the data that you also have?
You might also want to take a look at a question I posted some time ago:
Best tool for auto-generating SQL change scripts
You can store backup (.bak file) of you database rather than .MDF & .LDF files.
You can restore your db easily using following script:
use master
go
if exists (select * from master.dbo.sysdatabases where name = 'your_db')
begin
alter database your_db set SINGLE_USER with rollback IMMEDIATE
drop database your_db
end
restore database your_db
from disk = 'path\to\your\bak\file'
with move 'Name of dat file' to 'path\to\mdf\file',
move 'Name of log file' to 'path\to\ldf\file'
go
You can put above mentioned script in text file restore.sql and call it from batch file using following command:
osql -E -i restore.sql
That way you can create script file to automate whole process:
Get latest db backup from SVN
repository or any suitable storage
Restore current db using bak file
We use a combo of, taking backups from higher environments down.
As well as using ApexSql to handle initial setup of schema.
Recently been using Subsonic migrations, as a coded, source controlled, run through CI way to get change scripts in, there is also "tarantino" project developed by headspring out of texas.
Most of these approaches especially the latter, are safe to use on top of most test data. I particularly like the automated last 2 because I can make a change, and next time someone gets latest, they just run the "updater" and they are ushered to latest.