I have a SQL server database project that I am building using Visual Studio. The project contains 4 DACPAC references. The DACPAC files are currently part of the github repository, so could in theory become stale as time goes on.
I'd like to avoid the references becoming stale and not have to store the files in the repo.
Therefore, my question consists of 2 parts:
I think I can generate the referenced Dacpacs when developing locally by issuing a pre-build set of commands such as SQlPackage - can anybody think of other approaches that may work?
Am I able to do something similar in Azure Devops as a pre-build step, or as part of the build process?
Related
I am currently working on a continuous integration project to auto build and deploy database changes to target environment.
We are using Perforce P4 for source code repository, Nexus for artefacts repository and MS SQL 2008.
We are not using Redgate for the database repository.
Check-in process
- Developers manually extract database objects (e.g. table, stored proc, function) using Management Studio and check-in to the source repository of Perforce.
Requirement:
As part of the CI process, when developers check-in their code to the source repository, the build process should get triggered and create artefacts of checked-in code and get copied to the artefacts repository.
The deployment process should get automatically triggered when it finds any new artefacts and deploy the artefact to the target environment.
I would highly appreciate if someone helps me to know :
build and deployment steps
requirement of manifest file
if it is possible to extract incremental changes
Get ssdt in visual studio (express works if you don't have licenses)
This will mean your developers check in create statements and you deploy incremental changes, it is pretty simple to setup just have a build step call sqlpackage.exe to deploy or generate scripts
We are using TFS (service, not server) to manage the versions of our database schema. Periodically TFS generates a build and saves it in a drop folder.
Following best practices (http://martinfowler.com/articles/evodb.html), recommendations, and above all, based in our own experience (and suffering), we want to generate automatically a daily/weekly diff script with the changes checked in.
There are a lot of tools (RedGate, Visual Studio, Open Sourece) that help to do this job, and I've tried all. But in all the cases needs to be done manually.
We are doing as often as we can… but since it is manually it is not as often as it should be necessary ;)
Is there a way to do it automatically (unattended)? Can be done between 2 deployment scripts of builds? Or to do it between 2 databases? Is it also possible to compare data automatically too?
Yes, it is possible to automatically generate the difference script. We have done that in my company.
We're using vsdbcmd command line tool from Visual Studio to generate the deployment script from the build and later use that script to deploy in the test servers. We do that using Visual Studio 2010.
I am trying to create a database solution in Visual Studio 2012.
I have 5 databases that all exist on the same server, so I have imported each of them into a database project.
Unfortunately there are quite a few objects in the databases that reference objects in other databases, so I have been trying to resolve these by adding a Database Reference to create a Database variable to refer to these cross database instances.
The problem is that I have database A referencing B and also B referencing A, and it won't allow me to add these two references. I get the message box saying "Adding this project as a reference would cause a circular dependency".
Any ideas on approaches to resolving this? I guess one way might be to create a solution for each database, but I would rather not if there is a better way to do it.
We are trying to set up an automated build, so I really need the database projects to compile.
You'll want to extract your database schemas into dacpac files (assuming that you're using SSDT SQLProj files and not the old DBProj files for your projects). You can do this through SSMS or through the SQLPackage command line. Once extracted, put them in some place that all of the projects can hit (and preferably that's still under source control so everyone else can reference it). Add those dacpac files to your projects as database references, probably without the option to use a variable for the DB Name.
I've written up adding database reference on my blog here:
http://schottsql.blogspot.com/2012/10/ssdt-external-database-references.html
One additional note - if you're building new databases from these projects, you'll likely need to go through several passes. Turn off the option to run as a transaction and to fail when there's an error so some objects will be created, repeat as needed until all objects are created. I used a variable in my projects called "DeployType" and set up my pre and post deploy scripts to handle a DeployType of "New" differently so it wouldn't attempt to populate/update any data for "New" builds.
I have Git Source Control Provider setup and running well.
For Visual Studio projects, that is.
The problem is that each such project is tightly tied to a SQL Server database.
I know how to version control a database in its own .git repository but this is neither convenient nor truly robust because ideally I would want the same ADD, COMMIT, TAG and BRANCH commands to operate on both directory trees simultaneously, in a synchronized manner.
Is there a way to Git SQL Server database with Visual Studio's Git Source Control Provider in the manner I described?
You can install the SQL Server Data Tools if you want to, but you don't have to: You can Use the Database Publishing Wizard to script your table data right from Visual Studio into the solution's folder, then Git it just like you do with the other project files in that folder.
You can store your database schema as Visual studio project using SQL Server Data Tools and then version control this project using Git.
Being in the database version control space for 5 years (as director of product management at DBmaestro) and having worked as a DBA for over two decades, I can tell you the simple fact that you cannot treat the database objects as you treat your Java, C# or other files and save the changes in simple DDL scripts.
There are many reasons and I'll name a few:
Files are stored locally on the developer’s PC and the change s/he
makes do not affect other developers. Likewise, the developer is not
affected by changes made by her colleague. In database this is
(usually) not the case and developers share the same database
environment, so any change that were committed to the database affect
others.
Publishing code changes is done using the Check-In / Submit Changes /
etc. (depending on which source control tool you use). At that point,
the code from the local directory of the developer is inserted into
the source control repository. Developer who wants to get the latest
code need to request it from the source control tool. In database the
change already exists and impacts other data even if it was not
checked-in into the repository.
During the file check-in, the source control tool performs a conflict
check to see if the same file was modified and checked-in by another
developer during the time you modified your local copy. Again there
is no check for this in the database. If you alter a procedure from
your local PC and at the same time I modify the same procedure with
code form my local PC then we override each other’s changes.
The build process of code is done by getting the label / latest
version of the code to an empty directory and then perform a build –
compile. The output are binaries in which we copy & replace the
existing. We don't care what was before. In database we cannot
recreate the database as we need to maintain the data! Also the
deployment executes SQL scripts which were generated in the build
process.
When executing the SQL scripts (with the DDL, DCL, DML (for static
content) commands) you assume the current structure of the
environment match the structure when you create the scripts. If not,
then your scripts can fail as you are trying to add new column which
already exists.
Treating SQL scripts as code and manually generating them will cause
syntax errors, database dependencies errors, scripts that are not
reusable which complicate the task of developing, maintaining,
testing those scripts. In addition, those scripts may run on an
environment which is different from the one you though it would run
on.
Sometimes the script in the version control repository does not match
the structure of the object that was tested and then errors will
happen in production!
There are many more, but I think you got the picture.
What I found that works is the following:
Use an enforced version control system that enforces
check-out/check-in operations on the database objects. This will
make sure the version control repository matches the code that was
checked-in as it reads the metadata of the object in the check-in
operation and not as a separated step done manually. This also allow
several developers to work in parallel on the same database while
preventing them to accidently override each other code.
Use an impact analysis that utilize baselines as part of the
comparison to identify conflicts and identify if a difference (when
comparing the object's structure between the source control
repository and the database) is a real change that origin from
development or a difference that was origin from a different path
and then it should be skipped, such as different branch or an
emergency fix.
Use a solution that knows how to perform Impact Analysis for many
schemas at once, using UI or using API in order to eventually
automate the build & deploy process.
An article I wrote on this was published here, you are welcome to read it.
I am trying to add some stored procedures to a build process in visual studio (using MSBuild) with a database project (*.dbp).
However unlike some of the other database projects available in VS, it cannot be built or run. ( I believe this is the project file i want as I am using SQL2000)
How can I get the SQL scripts to run as I build the project?
I know that I can run the scripts by right clicking and pressing run....but that is not the question.
Database projects are not built like normal projects. If you want to execute scripts inside this project, my advice would be to extend the associated project(s) which are MSBuild compatible (csproj, vbproj, etc.) and reference/execute your scripts from there using your tool of choice.
How you implement it is up to you, for example, you could hook into BeforeBuild target and execute scripts using SqlExecute task.
FWIW, we have to support SQL Server 2000, 2005, 2008+, so the task becomes much harder; think NVarchar(max) (2005+) vs NText (2000+) plus all the sp renames, etc between versions. We also wanted a single script which developers, QA and installer used to ensure consistency. So our approach was a custom tool which allowed token/keyword replacements inside scripts, which provides flexibility between different SQL Server versions and customers installations (e.g. user/login details, db names), as well as integration into various technologies so scripts can run from command line, MSBuild, MSI custom action using Wix projects, or whatever we need them to run from.
Database projects can be Built.
They cannot be run - they can be Deployed.
When you deploy, the project builds and then deploys. The code will be deployed along with the Pre-Build and Post-Build scripts and the Pre-Deployment and Post-Deployment scripts.
So if you want scripts to run as you BUILD the project, you want to look into the Pre-Build and Post-Build under Build Events in the Project properties.