I am using Redgare DLM Automation for database CI in a SQL Server and Visual Studio Team Services environment. I can easily deploy to multiple databases in a single environment, but apparently DLM Automation does not support multiple environments out of the box. Redgate support suggested using VSTS post-scripts in PowerShell, sqlcmd or something called "account_y" (I'm not sure what this refers to) to potentially add multiple environments.
Has anyone tried using DLM Automation for multiple environments? I have explored the PowerShell CmdLets, looked at SQL Compare options and filters, thought about using VSTS's Tokenizer for script alterations, but am still struggling with how to put all of this together to deploy to more than one environment.
Any experience or guidance would be greatly appreciated.
Thank you!
You definitely can deploy to multiple environments, however the issue of needing different user accounts for different environments is not a trivial problem to solve. Ultimately whatever you source control will be deployed to each environment, so if you need different user accounts then you will need to take care of it yourself by using some sort of post-deployment script.
I would suggest not source controlling user accounts and then adding a custom step after deployment to add the users - either command line using sqlcmd or the equivalent powershell cmdlets.
There are some blog posts that go into detail regarding this problem and their answers are probably more detailed than anything I can provide. I'd suggest that you have a read of them.
https://www.red-gate.com/blog/building/source-controlling-database-permissions
http://workingwithdevs.com/source-controlling-database-users/
I hope this helps.
Related
After an initial publish to a SQL Server(2019) and after the initial create of my DB project (data tier app) when publishing again it is failing with a drift report. No changes done to the database externally or within the VS project.
Drift report:
<Modifications>
<Object Name="[DummySqlLogin]" Parent="" Type="SqlUser" />
</Modifications>
To try and circumvent this by following a suggestion in answer on a question/answer from 2016.
"From your publish config file, use the following"
<ExcludeUsers>True</ExcludeUsers>
<ExcludeLogins>True</ExcludeLogins>
Unlike the OP this does allow me to publish to my database which now leaves me to the question of how to deal with logins/passwords especially in a scenario where we are going to be publishing to different environments.
I was planning on using SQLCMDVARIABLES to maintain separate Publish profiles for different types of environments and within there could specify passwords for each and then within the VS Database Project place a PreDeployment script which would setup Logins/Passwords for SQL accounts and make use of the SQLCMD variables.
Is there no better way of doing this? This probably works great for when you only have 5-10 environments but what if you have a 100?
Note* I want to avoid using commercial tools such as Redgate.
Here is one of common approaches:
environments, deploy targets are no project sources.
Same for specific logins, users, passwords. Those are security/administrations items server-specific. They depend on the environment, not product features. Why would the product, project sources depend on them? Imagine you have to change password due to security reasons on one of the servers. Can you logically connect it to recompiling all the sources?
I'd suggest to consider removing all the security items from SSDT project except maybe Roles (and configure publish.xml to ignore all the removed kinds of objects). Maintaining 100+ servers surely requires different tools and approaches, SSDT, dacpacs has nothing to do with it. The solution could be based on Octopus deploy, Ansible or something else.
I'm working on developing YAML build and deploy pipelines for dacpacs built from Visual Studio database projects , and one of the requirements is version numbers. In my research, I've found two ways we might want to implement it:
Extended Properties via the add- and updateextendedproperty stored procedures.
sysdac_instances via setting the DacVersion when the .dacpac is built and using the RegisterDataTierAppliction flag when the dacpac is deployed.
Is one of these methods preferred over the other? Am I trying to compare apples to oranges? Is there a third way to store database versions that I haven't found? Is there a DevOps or DBA best practice that I don't know and should be following?
I,
We are currently working on a .net core project that will use multiple databases with the same structure.
In short, this is a multi tenant project and each tenant will use the same web application (multiple instances behind a load balancer) BUT each tenant will have its own database.
We are searching for the best solution to ease our deployment process.
When the application (and DB) is updated, we need to update the database structure on all SQL servers and all databases (one SQL can contain x databases).
FYI, application and SQL server are hosted on AWS, our CI/CD is Azure DevOps.
And last (but not least) limitation: we are working on VSCode only (MAC & Linux laptop).
So, we looked for some solutions :
Using Database projects (.sqlproj) + DACPAC generation deployed using DevOps, but it's not available on VSCode
Using Migration: not working with multiple databases and dynamic connection strings
Using SQL script: too complicated to maintains by hand a SQL script that takes care of possible cases
So could someone give us some advice to solve this problem?
The general solution here is to generate SQL Scripts for each deployment, and integrate those into your CI/CD process.
You could use EF Migrations to generate a SQL Script, that is then tested, deployed to your repo as a first-class asset, and deployed by your CI/CD pipeline. Or you could use SSDT to manage the schema and generate change scripts. But those aren't the only reasonable ways.
If you are modifying the schema by hand without using SSDT, you would normally just use a tool to generate the change script. And go from there.
There are many tools (including SSDT) that help you to diff a development environment against a target production schema and generate the change scripts. Eg Redgate ReadyRoll
Note that if you intend to perform online schema updates you need to review the change scripts manually for offline DDL operations, and to ensure that your code/database changes have the correct forward and backward compatibility to support a rollout while the application is online.
And preparing, reviewing, testing, and editing the database change scripts is not something that everyone on the team dev needs to do. So you can always consider jumping onto a Windows VM for that task.
Off late I am searching the internet for an automated solution for SQL server database deployments primarily to avoid manual steps of maintaining and deploying sql scripts for deployments.
My situation is peculiar. It does not require me to maintain a whole database but 'only' customizations to the databases. Let me elaborate.
There are atleast 3 to 4 different SQL server databases that pertain to different 3rd party applications. I am tasked with writing integration between these systems(essentially between these many different databases in reality).
While considering to use SSDT(Sql server data tools) it advocates me to create a database project so I am not sure if it will exactly fit my need to maintain 'only' a subset of a database(I am assuming SSDT would want me to start with importing the entire database and then maintain it as I need.. which i do not want to do it this way).
If this can be achieved with SSDT, I am better off iwth that and appreciate if someone can point me in that direction to some how-tos. Otherwise, are there any other options or customized solutions that can help.?
I recently spent lot of time reading through this Simpletalk Article, but it doesn't seem to be a completely free solution. For instance, it uses an encrypted(.vbe) vb script which seems to be doing most of the tasks but the author(for some reason!) did not provide the source code for the vb script which stops me from considering..
I am not sure if I can build on top of this without having to worry about the vb script source code not available..
Overall, I am looking for a customized solution or a tool that can help me maintain changes to databases and automate deployment of my changes while adding mechanisms to maintain database versions.. Any help is much appreciated!!
TIA.
Found dbdeploy to be the tool that I can use for my task.
It has several falvors and different authorings though.
https://code.google.com/archive/p/dbdeploy/
DbDeploy.Net for use with .Net/SQL server environments.
adapted from http://sourceforge.net/projects/dbdeploy-net/ ...
... https://github.com/rakker91/dbdeploy.net
available as Nuget package
From another author: https://github.com/brunomlopes/dbdeploy.net
I'm working with the Redgate SQL Comparison SDK at the moment and have got it set up to nicely diff 2 databases.
What I would like to do now is be able to diff an sqlproj from source control with a destination database.
I have tried pulling the sql files using the tfs/vsts SDKs but to no avail.
Is there any way to either build a sqlproj from source control into a dacpac and then pull this in as a source database, or to directly pull the sqlproj in as a source?
Edit:
My ultimate goal with this is to be able to basically compare the version of the database that is in source control with the the database running across many different environments and create delpoyment scripts for the diffs.
I have another couple of Redgate tools that accomplish this (SQL compare & SQL Source), but these can only be installed on 1 (maybe 2 max?) devices, the difficulty I have is in using Amazon RDS (where the endpoints are unreachable outside the VPC), I cannot connect one central install of these tools to all of my environments, and I can't buy an additional license for every environment. So I was trying to use the Comparison SDK to attempt to "roll my own" middle ground.
Many Thanks,
I also work at Redgate, please do email me via dlm#red-gate.com if you want to go into more details into your specific questions and I'll set up a call for us.
In general the process that Redgate recommends for what you're doing below would be to keep the canonised schema that you want all the database to have in version control. You could get that schema in either by each developer using the SQL Source Control product to bring their changes in from SSMS as they develop them, or by using the SQL Compare product to put a version in at the end of a sprint.
You can then use our DLM Automation tools in conjunction with a CI server to automate creating difference reports and sync scripts for your target servers. DLM Automation is a set of PowerShell commandlets and plugins for common CI servers like TeamCity, Jenkins, VSTS, TFS etc. You could also use the SQL Compare Pro command line.
If your whole team have our SQL Toolbelt product then you're licensed to install the DLM Automation tools as many times as you like on build/release agents, so you don't need additional licences per environment.
Are you doing this in the context of an automation build/ci system? You mention VSTS, so the way this normally works is that this would have already pulled the files from source control. Once the files are in the build agent's working folder, you should be able to point the SDK (or SQL Compare command line) at this. Bear in mind that a sql proj isn't an officially supported data source for Redgate tools, although it will work in many instances.
It would be good if you could edit your question and give some background on the higher level problem you're trying to solve just in case we (I work for Redgate) can recommend a more suited set of tools or techniques.