SQLPackage Post Deployment Script not running - sql-server

I'm using Visual Studio 2013 with TFS, over SQL-Server 2012. Specifically I'm using SQL-Server Data Tools within VS for our SQL development, rather than within SQL-Server Management Studio.
In short, the Post-Deployment SQL script appears not to be executed by SQLPackage.
The specifics are: I have a VS SQL Database Project 'ADMIR', which has been developing over the last few weeks and gradually becoming more sophisticated both in terms of SQL and it's deployment. The database includes some dynamically generated tables and therefore I've introduced SQL Command Variables to reference two databases (one is the actual ADMIR database, the other a DB on the same Server). Therefore I've added Database References to dacpacs for these two DB's.
The ADMIR Project will Build fine.
There is also a Post-Deployment SQL script, and prior to the introduction of SQL Command Variables, this would also run fine and initialize various table data.
However, my understanding is that SQL Command Variables won't work with the VS/TFS Publish action, in that the Post-Deployment script is no longer being executed.
This is ok, as ultimately I want to script/automate the Publish/Deploy of the ADMIR Project and so I've moved to using SQLPackage to perform the Publish.
Again, this appears to work fine - My SQL Package batch command will execute and log errors/output (as applicable) and Publish my Build dacpac to the target Server successfully. However, it seems to completely ignore the Post-Deployment script. In addition, the output reports that the SQL Command Variables are not set.
These are the various scripts and configuration settings:
SQLPackage batch command:
::Publish the ADMIR database using the ADMIR.DEV.Publish.xml Profile.
"C:\Program Files (x86)\Microsoft SQL Server\120\DAC\bin\SqlPackage.exe"
/Action:Publish
/Profile:"C:\Source Control\ADMIR\ReleaseControl\ADMIR.DEV.publish.xml"
/SourceFile:"C:\Source Control\ADMIR\Build\ADMIR.dacpac"
2> "C:\Source Control\ADMIR\ReleaseControl\Publish_Error.txt"
> "C:\Source Control\ADMIR\ReleaseControl\Publish_Output.txt"
ADMIR.DEV.publish.xml Publishing Profile:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<IncludeCompositeObjects>True</IncludeCompositeObjects>
<TargetDatabaseName>ADMIR</TargetDatabaseName>
<DeployScriptFileName>ADMIR.sql</DeployScriptFileName>
<TargetConnectionString>Data Source=FOOBAR;Integrated Security=True;Pooling=False</TargetConnectionString>
<ProfileVersionNumber>1</ProfileVersionNumber>
</PropertyGroup>
<ItemGroup>
<SqlCmdVariable Include="ADMIRDatabaseName">
<Value>ADMIR</Value>
</SqlCmdVariable>
<SqlCmdVariable Include="XYZDatabaseName">
<Value>XYZDEV</Value>
</SqlCmdVariable>
</ItemGroup>
</Project>
Script.PostDeploy.sql (first few lines):
/* ADMIR Post-Deployment Script Template */
PRINT N'Post Deploy Script started:'+CAST(GETDATE() AS NVARCHAR(20))+N'.'
PRINT N'SQLCMD Variable $ADMIRDBName:'+[$(ADMIRDatabaseName)]+N'.'
PRINT N'SQLCMD Variable $XYZDBName:'+[$(XYZDatabaseName)]+N'.'
/* Purge existing data as we'll repopulate with what is required for a fresh deploy */
PRINT N'Purging existing table data...'
DELETE FROM [ADMIR].[Engine].[ProcessingStatus]
DELETE FROM [ADMIR].[Engine].[DataField]
DELETE FROM [ADMIR].[Engine].[DataKeyField]
...
...
When running a Build, I can see the ADMIR_Create.sql script does contain all the Post-Deploy statements. i.e. I can see the above lines of code in this file.
** ADMIR.sqlproj file contains the Post-Deploy scripts and the SQL Command Variables:**
<ItemGroup>
...
...
<PostDeploy Include="ReleaseControl\Script.PostDeploy.sql" />
</ItemGroup>
...
...
<ItemGroup>
<SqlCmdVariable Include="ADMIRDatabaseName">
<DefaultValue>ADMIR</DefaultValue>
<Value>$(SqlCmdVar__7)</Value>
</SqlCmdVariable>
<SqlCmdVariable Include="XYZDatabaseName">
<DefaultValue>XYZDEV</DefaultValue>
<Value>$(SqlCmdVar__6)</Value>
</SqlCmdVariable>
Publish log, Publish_Output.txt:
Publishing to database 'ADMIR' on server 'FOOBAR'.
Initializing deployment (Start)
*** The following SqlCmd variables are not defined in the target scripts: ADMIRDatabaseName XYZDatabaseName.
Initializing deployment (Complete)
Analyzing deployment plan (Start)
Analyzing deployment plan (Complete)
Updating database (Start)
Update complete.
Updating database (Complete)
Successfully published database.
So:
Why is the Post-Deployment script not being executed? I'm not seeing any logging information relating to it.
Why does the SQLPackage output log state that the SQL Command Variables are not defined, when they are in the Publish profile?
Possibly related to 2. Is the .sqlproj file correct in have the values of the SQL Command Variables as variables themselves e.g. $(SqlCmdVar__7)? where __6 & __7 coming from?
Thanks for any assistance!

I found that a new .sqlproj file created in Visual Studio 2017 did not include "default" Predeployment.sql or PostDeployment.sql files, so I added one manually.
Publishing did not execute the PostDeployment.sql because the file's Build Action was set to None. Changing the Build Action to PostDeploy fixed the issue.

As per my comment, the issue logged in MSDN Connect, appears to be cause. i.e. an SQLCMD variable with the same name as the dacpac for the database is resulting in the script not being executed. When I remove this reference, and change the related SQLCMD variables to literal values for the DB Name, then the project builds and the post-deploy script is executed as required.

Related

SQL Server Project Publish to a Docker Hosted Instance - SSDT and DefaultDataPath

I am having a VERY difficult time publishing a pre-existing SQL Server project to a Docker hosted instance of SQL Server.
What I am attempting to do is make a clean pipeline for a Docker hosted instance to use in testing a SQL Server project, which of course starts with doing it first by hand to understand all the steps involved. The SQL Server project itself has been around for many years, and has no problems deploying to SQL Server instances hosted on Windows boxes.
As near as I can tell, the issue comes while SSDT is generating the SQL Server deployment script itself. In a normal deployment to a Windows hosted SQL Server, the generated script starts out with some :setvar commands, including:
:setvar DefaultDataPath "C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\"
:setvar DefaultLogPath "C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\DATA\"
However, when publishing to a Docker hosted instance of SQL Server, and the same deployment process, the SQL script has:
:setvar DefaultDataPath ""
:setvar DefaultLogPath ""
The 1st thing this deployment does is to alter the database by adding in an additional data file, e.g.:
ALTER DATABASE [$(DatabaseName)]
ADD FILE (NAME = [ARCHIVE_274A259D], FILENAME = N'$(DefaultDataPath)$(DefaultFilePrefix)_ARCHIVE_274A259D.mdf') TO FILEGROUP [ARCHIVE];
The Docker based deployment then craps itself because the file path is (obviously) invalid.
In researching this problem, I've seen MANY solutions which hand-edit the generated deployment SQL script, and manually set the "proper" values for DefaultDataPath and DefaultLogPath ... and even one solution that ran the generated Sql through some sort of post-processor to make that same edit in a programmatic way with string replacement. This does work, but is less than optimal (especially in an automated build/test/deploy pipeline).
I've checked in the Docker instance itself, and its mssql.conf file does have defaults defined:
$ cat /var/opt/mssql/mssql.conf
[sqlagent]
enabled = false
[filelocation]
defaultdatadir = /var/opt/mssql/data/
defaultlogdir = /var/opt/mssql/log/
Can anybody shed light on why these are not being picked up by the SSDT process of generating the deploy script?
I spent a few days trying various workarounds to the problem ...
Defined the DATA and LOG directories in the Docker "run" command, but this had no effect on the gnerated Sql deploy script, e.g.: -e 'MSSQL_DATA_DIR=/var/opt/mssql/data/' -e 'MSSQL_LOG_DIR=/var/opt/mssql/log/'
Configure the Sql Project with SQLCMD Variables. This method could not override the DefaultDataPath or DefaultLogPath. I could add new Variables, but those would not affect the file path of the ALTER DATABASE command above.
Tried a Pre-Deployment script specifically tailored to override the values of DefaultDataPath and DefaultLogPath. While this technically CAN override the default values, the Pre-Deployment script is included in the generated Sql deployment script after the ALTER DATABASE commands to add data files. It would effectively work for the rest of the script, just not the specific portion that was throwing the error on initial deployment of the database.
At this point I feel there is either a Sql Server configuration option that I am simply unaware of, or possibly a flaw in SSDT which is preventing it from gathering the Default Path values from the Docker Sql Server instsance. Any ideas?

Azure DevOps - how to execute Pre and Post Deployment SQL scripts

In Azure DevOps release pipeline, How do I get our Script.PreDeployment.sql and Script.PostDeployment.sql files to execute during our SQL Server database deploy task?
In our release pipeline we have a Database Deployment phase, which has a SQL Server database deploy task. The tasks publishes our DACPAC file just fine. However I cannot figure out how to get the pre and post deployment scripts to execute.
Both of the scripts are included in the project, both have the appropriate BuildAction set to PreDeploy and PostDeploy. Yet in the logs of the dacpac deployment, there is no indication that the files were run - I have a bunch of PRINT statements in there.
I am also working on the post deployment scrip by using SSDT approach, and for deployment I am using the Azure SQL DacpacTask task in my Azure Pipeline, so you just need to create the post deployment scripts as you can see this in image and save it, after you run the Azure build this will add it in the Azure Pipeline artifact ill automatically executed in above azure tasks under release pipeline. First it will executed the database deployment and after that it runs the post deployment script. It works for me .
You can make use of Command line task to run those pre and post deployment SQL script using the SQLCMD tool.
The arguments to this its execution script are:
-S {database-server-name}.database.windows.net
-U {username}#{database-server-name}
-P {password}
-d {database-name}
-i {SQL file}
If you store the pre/postdeployment script in artifact, you can specify -i as like $(System.DefaultWorkingDirectory)/drop/Script.PreDeployment.sql.
Once Postdeployment script is added to the project, it will integrate with DacPac.
.sqlproj should have below PostDeploy Item group like -
<ItemGroup> <PostDeploy Include="PostDeploymentScript path"/> </ItemGroup>.
Sometime it does not update in .sqlproj and postdeployment does not work.
It should be added by default when postdeployment is added, just verify before publish.
.sqlproj

SSDT: incremental update script including data using sqlpackage.exe

I have an SSDT project with VS2015 and using SqlPackage.exe to generate deployment scripts and incremental update scripts. The database project contains post deployment scripts. here's how I generate the main script for DB installation:
sqlpackage.exe
/a:Script
/op:"database.sql"
/sf:"database.dacpac"
/tsn:"localhost"
/tdn:"MyDbName"
/p:CommentOutSetVarDeclarations="True"
/p:ScriptDatabaseCompatibility="True"
/p:IgnoreFileAndLogFilePath="True"
/p:IgnoreFilegroupPlacement="True"
/p:ScriptFileSize="True"
/p:PopulateFilesOnFilegroups="False"
This will output database.sql file which I run with sqlcmd.exe. the script file also contains all my post deployment scripts (Insert, Update, etc..)
Once I have a new version of the database, I create incremental update script from previously taken dacpac snapshot:
sqlpackage.exe
/a:Script
/op:"%migratePath%\%database%.!build!-%version%.sql"
/sf:"%snapshotPath%\%database%.%version%.dacpac"
/tf:"%snapshotPath%\%database%.!build!.dacpac"
/tdn:"MyDbName"
/p:CommentOutSetVarDeclarations="True"
/p:AllowIncompatiblePlatform="True"
/p:IgnorePermissions="True"
/p:IgnoreRoleMembership="True"
/p:ExcludeObjectTypes=Users;Credentials;Logins;Permissions;RoleMembership;ServerRoles;ServerRoleMembership;
this works fine, however, incremental update script does not contain metadata/reference data (e.g. e.g. list of countries, states, currencies, etc.).
Is there a way to tell SqlPackage to generate incremental script that contains the table reference data as well?

Overriding DefaultDataPath and DefaultLogPath variables when using SqlPackage to publishing a dacpac

The environment:
Using a SQL Server Database Tool in Visual Studio 2013, Update 4 to create a DacPac for publishing a database.
Using TFS build to build and drop the database project.
Using SqlPackage version to publish the dacpac to SQL Server 2012.
Using the SqlPackage Action "Publish", the script for deployment is generated using the following variables based on the target database's defaults:
:setvar DefaultDataPath "H:\YourServerDefault"
:setvar DefaultLogPath "H:\YourServerDefault"
I would like to override these locations to land in a specific pair of directories where we have some external encryption configured.
I have attempted to override these SQLCMD values in a couple ways:
Override on the command line:
SQLPackage /Action:Publish /SourceFile:./Db.dacpac /Profile:./MyProfile.publish.xml /Variables:DefaultDataPath=H:\MyNewLocation
Add an override in the publish profile (snippet below):
<ItemGroup>
<SqlCmdVariable Include="DefaultDataPath ">
<Value>H:\YourServerDefault</Value>
</SqlCmdVariable>
</ItemGroup>
While I can use either technique to override my own defined SQLCMD variables, the built in ones like DefaultDataPath and DefaultLogPath are ignored.
Is there a way to override these values when publishing with SqlPackage?
There is no built in support in SqlPackage but you can do this using a deployment contributor. Take a look at the DbLocationModifier sample on GitHub that solves exactly this issue. To actually use this, you need to:
Download the code from Github
Add a signing key to the project
Install to the correct extensions directory, or put it in the same folder as SqlPackage itself.
Pass additional arguments to SqlPackage specifying your contributor name and arguments - see the test code for how to do this via the API, the SqlPackage calls are essentially the same.
For more information about contributors and how to use them you can read the tutorial I wrote or read the walkthroughs on MSDN.
Disclosure: I work on the SQL Server tools team.

How to configure Visual Studio 2012 Database Project to Build deployment .sql file

Does anyone know How to configure Visual Studio 2012 Database Project to Build a deployment .sql file package.
I'm attempting to not have to use the .dacpac as my only deployment option.
e.g. generate a .sql file of all database schema alters.
If you're trying to do this automatically, you'll want to use the SQLPackage command to generate a script. If you want to do this within the IDE, publish the database and choose the option to generate a script instead of publishing the changes.
I'll usually build the project first with msbuild. That will generate a dacpac against which I can run the SQLPackage command.
My batch file looks something like this:
msbuild .\MyDB\MyDB.sqlproj /t:build /p:Configuration="Local"
sqlpackage /a:DeployReport /tsn:FTPROD /sf:.\MyDB\sql\Local\MyDB.dacpac /pr:.\MyDB\Publish\Production.publish.xml /op:.\Release\MyDB-Production.xml
sqlpackage /a:script /tsn:DBServer /sf:.\MyDB\sql\Local\MyDB.dacpac /pr:.\MyDB\Publish\Production.publish.xml /op:.\Release\MyDB-Production.sql
The first line builds the project. The second creates a deploy report so I can easily see what's going to be changed. The last generates the script. Your paths may vary so you'll need to tweak as appropriate for your environment. You'll need to be able to access the database against which the script will be run in order to generate the script.

Resources