There are two distinct targets, Deploy and Publish, in SSDT project alongside Build but I can't find anything on what's the difference between them and which one should be used when? From what I can gather .dbproj used to use Deploy but .sqlproj supposedly replaced it with Publish, how come it's still there and what is it still used for?
$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v$(VisualStudioVersion)\SSDT\Microsoft.Data.Tools.Schema.SqlTasks.targets
<UsingTask TaskName="SqlBuildTask" AssemblyFile="$(SqlServerRedistPath)\Microsoft.Data.Tools.Schema.Tasks.Sql.12.dll" />
<UsingTask TaskName="SqlDeployTask" AssemblyFile="$(SqlServerRedistPath)\Microsoft.Data.Tools.Schema.Tasks.Sql.12.dll" />
<UsingTask TaskName="SqlPublishTask" AssemblyFile="$(SqlServerRedistPath)\Microsoft.Data.Tools.Schema.Tasks.Sql.12.dll" />
<DeployDependsOn>
BeforeDeploy;
PreDeployEvent;
SqlDeploy;
PostDeployEvent;
AfterDeploy
</DeployDependsOn>
<Target Name="Deploy" DependsOnTargets="$(DeployDependsOn)">
<PublishDependsOn>
BeforePublish;
PrePublishEvent;
SqlPublish;
PostPublishEvent;
AfterPublish
</PublishDependsOn>
<Target Name="Publish" DependsOnTargets="$(PublishDependsOn)">
The main difference is that Publish is used when you have a publish profile file describing the connection information, whereas Deploy is used when you do not have this file / do not wish to use one. Both are used by SSDT, the Deploy task is not just there for backward compatibility.
The Publish task takes in a "PublishProfile" property that specifies a saved publish profile xml file. The required server name, database name and other properties needed are read from that file. See this forum post for some more information.
The deploy task is used when you do not have a publish profile (for example, the F5 debug deployment uses this, if I recall correctly). I believe this requires that the server, database name and other properties be set explicitly.
I'm assuming it's for backwards compatibility. Upgrading a SQL2008 dbproj to SQL2012 sqlproj after installing the SSDT update, you'd probably still want your dbprojects to function correctly until they are converted.
Related
I have a SQL Server database project (.sqlproj) which I am using as part of a CI/CD pipeline to deploy database changes. I would like to deploy the same code to two databases (Dev and Production) but each with a slightly different configuration:
In Dev, I have an Azure AD group Database-Dev-Developers:
CREATE USER [Database-Dev-Developers] FOR EXTERNAL PROVIDER;
In Production, I have an Azure AD group Database-Prod-Developers:
CREATE USER [Database-Prod-Developers] FOR EXTERNAL PROVIDER;
I can find no way to alter which scripts are build/published based on the configuration. Ideally I'd like to be able to specify the project configuration at build time (Debug/Release), which changes the output.
I have tried adding conditional expressions for the relevant files in the .sqlproj file, but this has no effect:
Condition=" '$(Configuration)' == 'Debug' "
You should look into using a Token Replacement step in your pipeline. You can add different variable values for Dev vs Prod to replace the tokens with. Then you just need one tokenized configuration file that can be used for both Dev and Prod.
I'm not exactly sure how kosher it is to use tokens in a .sqlproj file, it depends on what configurations you're trying to replace. But I've seen it used very successfully on ...config.json files in modern .NET Core based projects.
Another thing you can look into is File Transformations. I don't have any experience using these though.
I have found a partial solution to this problem. One can create a publish profile, which contains instructions to ignore certain object types. See this helpful blog post which details the process, summarised below:
In Visual Studio, right-click SSDT project
Publish -> Advanced
Select the 'drop' tab, and check 'Drop objects in target but not in source'
Check 'Do not drop...' next to the object types you wish to ignore. For me this was 'Do not drop users' and 'Do not drop roles'
Save the publish profile
Extra step for Azure DevOps Azure SQL Database deployment task, specify the generated publish profile xml file in the 'Publish Profile' setting.
This has the drawback that non-sensitive security settings (such as role membership) cannot be deployed, but this was a trade-off I was willing to make in my situation.
I want to use database project for script deployment in Azure SQL Server, I don't want to import full database. I just want to use database project for delta script. I added a project and included one script file with none as build action that contains create table statement , I am publishing the project, It's completing successfully but create statement is not executing. What is wrong here? Is there any other way to do this?
TLDR: Set your build action to "Post Deployment Script".
Longer:
What happens in SSDT is that all the files that have a build action of "Build" are built into a model of what the database should look like. When the deploy happens that model is compared to the target database and if there are any changes, a change script it generated and then optionally deployed.
If you have any file marked pre or post deployment script then they are either prepended or appended to the change script and will be run as part of the deployment.
If you have any files with a build action of "None" then SSDT ignores them, you could put anything in there, even an ascii picture of a donkey and the project will still build and deploy (obviously your ascii donkey won't get deployed anywhere).
If you just want to use SSDT to do your deployments you can just set the build action to pre or post deploy and it will be included. This is pretty odd though, either don't use SSDT or use SSDT and put the model of your entire database in there.
Personally, I would use SSDT properly and live the dream.
Ed
I have 3 2008-style SSIS packages that I think I've done a pretty good job upgrading to the 2016 tooling. I've migrated to Project Deployment at the top level and I'm using project params - it all seems like a big improvement.
My first problem is that when I deploy to the server, it seems to succeed, but the Integration Services explorer mode in SSMS shows no packages in the place I expect to see them. The new folder is there but there's nothing in it. I was able to use 7zip to uncompress the .ispac file in the /bin folder which is being deployed and it does indeed contain the .dtsx files that I expect to see.
When I deploy, the deployment wizard lists the .ispac file Path under Source section, but not the individual packages. That's probably fine but I'll mention it in case I should see the individual packages listed.
I also notice that there's an option to "Convert Deployment Model" under the Visual Studio project's SSIS Packages section - that's separate from the "Convert to Package/Project Deployment" at the project-level. It's also separate from the "Upgrade All Packages" option that has already been done and for which there are no remaining upgradable packages.
When I run the "Convert Deployment Model" wizard and try to "Next" past the screen where the packages are listed as "Not loaded" Status, I get an error that "One or more selected packages are not ready" an an Error status on all packages with the message that The variable "$Project::ServerB" was not found in the Variables collection. The variable might not exist in the correct scope.
#[$Project::ServerB] is indeed a variable in all of the packages. And in at least one of the xml content of the package files I can see it listed in just the one place. In the editor (the Expression field of the SQL Connection Manager) where we use the variable this project parameter evaluates to the configured value just fine.
What is this "Convert Deployment Model" option anyway, separate from the "Convert to Package/Project Deployment" option? Are they the same, and the one on the "SSIS Package" folder just failing to validate the conversion (back to Package Deployment) because there are project parameters that the resulting Package Deployment mode doesn't support, hence the error?
And most importantly, why aren't my packages actually getting deployed? Is this deployment model thing just a red herring at this point? What should I be seeing?
Thanks!
I was looking in the wrong spot. In SSMS, I see them under the Database Engine side of the Object Explorer, but not under the Integration Services side. Integration Services had a folder that might have been carried over from the old deployments and seems no longer necessary. Under the database there is a new "Integration Services Catalogs" folder that now shows the projects along with the expected deployed packages.
I guess the "Convert Deployment Model" was just a distraction.
I have a simple WPF application that uses ClickOnce to handle installing. Within this application is a compact database. Through testing I have found that when I publish a new build this database will get overwritten, which is not what I want. Is there anyway I can have fine grained control over what files are updated? I assume ClickOnce is simply checking the hash of the database file, deciding that it has changed and pulling the update.
As a workaround I have since removed the database from the files that are included with the published application so the original remains on the client machine after an update, untouched.
Not a great solution I know
Thanks,
ClickOnce deployments segregate the Application Files into "Include" or "Data file". You can specify what each file is in visual Studio by going to the project Properties page, Publish tab, then clicking the "Application Files..." button. You can then set your .sdf file to "Data File" under the Publish Status column.
Data Files that are downloaded with a ClickOnce application are then placed in a separate directory for each new version.
The idea is that on the first run of the new application version, you go retrieve all the user's private data from their old-version data files and incorporate that data into the new data files which have just been downloaded with your new version.
I think you'll find the information you need at Accessing Local and Remote Data in ClickOnce Applications. Specifically, look at the sections "ClickOnce Data Directory" and "Data Directory and Application Versions."
To access a SQL Server CE database located in your Data directory, use a connection string similar to the following:
<add
name="MyApplication.Properties.Settings.LocalCacheConnectionString"
connectionString="Data Source=|DataDirectory|\LocalCache.sdf"
providerName="Microsoft.SqlServerCe.Client.3.5" />
The "|DataDirectory|" is a special syntax supported by SQL CE and SQL Express and resolves at runtime to the proper directory.
If you so much as open that SQLCE database included in your project, it will change the time stamp on the database, and ClickOnce will deploy it and put the old version under the \pre subfolder.
You might want to consider this method for handling this. Then if you accidentally deploy a new version of the database and don't realize it, you're not hosed. If you intentionally make changes, you can change the database structure of your current database with SQL queries, and pull data from the new copy deployed to the Data Directory (that you're otherwise ignoring) when you need to.
RobinDotNet
I'm just trying to understand SSIS packages a bit better and how they are deployed. Correct me I'm wrong but for any deployment, I believe there needs to be at least two files a .SSISDeploymentManifest and a .dtsx. The .SSISDeploymentManifest acts as the equivalent windows installer package which points to the .dtsx. The dtsx is the actual package of "stuff" that is referenced as an external file some how when you run the installer. When you install it, the package gets added to a list of ssis packages for that instance.
My further questions:
If i wanted to keep previous version of the same package, can I just copy the bin directories with the two above files and keep separately should I need to roll back to a previous package?
Where are these packages installed to? How does SSIS know where the packagess are?
Correct me I'm wrong but for any deployment, I believe there needs to
be at least two files a .SSISDeploymentManifest and a .dtsx. The
.SSISDeploymentManifest acts as the equivalent windows installer
package which points to the .dtsx. The dtsx is the actual package of
"stuff" that is referenced as an external file some how when you run
the installer. When you install it, the package gets added to a list
of ssis packages for that instance.
Your assumptions are mostly correct. You don't need the deployment manifest, but it can be handy. Also, you don't need to deploy to the SQL Server instance. You have the option to deploy to the file system as well. I'll explain both below.
Regarding your 1st question:
Version Control:
Make sure you're developing and checking in your dtsx packages via visual studio. Label your releases in sourcesafe or whatever version control you're using. If you are checking in and labeling, then you should be able to easily roll back to a previous version. As you mention, you also can just save a copy of your old bin directory but naturally put them in dated subfolders or something. However, this does not take the place of proper version control.
Regarding your 2nd question:
Deployment:
As the other poster states, you first have a decision to make:
a) Deploy packages to the file system
b) Deploy packages to MSDB
There are benefits to each, and everyone has their preference. I have used both, but I prefer the filesystem because it's more transparent, however there is more to maintain.
See this post for much more on this: http://blogs.conchango.com/jamiethomson/archive/2006/01/05/SSIS_3A00_-Common-folder-structure.aspx
The code is in the dtsx package. Generally,in order to make your packages portable you also abstract your connection strings and other configurable information into a config file (.dtsconfig) or environment variable (no file needed). See BOL to learn more about configuration.
The manifest file contains metadata about which dtsx and config files to install. If you open one, you'll see it's a simple readable xml file.
The manifest file makes it easy to hand over to a DBA to deploy (ask them to double-click the manifest file and follow directions, but they'll need instructions.
To me, the manifest file is more useful for deploying to SQL Server than to the file system. Really, all it does is make a copy of the dtsx and config files and puts them where you tell it. You could just as easily instruct the DBA to copy your dtsx files to a common folder on the server, and the config files to another folder on the same server.
Then when you schedule your jobs using SQL Agent, you specify that you're going to run an SSIS package that is stored on the file system and browse to where it's located. If you're using configurations, then there's a tab to specify where the config file is located.
There is so much to know about configuring/deployment/versioning of SSIS packages. But hopefully this will get you started on the right path.
When you export your DTS packages using the Import/Export Wizard in SQL Server you have the option of saving them to SQL Server or locally on the file system.
Regarding the versions of your SSIS packages, you need to query SSISDB to extract the version numbers. It's annoying this kind of info isn't shown directly in the Management Studio but, until it is, someone may find this useful:
SELECT prj.[name] as Project
,pkg.[name] as Package
,pkg.[version_major]
,pkg.[version_minor]
,pkg.[version_build]
FROM [SSISDB].[internal].[packages] as pkg
JOIN [SSISDB].[internal].[projects] as prj
ON pkg.[project_id] = prj.[project_id]
ORDER BY prj.[name]