I have updated the package in BIDS 2005 (I changed the backup routine to save to a different drive) and now I'm trying to get it back on the server (2005). I tried File > Save Copy As... Then ran the job that executes the package and it's still saving to the old drive, thus, my package didn't get saved.
In my opinion always create a deployment utility with your SSIS Project. This is configured under the Project Properties (see below). Once you have configured the project deployment utility, go to your project, find the "bin" folder and double-click the deployment utility. I will walk you through getting your package(s) onto the server really easily.
Good Luck!
The quick and dirty answer is to use dtutil
dtutil /file C:\Src\MyPackage.dtsx /destserver thatDatabase /COPY SQL;MyPackage
I too am a fan of the manifest files but, while probably overkill for your problem, I prefer to use tools that allow for unattended use. I combine the ssisdeploymanifest with a PowerShell script to handle all of SSIS deployments.
Powershell SSIS Deployment and maintenance
Related
I have made updates to approx. 100 ssis packages which are stored in ( msdb ), which is the location I need to deploy them.
I do not have access to rdp into the server and hence I cannot run multiple dtutil commands on the server thru a bat file. I am looking for a way to deploy all the packages in some easy manner, but looking at this image, my only option is one package at a time. Without having to change the current setup what option do I possibly have?
My understanding was incorrect with regard to usage of DTUTIL.
Also I was using an older version of DTUTIL and it was throwing this error - Description: The package failed to load due to error 0xC0011008 "Error loading from XML", which made me think I will have to run my scripts locally on the server.
This post helped me .I changed it to use the latest version of DTUTIL. Mine was at c:\...\150\DTS\Binn\DTUTIL.exe & it uploaded all the packages.
I have been doing file deployment and database deployment in sql server 2008.I have gone through the ssis(on sql 2014) and seen only catalogue based deployment.
Just wondering ,is there file deployment still available in ssis 2014.
can not find any file deployment option in deployment wizard.
There is an option called convert to package deployment.Does it mean the file deployment is only available under the package deployment option .
If i select the "Convert to package deployment" option,then the project deployment is not going to work in the future!
Also,Is there any concept of configuration in ssis 2014!
In ssis 2008 ,we have a file called configuration file and it contains all the
connection information.After the package deployment,we can change all the package connection information in this single file ,instead of opening and doing it in multiple files
There are two deployment models available to SSIS from the 2012 release going forward: Project Deployment Model (new, default) and Package Deployment Model (classic).
Which deployment model am I using?
The easiest way is to look at Solution Explorer. If you see a node under there specifying Project.params, then you are in the Project Deployment Model (left half of the image). Otherwise, you should see a "Data Sources" folder along with the project title's giant hint (package deployment model) (right half of the image)
Is File Deployment still a thing?
Package Deployment Model allows you to keep your process if you like your process. Happy deploying to the file system? Great, keep it up. Enjoy explicitly turning on logging and configuration - it's all yours. Just convert to the package deployment model and it's all there.
My DBA and I are trying to work out how to effectively use Microsoft's Database projects and the Dacpacs they generate to simplify our production deployment system.
Ideally, I would be able to build and/or publish the .sqlproj, generating a .dacpac file, which can then be uploaded to the production server and used to upgrade the database from whatever version it was to the latest version. This is similar to how we're doing website deployments, where I publish to a package, and then that package is uploaded to the server and imported into IIS.
However, we can't work out how to make this work. The DBA has already created the database and added it to our Availability Groups. And every time we try to apply the Dacpac, it tries to adjust settings which it can't because of the AGs.
Nothing I've been able to do has managed to create a .dacpac file which doesn't try to impose settings on the database. The closest option I've found will exclude them when publishing, but as best as I can tell you can't publish to an inaccessible database, and only the DBA has access to the production server.
Can I actually use dacpacs this way?
There are two parts to this, firstly how do you stop deploying settings you don't want to deploy - can you give an example of one of the settings that doesn't apply?
For the second part where you do not have access to the SQL Server there are a few different ways to handle this:
Use an offline copy to generate the deploy script
Get the DBA to generate the deploy script
Get the DBA to deploy using the dacpac
Get read only access to the database
Option 1: "Use an offline copy to generate the deploy script"
You need to compare the dacpac to something and if you do not have a TDS connection (default instance default port tcp:1433) then you can use a version of the database that matches production either through:
Use log shipping to restore a copy of production somewhere you can access it
Get a development db and production in sync, then every release goes to the dev and prod databases, ensuring that they stay in sync
The log shipped copy is the easiest, if it is to a development server you can normally have server permissions to give you acesss or you can create the correct permissions at the database level but not on the production server level.
If the data is sensitive then the log shipped copy might not be appropriate so you could try to keep a development and production database in sync but this is difficult and requires that the DBA be "well trained" into not running anything that isn't first run against the db database as well.
Once you have access to a database that has exactly the same schema as the production database you can use sqlpackage.exe /action:script to generate a deploy script, in fact because it isn't the production database you can generate the script as part of your CI process :).
Option 2: "Get the DBA to generate the deploy script"
This is to get the DBA to copy the dacpac to the productions server and to use sqlpackage.exe that will be in "Program Files (x86)\Microsoft Sql Server\Version\DAC\bin" folder to compare the dacpac to the database and generate a script that he can review before deploying.
Option 3: "Get the DBA to generate the deploy script"
This is simlar to option 2 but instead of generating a script he deploys in SSMS he just use sqlpackage.exe /Action:Publish to deploy the changes directly.
Option 4: "Get read only access to the database"
This is actually my preferred as it means that you always build scripts against what is guaranteed to be the state of production (as it is production). In your case you would need to get the tcp port between your machine or ideally your build machine and the SQL Server and then you will need these permissions:
https://the.agilesql.club/Blogs/Ed-Elliott/What-Permissions-Do-I-Need-To-Generate-A-Deploy-Script-With-SSDT
As I said option 4 is always my preferred but I understand that it isn't always possible.
Option 2 + 3 are fraught with worry as you will be running scripts that haven't been tested anywhere, with option 4 and 1 you can generate the scripts and then deploy to a test / QA database as long as they themselves have the same schema as production. The scripts can also go through a code review process.
If you do option 2 / 3 then I would create a batch file or powershell script that drives sqlpackage.exe and if they deploy from a different server that doens't have sqlpackage.exe then you can copy the DAC folder to that machine and run sqlpackage from that, you do not have to actually install it (you may need to also copy in the Microsoft.SqlServer.TransactSql.ScriptDom.dll from the "Program Files (x86)\Microsoft Sql Server\Version\SDK\Assemblies" folder.
I hope this helps, if you have any more questions feel free to post here or ping me :)
ed
We have several SSIS packages (in a solution) saved in a directory on the server. The packages are run via a SQL Server Agent job, with 'File system' as the Package source. Packages are not in any way added to 'Integration Services' (the thing you can connect with through the "Connect" button in SSMS). This seems to work fine.
Now I'm trying to understand the practical use and workings of the Integration Services' Stored Packages. It is probably there for a good reason - but I'm not getting it yet (reading several SSIS topics didn't help).
If I open Integration Services and add an existing package to the MSDB folder in Stored Packages, it is added to the msdb System Database of my SQL Server instance. I can then query some information on it from sysssispackages or sysssispackagefolders, and run the package directly.
If I add an existing package to the File System folder in Stored Packages, nothing really seems to happen, except that it also allows me to run it by right-clicking and choosing 'Run Package'.
I feel like I'm missing the important stuff here. My questions are as follows:
What reasons are there for using Integration Services to store
packages in addition to saving them as files on the server?
What exactly happens when you add a package to the 'File System' or
'MSDB' folders in the Integration Services 'Stored Packages' folder?
What's the advantage?
Any pointers/insights very much appreciated!
Here are a few advantages/disadvantages of database stored packages vs file system stored packages:
File System
OS-based file encryption and ACLs
Easier direct access for viewing or editing the package
Generally easier for the developer to manage (to change a package, just replace the file)
SQL Server
Easier access by multiple individuals
Benefits from database security, roles and Agent interaction
Packages get backed up with normal database backup processes
Generally easier for the DBA to manage (he will have the control of what happens with the packages in terms of changes, etc..)
In regards to you second question, a package is an XML file that SSIS can read and execute. On the file based deployment, the SSIS locates the package on your file-system and executes it, on the DB deployment the SSIS locates the package on an MSDB table and executes it. There are no differences in terms of performance whatsoever.
Backup. If packages are on the msdb database, when you backup the database, you backup your packages. Also they can take advantage of SQL Server security and they are not just laying around on a file system.
The advantage is that you dont have to worry about the file path, or that someone will delete the file or move the folder to another location. The package will always be on the the "same place".
Okay, so I misunderstood a lot about the SSIS Package Store and saving packages. Here is what I learned. First of all, 'the thing you can connect with through the "Connect" button in SSMS' (Connect -> Integration Services...) is called the SSIS Package Store.
What exactly happens when you add a package to the 'File System' or 'MSDB' folders in the Integration Services 'Stored Packages' folder? What's the advantage?
File System
If you want to work with the File System folder in the SSIS Package Store, save your packages to the default File System directory (...\Microsoft SQL Server\100\DTS\Packages) or change the root folder for File System to the directory you want to use. (You can change the root by changing the default value of <StorePath>..\Packages</StorePath> in the MsDtsSrvr.ini.xml file, which can be found in the ...\Microsoft SQL Server\100\DTS\Binn directory. Don't forget to restart the Integration Services service after you're done.) When you add a package to this directory, it will appear in the File System folder in the SSIS Package Store. You can then run the package directly from the SSIS Package Store or through an SQL Server Agent job (by choosing SSIS Package Store as the Package Source in the Job Step Properties and then selecting the package).
Editing packages is easy: open the package in the File System directory, edit and save, and the new version will be instantly available through the SSIS Package Store.
Advantages:
Deployment and troubleshooting of packages is easy
Packages are still available when the database engine is down
SQL Server / MSDB
If you want to rely on the msdb database to save your packages, you have to import each package into msdb via the SSIS Package Store. Right-click on the MSDB folder and choose Import Package. This will save the package to the msdb database. You do not need to save the original .dtsx package files afterwards.
Editing packages is a little harder: you have to export the package, edit it and import the package again in SSIS Package Store. Or you can open a new project in BIDS, add the package by right-clicking SSIS Packages and choosing Add Existing Package from SQL Server, edit it and then import the package again in SSIS Package Store.
Advantages:
Security of packages can be tightly configured through database security
Packages will be backed up as the msdb database is backed up
Packages are stored in a central place
What reasons are there for using Integration Services to store packages in addition to saving them as files on the server?
So why would you add a package to the SSIS Package Store and not just run it like we did, by directly referencing to a package.dtsx file from the Job Step Properties window)? It depends: if you want your packages in the msdb database, you need the Package Store, because there is no other way to maintain your packages. If you use the File System, might be that you have a separate 'Development' and 'Deployment' directory and all packages that are ready for deployment can be found through the SSIS Package Store. In each case, the SSIS Package Store provides an easy interface to your packages.
Thanks to João Leal and Diego for your answers!
This one explain the differences better
social.msdn.microsoft.com
File System:
I'm sure you know exactly what the FileSystem option is, but to
complete this section: you can save the package on a physical location
on hard drive or any shared folder with this option, and you should
provide a full qualified path to stored package in the FileSystem
option.
SQL Server:
with this option, SSIS packages will be stored in the msdb database,
in the sysssispackages table. you can separate them in any folder you
want. these folders are virtual folders, which listed in the
sysssispackagefolders table. so with this option ssis package will not
store as physical file. it will store in the msdb database.
SSIS Package Store:
this option is not actually a real option for deployment. why? because
this is a redirection step only. what I mean is that if you choose
this option in the BIDS when you try to save a copy of package ... ,
you will see a tree with a parent "SSIS packages" and two child :
"MSDB" and "File System". if you choose the "MSDB" the package will
save under msdb database same as the sql server option ( previous
option which described ).if you choose the "File System", the ssis
package will store as a physical file with .dtsx extension but in a
folder in this address: \Microsoft SQL
Server\100\DTS\Packages . this is like file system option but you can
not store package at any location , you can store it under a
predefined physical location and of course you can create subfolders
there, but you can not cross the parent to save package any other
place than predefined.
I'm just trying to understand SSIS packages a bit better and how they are deployed. Correct me I'm wrong but for any deployment, I believe there needs to be at least two files a .SSISDeploymentManifest and a .dtsx. The .SSISDeploymentManifest acts as the equivalent windows installer package which points to the .dtsx. The dtsx is the actual package of "stuff" that is referenced as an external file some how when you run the installer. When you install it, the package gets added to a list of ssis packages for that instance.
My further questions:
If i wanted to keep previous version of the same package, can I just copy the bin directories with the two above files and keep separately should I need to roll back to a previous package?
Where are these packages installed to? How does SSIS know where the packagess are?
Correct me I'm wrong but for any deployment, I believe there needs to
be at least two files a .SSISDeploymentManifest and a .dtsx. The
.SSISDeploymentManifest acts as the equivalent windows installer
package which points to the .dtsx. The dtsx is the actual package of
"stuff" that is referenced as an external file some how when you run
the installer. When you install it, the package gets added to a list
of ssis packages for that instance.
Your assumptions are mostly correct. You don't need the deployment manifest, but it can be handy. Also, you don't need to deploy to the SQL Server instance. You have the option to deploy to the file system as well. I'll explain both below.
Regarding your 1st question:
Version Control:
Make sure you're developing and checking in your dtsx packages via visual studio. Label your releases in sourcesafe or whatever version control you're using. If you are checking in and labeling, then you should be able to easily roll back to a previous version. As you mention, you also can just save a copy of your old bin directory but naturally put them in dated subfolders or something. However, this does not take the place of proper version control.
Regarding your 2nd question:
Deployment:
As the other poster states, you first have a decision to make:
a) Deploy packages to the file system
b) Deploy packages to MSDB
There are benefits to each, and everyone has their preference. I have used both, but I prefer the filesystem because it's more transparent, however there is more to maintain.
See this post for much more on this: http://blogs.conchango.com/jamiethomson/archive/2006/01/05/SSIS_3A00_-Common-folder-structure.aspx
The code is in the dtsx package. Generally,in order to make your packages portable you also abstract your connection strings and other configurable information into a config file (.dtsconfig) or environment variable (no file needed). See BOL to learn more about configuration.
The manifest file contains metadata about which dtsx and config files to install. If you open one, you'll see it's a simple readable xml file.
The manifest file makes it easy to hand over to a DBA to deploy (ask them to double-click the manifest file and follow directions, but they'll need instructions.
To me, the manifest file is more useful for deploying to SQL Server than to the file system. Really, all it does is make a copy of the dtsx and config files and puts them where you tell it. You could just as easily instruct the DBA to copy your dtsx files to a common folder on the server, and the config files to another folder on the same server.
Then when you schedule your jobs using SQL Agent, you specify that you're going to run an SSIS package that is stored on the file system and browse to where it's located. If you're using configurations, then there's a tab to specify where the config file is located.
There is so much to know about configuring/deployment/versioning of SSIS packages. But hopefully this will get you started on the right path.
When you export your DTS packages using the Import/Export Wizard in SQL Server you have the option of saving them to SQL Server or locally on the file system.
Regarding the versions of your SSIS packages, you need to query SSISDB to extract the version numbers. It's annoying this kind of info isn't shown directly in the Management Studio but, until it is, someone may find this useful:
SELECT prj.[name] as Project
,pkg.[name] as Package
,pkg.[version_major]
,pkg.[version_minor]
,pkg.[version_build]
FROM [SSISDB].[internal].[packages] as pkg
JOIN [SSISDB].[internal].[projects] as prj
ON pkg.[project_id] = prj.[project_id]
ORDER BY prj.[name]