I'm trying to replace a scheduled job which runs a .dtsx file with the same .dtsx file, modified.
I want to use the same .dtsxonfig file as the old package, but I don't know where it is located. How can I find the location of the config file? How can I ensure the new package will reference it?
If the current sql agent job provides an explicit path for the configuration, the job step will list that (grab the details from the Command Line tab).
Otherwise, look inside the package. SSIS, Package Configurations... and look for a Configuration Type of XML configuration file
Update
In your control flow, right click on the background and in the context sensitive menu, select Package Configurations... If they have turned on configurations, I would expect the "Enable package configurations" check box to be checked and the section below that to not be greyed out.
With XML configuration, you get two options with it. The first (Sample) is what is referred to as direct configuration. You can see the Configuration String specifies a specific location on disk that the package will examine when the package runs. The second (XmlEnv) option demonstrates indirect configuration. This works by using an environment variable, SSIS_CONFIG in my example, to determine the location of the .dtsConfig file
If you don't see anything listed in the Package Configurations section, then you can assume they are applying configuration at run-time. There are a few different ways of accomplishing this so I would ask your DBA to provide the details of the particular Agent's Integration Services job step.
Related
I have a SQL Server 2005 SSIS package with an Execute SQL task. I edit the SQL statement, go through the OKs to make the change. I go back in to the Execute SQL task- and the changes have taken hold. Great.
I now go to save the package. I click save. I look back in the Execute SQL task, and the changes have been lost.
What is going on here and how can I stop it?
Click the Execute SQL Task
Click F4 to see the list of properties
Click the Expressions line and then the little [...] box to see the expressions
Change the expressions (or remove them if you don't want your name/sqlstatementsource/etc. to be set dynamically).
This is a nice feature for reusability and template based development of SSIS packages. We use it allt he time.
I had the same issue. The reason was that I used a config file with all the executables included, and sql code is just a property of one of the executable which is stored in the config file. When you change your code the config file does NOT get updated, but when you close/open your project the values are getting pulled from the config file which reverse your change to the initial state!
In order to fix this you need either to exclude your executable from the config file (what I did) or re-create the config file every time you change the package.
I hope this helps.
I found I was unable to change the "To" in the Send Mail Task. Or rather, I would press OK on the new value and then bring it up again and it would be the old value. What I did -- was to edit the dtsx file in Notepad by searching for the old value and changing it.
We are creating several SSIS packages to migrate a large database as part of a release cycle.
We may end up with about 5-10 SSIS packages.
As we have 4 environments (dev, QA, staging, production, etc.), is there an efficient way to change the destination server for each SSIS package as they go through the different server environments? Ideally, there could be a script that is run that would take as a parameter the server that was needed.
You could use a configuration file to store the connection strings for the servers. Then as you moved from environment to environment, you would simply change the config file. To simply create a config file, on the control surface of your package,
1) right click and choose Package Configurations from the context menu.
2) Check the box for Enable package configurations if it is not already selected,
3) then Click the Add... button.
4) Click next on the dialog,
5) then add a Configuration file name: and click next.
6) In the Objects View, Under Connection Managers, expand your connection, then expand Properties and check the box next to ConnectionString.
7) Then click next
8) then finish.
You now have an xml file named what you named it in step 5 above. You can edit this file with a text editor and change the connection string to map to whichever server you need it to before each run.
Once created you can share the config file between multiple packages as long as the objects referenced are named the same between the packages.
This is a rudimentary tutorial on configurations, there are many ways of saving configurations of which this is only one. For more information on configurations consult your favorite SSIS book
We use a config table that stores the configurations for the server. But config files work well too. We like the table because we are doing reporting on SSIS package meta data and it's easier to grab this data (along with a lot of other data we store as well) when stored in a table.
William Todd Salzman's answer covers most points. I have a couple more to add:
Make sure the pacakge ProtectionLevel property is DontSaveSensitive
If you are working with different shipping environments, then a SQL Server table as a source for the package configurations is maybe not for you, as you will require one central database containing all the connection strings for all the servers.
Having worked with package configurations retrieved from the registry, you will need to be aware that these settings are retrieved from the HKEY_CURRENT_USER hive. This has implications for when the package is run through a SQL Agent Job.
I'm trying to move my packages to production using a configuration file, but file is changed only partly and the results go still to DEV server.
Does anybody know what to do?
It is difficult to isolate the cause of your issues without access to your configuration files.
What I suggest you do is make use of package configurations that reference a database within your environment. The databases themselves can then be referenced using environment variables that are unique to each environment.
This a brilliant time saver and a good way to centrally manage the configuration of all your SSIS packages. Take a look at the following reference for details.
http://www.mssqltips.com/tip.asp?tip=1405
Once configured, you can deploy the same identical package between dev and production without needing to apply a single modification to the SSIS package or mess around with configuration files.
You could still have hard-coded connections in your package even though you are using a configuration file. You'll need to check every connection as well.
You can also go the long way around. Go into Integration Services and Export the stored package to its dtsx file. Then you can pull open the file in any good text editor, do a find/replace on your server name and then go back into Integration Services and Import the updated package. Alot of times it's just easier...
everybody and thanks for answering. I'd managed to solve this problem in an ugly way - editing packages on server, but I'd like very much more elegant solution - now I'm trying with environment variable,it seems great, but the wizard that I'm getting is different from that is given in link - and I don't know how to continue.(I'm using VStudio 2005) Besides, I tried configuration file as XML, but package run fails even on the source machine, so I'm stuck !
My personal technique has been to first have a single config file that points the package to a SQL Based Package Config (the connection string to the config DB). Subsequent entries in the package config use the SQL store to load their settings. I have a script that goes into the XML of the package and preps them for deployment to stage or prod. A config file holds the name of the Package Configuration's initial file config entry and where the stage and prod configuration db configruation file is located. The script produces two subdirectories for stage and prod. Each directory has a copy of the solution packages modified for their particular deployment.
Also! Don't forget to turn off encryption in the package files!
I'm just trying to understand SSIS packages a bit better and how they are deployed. Correct me I'm wrong but for any deployment, I believe there needs to be at least two files a .SSISDeploymentManifest and a .dtsx. The .SSISDeploymentManifest acts as the equivalent windows installer package which points to the .dtsx. The dtsx is the actual package of "stuff" that is referenced as an external file some how when you run the installer. When you install it, the package gets added to a list of ssis packages for that instance.
My further questions:
If i wanted to keep previous version of the same package, can I just copy the bin directories with the two above files and keep separately should I need to roll back to a previous package?
Where are these packages installed to? How does SSIS know where the packagess are?
Correct me I'm wrong but for any deployment, I believe there needs to
be at least two files a .SSISDeploymentManifest and a .dtsx. The
.SSISDeploymentManifest acts as the equivalent windows installer
package which points to the .dtsx. The dtsx is the actual package of
"stuff" that is referenced as an external file some how when you run
the installer. When you install it, the package gets added to a list
of ssis packages for that instance.
Your assumptions are mostly correct. You don't need the deployment manifest, but it can be handy. Also, you don't need to deploy to the SQL Server instance. You have the option to deploy to the file system as well. I'll explain both below.
Regarding your 1st question:
Version Control:
Make sure you're developing and checking in your dtsx packages via visual studio. Label your releases in sourcesafe or whatever version control you're using. If you are checking in and labeling, then you should be able to easily roll back to a previous version. As you mention, you also can just save a copy of your old bin directory but naturally put them in dated subfolders or something. However, this does not take the place of proper version control.
Regarding your 2nd question:
Deployment:
As the other poster states, you first have a decision to make:
a) Deploy packages to the file system
b) Deploy packages to MSDB
There are benefits to each, and everyone has their preference. I have used both, but I prefer the filesystem because it's more transparent, however there is more to maintain.
See this post for much more on this: http://blogs.conchango.com/jamiethomson/archive/2006/01/05/SSIS_3A00_-Common-folder-structure.aspx
The code is in the dtsx package. Generally,in order to make your packages portable you also abstract your connection strings and other configurable information into a config file (.dtsconfig) or environment variable (no file needed). See BOL to learn more about configuration.
The manifest file contains metadata about which dtsx and config files to install. If you open one, you'll see it's a simple readable xml file.
The manifest file makes it easy to hand over to a DBA to deploy (ask them to double-click the manifest file and follow directions, but they'll need instructions.
To me, the manifest file is more useful for deploying to SQL Server than to the file system. Really, all it does is make a copy of the dtsx and config files and puts them where you tell it. You could just as easily instruct the DBA to copy your dtsx files to a common folder on the server, and the config files to another folder on the same server.
Then when you schedule your jobs using SQL Agent, you specify that you're going to run an SSIS package that is stored on the file system and browse to where it's located. If you're using configurations, then there's a tab to specify where the config file is located.
There is so much to know about configuring/deployment/versioning of SSIS packages. But hopefully this will get you started on the right path.
When you export your DTS packages using the Import/Export Wizard in SQL Server you have the option of saving them to SQL Server or locally on the file system.
Regarding the versions of your SSIS packages, you need to query SSISDB to extract the version numbers. It's annoying this kind of info isn't shown directly in the Management Studio but, until it is, someone may find this useful:
SELECT prj.[name] as Project
,pkg.[name] as Package
,pkg.[version_major]
,pkg.[version_minor]
,pkg.[version_build]
FROM [SSISDB].[internal].[packages] as pkg
JOIN [SSISDB].[internal].[projects] as prj
ON pkg.[project_id] = prj.[project_id]
ORDER BY prj.[name]
I have an SSIS Package that sets some variable data from a SQL Server Package Configuration Table. (Selecting the "Specify configuration setings directly" option)
This works well when I'm using the Database connection that I specified when developing the package. However when I run it on a server (64 bit) in the testing environment (either as an Agent job or running the package directly) and I Specify the new connection string in the Connection managers, the package still reads the settings from the DB server that I specified in development.
All the other Connections take up the correct connection strings, it only seems to be the Package Configuration that reads from the wrong place.
Any ideas or am I doing something really wrong?
The only way I was able to do this was to use Windows Environment Variables. You can specify things like connection strings and user preferences in environment variables, and then pick up those environment variables from your SSIS Task.
I prefer to use Server Aliases in the SQL Client Configuration. That way, when you decide to point the package to another SQL Server it is as simple as editing the alias to point to the new server, no editing necessary in the SSIS package. When moving the package to a live server, you need to add the aliases, and it works.
This also helps when you have a real painful naming convention for servers, the alias can be a more descriptive name than the actual machine name.
I didn't actually understand your question completely but I store my connection settings in a configuration files usually one for each environment like dev, production etc. The packages read the connection settings from the config files when they are run.
When you're creating a job to call the SSIS package, and you're setting up the step, there is a tabbed area. The default tab is where you set the package name, and the next tab over is where you can set the configuration file. Have a config file for each package, and change for the server (dev, test, prod). The config file can be put directly on the dev, test, and prod servers, and then point to them when setting up that job.
If u are using SQL Server Package Configuration then all the properties of the packages will come from SQL Server table - Please check that
SSIS security the way it stands is terrible. No one will be able to support things when I am out of the office. The job never reads from the configuration file...I give up. It only works when I edit the string in the Data sources tab. However the password gets lost if you happen to go into the job a second time. Terrible design, absolutely horrible. You would think that when you specify a xml file in the job step it would read the connection string from there that is defined, but it does not. Does this really work for anyone else?
Goto the package properties and set deployment True. This should work for what you have done.
I had the identical question, and got the same answer, i.e. you cannot edit the connection string used for package configurations hosted in SQL Server, except if you specify that the SQL Server connection string should be in an environment variable.
This unfortunately does not work in my dev setup, where two environments are hosted on the same machine. I ended up following Scott Coleman's approach as detailed on SQL Server Central [Free sign-up and a good site]. The trick is that you create a view to store your configuration settings on one central server, and then use the machine that connects to it to determine which environment is active.
I used that approach, but also used the User connecting to the environment to make a determination, because my test and dev setups run on the same SSIS instance, but as different user names. Scott suggests in the comments that the application name should be set, but this cannot be changed in the package execution job step, so it was not an option.
One other caveat that I found was that I had to add "Instead of" triggers to my view to do the inserts, updates and deletes for configuration variables.
We want to keep our package configs in a database table, we know it gets backuped with our other data and we know where to find it. Just a preference.
I have found that to get this to work I can use an environment variable configuration to set the connection string of the connection manager that I am reading my package config from. (Although I had to restart the SQL Server agent before it could find the new environment variable. Not ideal when I deploy this to Production)
Looks Like when you run an SSIS package as a step in a scheduled task it works in this order:
Load each of the Package Configs in the order they appear in the Package Configuations Organiser
Set the Connection Strings from the Data sources tab in the Job Step properties of the Scheduled Job
Start running package.
I would have expected the first 2 to be the other way around so that I can set the data source for my package config from the scheduled job. That is where I would expect other people to look for it when maintaining the package.