Can SSIS XML configuration actually alter the contents of a package?
I created a set of packages connecting to my local machine, and ran it.
Then i created XML configurations
then i modified dtsconfig XML , changing the DataSource in the ConnectionString and the ServerName as well to a second server.
i was getting login errors so i viewed the code of my packages. the connection managers in the packages now had the second server, not the localhost connection i had coded in the package. I am absolutely certain that i did not modify the connection managers in the package, only the configuration.
So my question is, can SSIS configurations change the package itself?
If you specify a config file at design time, then run your package, the properties of your design-time objects will be updated (in your design-time package) to the values specified in your config.
If you then save your package, it will be saved with the updated property values.
Related
I have build SSIS packages on Server (suppose A) defining it's configuration in a XML. I have copied the package and the config file to a different server (suppose B) and run it using a Batch command as below:
echo "Running SSIS Load" >D:\samplessis\log\Mylog.log dtexec.exe /f "D:\samplessis\ag_data_import.dtsx" /CONFIGFILE "D:\samplessis\config\AgDataLoad.dtsConfig" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF >>D:\samplessis\log\Mylog.log
This package seems to be using the configuration defined in package but not the xml(config file)
When DTEXEC is given an XML config file in MSSQL 2008 R2 or later, it configures itself in this order when run:
The utility first applies the design-time configurations.
The utility then applies the run-time options that you specified on the
command line when you started the utility.
Finally, the utility reloads and reapplies the design-time configurations.
(source: https://technet.microsoft.com/en-us/library/bb500430(v=sql.105).aspx)
This means that the configuration in the package at runtime is Design - XML - Design. If you were to change the contents of the XML file and run the DTEXEC command on your dev-box, you would get the same results...
To get around this, after setting up the configurations in the designer, clear out the initial values. For instance, if you have a connectino string as a parameter, design the Connection Manager, add the Connection String to the XML File, then in the Connection Manager's properties, clear out the connection string.
When there are no values specified in design-time, the designer will configure itself based on the XML file, as well as applying that XML file when run with DTEXEC.
I have an old SSIS package built under 2008R2. Other devs have tweaked it over the years. When I open it and manipulate it I have an issue where it WILL NOT persist the password in the connection manager. Otherwise I can more or less edit it.
When I save the package and deploy just the dtsx file to a test server, it is using a connection to a different server. I've opened up the raw XML and find no references to that server. The only connection in the package points to the server it is supposed to use.
How do I figure out where it is pulling this phantom connection string from?
Most likely its stored as part of a package configuration which has many possible sources from a SQL or a separate XML config file. Check the command line options for a configuration option where you are executing the package to determine the source. Package configurations override the default values of parameters used in a package, and abstract environment specific values (conn strs, file paths, credentials,...) for easier testing/QA, without hard coding values.
I have 100 SSIS package all the packages are created in Development environment.
I need to update the server name and database name of all the packages.
It consumes lot of time updating the server name and database name manually.
Is there a way i can update the database and server name in all the packages?
All the packages are stored at same location.
Ideally, you could have a single Master package calling each of your 100 packages which I assume are performing specific tasks. In this Master package, you can pass the Connection Manager details to each 'Child' package during run time itself. The way this is done is by creating variables in each package to hold the server name and database name, and then to update the ConnectionString property in each child package at the start of execution. That way, you only need to generate 1 configuration file for the Master package, and you can configure the database connection parameters as you want. However, this will entail extra development effort as follows:
A new SSIS package to act at Framework level and co-ordinate running the 100 packages
Variables in the new and existing packages to hold server name and database name
A new configuration file in the Master package
The alternative is to just generate configuration files for each of the 100 packages. Then, you will have a separate file for each package, but you will need to make your replacements in each of the configuration files.
You can check out the below links for resources on this:
Configuration Files in SSIS
Variables in SSIS
Have you looked into the "convert to project connection"? If you havent done this yet you are in for a bit of a grind to set it up the first time, but after that you can set your connection for the entire project in 1 place, and configure them in 1 go as well. Just right click on the connection and select the "convert to project connection" for the first connection. Then this project connection becomes visibile for all other components in the project and you can set it up.
Further, if you feel brave and smart you can try to find/replace these settings in the xml code of the packages. This is not easy and not a good idea if you dont know what your doing. If you do know what its all about you can complete this job in 10 minutes flat ;)
Dont say I didnt warn you!
Use update scripts to update internal.object_parameters table
update [SSISDBName].[internal].[object_parameters]
set design_default_value = ‘NewServername’
where design_default_value in (‘OldServerName1′,’OldServerName2’)
update [SSISDBName].[internal].[object_parameters]
set default_value = ‘NewServername’
where default_value in (‘OldServerName1′,’OldServerName2’)
Please find more details on https://moredvikas.wordpress.com/2016/11/17/script-to-update-server-name-in-ssis-package/
We are creating several SSIS packages to migrate a large database as part of a release cycle.
We may end up with about 5-10 SSIS packages.
As we have 4 environments (dev, QA, staging, production, etc.), is there an efficient way to change the destination server for each SSIS package as they go through the different server environments? Ideally, there could be a script that is run that would take as a parameter the server that was needed.
You could use a configuration file to store the connection strings for the servers. Then as you moved from environment to environment, you would simply change the config file. To simply create a config file, on the control surface of your package,
1) right click and choose Package Configurations from the context menu.
2) Check the box for Enable package configurations if it is not already selected,
3) then Click the Add... button.
4) Click next on the dialog,
5) then add a Configuration file name: and click next.
6) In the Objects View, Under Connection Managers, expand your connection, then expand Properties and check the box next to ConnectionString.
7) Then click next
8) then finish.
You now have an xml file named what you named it in step 5 above. You can edit this file with a text editor and change the connection string to map to whichever server you need it to before each run.
Once created you can share the config file between multiple packages as long as the objects referenced are named the same between the packages.
This is a rudimentary tutorial on configurations, there are many ways of saving configurations of which this is only one. For more information on configurations consult your favorite SSIS book
We use a config table that stores the configurations for the server. But config files work well too. We like the table because we are doing reporting on SSIS package meta data and it's easier to grab this data (along with a lot of other data we store as well) when stored in a table.
William Todd Salzman's answer covers most points. I have a couple more to add:
Make sure the pacakge ProtectionLevel property is DontSaveSensitive
If you are working with different shipping environments, then a SQL Server table as a source for the package configurations is maybe not for you, as you will require one central database containing all the connection strings for all the servers.
Having worked with package configurations retrieved from the registry, you will need to be aware that these settings are retrieved from the HKEY_CURRENT_USER hive. This has implications for when the package is run through a SQL Agent Job.
I have an SSIS Package that sets some variable data from a SQL Server Package Configuration Table. (Selecting the "Specify configuration setings directly" option)
This works well when I'm using the Database connection that I specified when developing the package. However when I run it on a server (64 bit) in the testing environment (either as an Agent job or running the package directly) and I Specify the new connection string in the Connection managers, the package still reads the settings from the DB server that I specified in development.
All the other Connections take up the correct connection strings, it only seems to be the Package Configuration that reads from the wrong place.
Any ideas or am I doing something really wrong?
The only way I was able to do this was to use Windows Environment Variables. You can specify things like connection strings and user preferences in environment variables, and then pick up those environment variables from your SSIS Task.
I prefer to use Server Aliases in the SQL Client Configuration. That way, when you decide to point the package to another SQL Server it is as simple as editing the alias to point to the new server, no editing necessary in the SSIS package. When moving the package to a live server, you need to add the aliases, and it works.
This also helps when you have a real painful naming convention for servers, the alias can be a more descriptive name than the actual machine name.
I didn't actually understand your question completely but I store my connection settings in a configuration files usually one for each environment like dev, production etc. The packages read the connection settings from the config files when they are run.
When you're creating a job to call the SSIS package, and you're setting up the step, there is a tabbed area. The default tab is where you set the package name, and the next tab over is where you can set the configuration file. Have a config file for each package, and change for the server (dev, test, prod). The config file can be put directly on the dev, test, and prod servers, and then point to them when setting up that job.
If u are using SQL Server Package Configuration then all the properties of the packages will come from SQL Server table - Please check that
SSIS security the way it stands is terrible. No one will be able to support things when I am out of the office. The job never reads from the configuration file...I give up. It only works when I edit the string in the Data sources tab. However the password gets lost if you happen to go into the job a second time. Terrible design, absolutely horrible. You would think that when you specify a xml file in the job step it would read the connection string from there that is defined, but it does not. Does this really work for anyone else?
Goto the package properties and set deployment True. This should work for what you have done.
I had the identical question, and got the same answer, i.e. you cannot edit the connection string used for package configurations hosted in SQL Server, except if you specify that the SQL Server connection string should be in an environment variable.
This unfortunately does not work in my dev setup, where two environments are hosted on the same machine. I ended up following Scott Coleman's approach as detailed on SQL Server Central [Free sign-up and a good site]. The trick is that you create a view to store your configuration settings on one central server, and then use the machine that connects to it to determine which environment is active.
I used that approach, but also used the User connecting to the environment to make a determination, because my test and dev setups run on the same SSIS instance, but as different user names. Scott suggests in the comments that the application name should be set, but this cannot be changed in the package execution job step, so it was not an option.
One other caveat that I found was that I had to add "Instead of" triggers to my view to do the inserts, updates and deletes for configuration variables.
We want to keep our package configs in a database table, we know it gets backuped with our other data and we know where to find it. Just a preference.
I have found that to get this to work I can use an environment variable configuration to set the connection string of the connection manager that I am reading my package config from. (Although I had to restart the SQL Server agent before it could find the new environment variable. Not ideal when I deploy this to Production)
Looks Like when you run an SSIS package as a step in a scheduled task it works in this order:
Load each of the Package Configs in the order they appear in the Package Configuations Organiser
Set the Connection Strings from the Data sources tab in the Job Step properties of the Scheduled Job
Start running package.
I would have expected the first 2 to be the other way around so that I can set the data source for my package config from the scheduled job. That is where I would expect other people to look for it when maintaining the package.