I'm maintaining a project using Microsoft SQL Server 2016 (SP1) (according to this script) which heavily depends on recurring jobs (mirroring certain external db's and so on).
Especially the mirroring jobs are essentially based on SSIS packages which define a datasource, then execute a hardcoded SQL query and afterwards store the results in the specified destination.
Unfortunately the source databases where moved to a different domain and thus aren't accessible via the previous url.
My issue right now is that I simply have to change the source destination url but I'm not able to do that. There are plenty of ways to 'modify' SSIS packages but none of them seem to work with me.
What I managed (and seems the most promising) to do is to open the 'Integration Services...' part of my db, export the jobs to my desktop, modify them with Notepad and reimport them. And they seem to work if I execute them separately. But as soon as I try to execute the packages via SQL Server Agent it fails screaming:
Description: Failed to decrypt protected XML node "DTS:Password" with error 0x8009000B "Key not valid for use in specified state."
Does somebody know whats going on here and how I'm able to solve this? No password or username changed, only the connection string.
Is it even possible to manage a package like that?
Thank you for your help!
After further investigation I detected that even a newly created job didn't run properly. It was kind of strange that a package would run without any issues while directly executed but not via the SQL Server Agent, so I assumed it may be a rights issue and it was!
Somehow the Server Agent wasn't allowed to decrypt (although I never changed the executing user of a step) the password anymore.
I was able to work around my issue by simply creating each SSIS package again (some click hell but ok) but this time I secured 'sensitive data' with a password instead of the users key.
Afterwards I had to change each job step with a reference to to the damaged ssis packages and obviously type in the new passwords.
Seems to work again.
Thanks anyway
I have an old SSIS package built under 2008R2. Other devs have tweaked it over the years. When I open it and manipulate it I have an issue where it WILL NOT persist the password in the connection manager. Otherwise I can more or less edit it.
When I save the package and deploy just the dtsx file to a test server, it is using a connection to a different server. I've opened up the raw XML and find no references to that server. The only connection in the package points to the server it is supposed to use.
How do I figure out where it is pulling this phantom connection string from?
Most likely its stored as part of a package configuration which has many possible sources from a SQL or a separate XML config file. Check the command line options for a configuration option where you are executing the package to determine the source. Package configurations override the default values of parameters used in a package, and abstract environment specific values (conn strs, file paths, credentials,...) for easier testing/QA, without hard coding values.
I recently had to move my files to a new SSIS server. Everything seems to be working except when I try to execute a bulk insert it tells me
(Cannot bulk load because the file "E:\FlatFiles\SSG\apmast.txt" could
not be opened. Operating system error code 21(The device is not
ready.).".
It does this for all my flat files. I found an article saying you need to give the MSSQLSERVER user full control of the files, which I did but this does not seem to fix it. Any other ideas? Do I need to give other files the same permissions? I really don't want to just throw full control around if I don't have to. Thanks
I figured it out, turns out that a bulk insert tells the server to look locally for the text file. I was trying to get SSIS to do a bulk insert of flat files from one server into another sql server on the network. As soon as I put the flatfiles on the remote server it grabbed and used them. This seems like a very odd way for it to work. I would expect it to push the files to the sql server instead of asking the sql server to look for the files locally via a hard path.
I am using the SQL Server Import/Export Wizard to import data from an Excel file into a table. I will need to do this exact import many times so I selected the option "Save SSIS Package."
I tried saving to SQL Server and I tried saving to File System.
Either way, I cannot figure out how to use the package in SQL Server. I read that I need to use Agent Job Task to do it but I cannot find that anywhere in the SQL Server program.
I also looked for Integrated Services in SQL Server and I found an icon that doesn't seem to do anything.
How do I go from having a .dtsx somewhere on my computer to executing the import (in a query preferably) without having to go through the whole wizard process again and again?
Thank you
The Import/Export Wizard creates a file called .DTSX. This is your SSIS package. An SSIS package is a text file filled with a bunch of XML. Contrast this with something that ends in .EXE/.COM which is an executable that any windows machine will be able to run.
We need something to consume that .dtsx file and perform the ETL operations described therein. There are three options open to you: dtexec.exe, dtexecui.exe, and custom .NET code. dtexecui is just a graphical wrapper to the command line so really, your choices for running packages are the prebuilt command line utility or custom code. Running a package via dtexec can be as simple as dtexec.exe /file C:\MyPackage.dtsx
Since you want to run this from the context of SSMS, then you'll need to either Create a SQL Agent Job or create a stored procedure that calls dtexec. Since you're new to the whole business, the Job will provide the lowest barrier of entry.
In your SSMS, connect to an instance and navigate to the bottom. You should see "SQL Server Agent" Expand that node and Right-Click on "Jobs" and choose New Job. In the Steps tab, click New.... Change the Type to "SQL Server Integration Services Package"
Pick your source (file system or SQL Server) Fill out the server name or where the package lives and you should be good to go.
One very important thing you will need to know is that Excel drivers are only going to work in 32 bit mode. This means you will need to use the dtexec.exe that exists at C:\Program Files (x86)\Microsoft SQL Server\XXX\DTS\Binn\dtexec.exe To get that behaviour from the SQL Agent job, then you will need to add the /X86 option to the command line like
The x86 flag only works for SQL Agent job calls. From the command line, you must use the correct version of dtexec.exe
I have an SSIS Package that sets some variable data from a SQL Server Package Configuration Table. (Selecting the "Specify configuration setings directly" option)
This works well when I'm using the Database connection that I specified when developing the package. However when I run it on a server (64 bit) in the testing environment (either as an Agent job or running the package directly) and I Specify the new connection string in the Connection managers, the package still reads the settings from the DB server that I specified in development.
All the other Connections take up the correct connection strings, it only seems to be the Package Configuration that reads from the wrong place.
Any ideas or am I doing something really wrong?
The only way I was able to do this was to use Windows Environment Variables. You can specify things like connection strings and user preferences in environment variables, and then pick up those environment variables from your SSIS Task.
I prefer to use Server Aliases in the SQL Client Configuration. That way, when you decide to point the package to another SQL Server it is as simple as editing the alias to point to the new server, no editing necessary in the SSIS package. When moving the package to a live server, you need to add the aliases, and it works.
This also helps when you have a real painful naming convention for servers, the alias can be a more descriptive name than the actual machine name.
I didn't actually understand your question completely but I store my connection settings in a configuration files usually one for each environment like dev, production etc. The packages read the connection settings from the config files when they are run.
When you're creating a job to call the SSIS package, and you're setting up the step, there is a tabbed area. The default tab is where you set the package name, and the next tab over is where you can set the configuration file. Have a config file for each package, and change for the server (dev, test, prod). The config file can be put directly on the dev, test, and prod servers, and then point to them when setting up that job.
If u are using SQL Server Package Configuration then all the properties of the packages will come from SQL Server table - Please check that
SSIS security the way it stands is terrible. No one will be able to support things when I am out of the office. The job never reads from the configuration file...I give up. It only works when I edit the string in the Data sources tab. However the password gets lost if you happen to go into the job a second time. Terrible design, absolutely horrible. You would think that when you specify a xml file in the job step it would read the connection string from there that is defined, but it does not. Does this really work for anyone else?
Goto the package properties and set deployment True. This should work for what you have done.
I had the identical question, and got the same answer, i.e. you cannot edit the connection string used for package configurations hosted in SQL Server, except if you specify that the SQL Server connection string should be in an environment variable.
This unfortunately does not work in my dev setup, where two environments are hosted on the same machine. I ended up following Scott Coleman's approach as detailed on SQL Server Central [Free sign-up and a good site]. The trick is that you create a view to store your configuration settings on one central server, and then use the machine that connects to it to determine which environment is active.
I used that approach, but also used the User connecting to the environment to make a determination, because my test and dev setups run on the same SSIS instance, but as different user names. Scott suggests in the comments that the application name should be set, but this cannot be changed in the package execution job step, so it was not an option.
One other caveat that I found was that I had to add "Instead of" triggers to my view to do the inserts, updates and deletes for configuration variables.
We want to keep our package configs in a database table, we know it gets backuped with our other data and we know where to find it. Just a preference.
I have found that to get this to work I can use an environment variable configuration to set the connection string of the connection manager that I am reading my package config from. (Although I had to restart the SQL Server agent before it could find the new environment variable. Not ideal when I deploy this to Production)
Looks Like when you run an SSIS package as a step in a scheduled task it works in this order:
Load each of the Package Configs in the order they appear in the Package Configuations Organiser
Set the Connection Strings from the Data sources tab in the Job Step properties of the Scheduled Job
Start running package.
I would have expected the first 2 to be the other way around so that I can set the data source for my package config from the scheduled job. That is where I would expect other people to look for it when maintaining the package.