Summary: A "master" database houses a set of connection strings. A .dtsConfig XML file is used to point the packages to this database. Dynamic connection assignment accomplished by using package variables and expressions on the connections. Works flawlessly in the development environment, but once deployed to live it falls over.
I'm currently running into issues when deploying .dtsx packages to a production environment. The issue is specifically related to the connection manager when the jobs run. The history log reports a ... network ... error with a Login timeout error as the reason.
(For reference, I'm using Visual Studio 2013 with SQL Server Data Tools)
I have a table in a master database that holds the connection strings that it needs to process.
The packages check for a configuration file, that points them to the database as stipulated in (1).
The connections are retrieved and are placed in an object variable.
The variable is mapped to a foreach loop container, where a set of connection string variables are mapped to the relevant columns.
The packages then progress as normal.
Some notes:
When I did the development, I provided default values on my network for the connection strings.
I have checked the connection string parameters and formatting inside the database, and they conform to Microsoft's specification.
Our implementers installed SSDT on a client's QA server, where I altered the connection variables to point to their network. This solved the problem, but it is not sustainable (in my mind at least).
So my question is: how do I get my production deployments to work correctly with dynamic connection management assignment without having to alter the connection string variables inside each package on a per client basis?
Any help will be appreciated.
After doing more research by posing plenty of questions to DuckDuckGo, I finally got my answer here.
A quote from the above blog:
I had a set of SSIS packages running for my client using the third option listed above. The packages worked fine for ages until one fine day when they failed. The logs showed the packages had failed validation and I discovered that all the packages had their connection managers’ DelayValidation property set to False. The variable used to set the connection string had a default value pointing to the DEV server. These packages in production were actually trying to validate against DEV database though the connection string was dynamically set via a variable to point to PROD. This was dangerous as the jobs will not run if DEV server was down, which is exactly what had occurred.
Related
I started in a large company as an IT consultant. One of my tasks is to manage an application that has a SQL database.
I have very limited knowledge of SSIDB and SQL Server Management Studio - but I am willing to learn.
The SQL database is updated with external data. This can be done by users directly in the application, but it also happens through a scheduled job. The job runs from SQL manager. The job has only two steps, one of which is to execute a dtsx package.
The dtsx package is set up retrieves data from an external FTP server and merges the data into the database. The job was made by my predecessor and has run flawlessly for a very long time.
Now we are in the situation where the FTP, supplying the data, has been changed.
I have therefore been inside the Connection managers and changed to the new FTP server.
Running the jobs however we still get the following error message:
Failed to configure a connection property that has the following path: \Package.Connections[FTPConnection].Propterties[ChunkSize]. An error occurred while setting the value of property “ChunkSize”. The error returned is 0x80020009 “The ValidateDates has been migrated. The package must be saved to retain migration changes.”
I have checked the Connection managers and the ChunkSize is unchanged from when the job was working correctly. ChunkSize is set to 1000, both in the Connection manager, but also in the dtsx package itself.
When I have searched for the problem, it is mentioned that it may have something to do with the connection to the FTP server. So I have checked the connection to the FTP server from the server where the SQL database is located - and there is a connection. In addition to this, I have also made sure that there is a firewall rule that allows traffic between the two servers. This is ensured across protocols and port 20-22
When the job itself is run, however, no traffic leaves the server. So I believe the problem is with the job itself.
Edit: after having done a validation of the package i have gotten the following.
Failed to configure a connection property that has the following path: \Package.Connections[FTPConnection].Propterties[ChunkSize]. An error occurred while setting the value of property “ChunkSize”. The error returned is 0x80020009 “The ValidateDates has been migrated. The package must be saved to retain migration changes.”.
: at Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ConnectionParametersManager.ConfigureProperties(Sting parameterName, object parameterValue)
at
Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ConnectionParametersManager.ConfigureProperties()
at
Microsoft.SqlServer.IntegrationServices.Server.ISServerExec.ProjectOperator.ValidatePackageWithReference(Int64 validationId, Int64 infold, Int64 projectId, String packageName, Int64 versionId, Nullable'1 referenceId, Project isserverProject, Boolean OfflineMode)
I hope my description is comprehensive enough - otherwise please do write follow-up questions.
ps english is not my first language. sorry if something didn't turn out too well.
I'm trying to use SSIS to load some data from Oracle database to MSSQL database.
I created the project and used the ADO.Net source and was able to create a connection to Oracle and run queries and view results.
However when I actually run the package I get the following error:
Error: 0xC0208449 at Data Flow Task, ADO NET Source 2: ADO NET Source has failed to acquire the connection {EECB236A-59EA-475E-AE82-52871D15952D} with the following error message: "Could not create a managed connection manager.".
It seems similar to the issue here
And I did find that I have two oracle clients version installed "11.1" and "12.2".
One is used by PL/SQL and the other by other entity framework project.
If this is the issue I just wanted a way to tell the SSIS to pick-up the correct one.
I tried adding Entry in machine.config for "oracle.manageddataaccess.client" section with the desired version.
I also tried using other types of data sources but couldn't even create a successful connection
I tried changing the Run64bitRuntime property in the project to False
Note: I don't have SSIS installed on my machine.
Eventually, I just had to remove the entries related to 11.1 in path variable then restarted my machine.
Also I switched to "dotConnectForOracle" for connection and now it seems to be working fine.
I'm expecting issues related to other applications that might still be using the 11.1 version, but that will be a problem for another day.
Always make sure to write the user (oracle schema) in uppercase and some special characters [in my case it was $] in the password needs escape character even if you're using the wizard not the cmd
I still don't understand the whole issue but I hope this helps someone some day.
I have a SSIS project containing 50 or so packages. A master package executes the child packages in the right sequence.
The project is deployed to an SSIS catalog and execution is invoked by a stored procedure with some control logic and error handling.
The Connection Managers are parameterised with values passed in from Environment variables setup in the IS catalogs. The ServerName and InitialCatalog properties of the connection manager are parameterised but the ConnectionString is not.
DelayValidation is set to False on all packages.
So far so good. This has been working in dev, test and production environments for quite a while.
When I deploy new a version of a project I like to validate the project in SSMS to make sure that all of the data structures in the live environment match those expected by the packages.
I now have a validation failure in my test environment which I believe is because the ConnectionString, last updated in VS debug environment is out of synch with the ServerName and InitialCatalog specified in the test systems environment variables. The ConnectionString in the package is pointing to a server with a schema difference compared to live.
So, my questions are:
1. Does the ConnectionString string get rebuilt during SSMS driven validation or does it only get built during execution of the package?
2. If I set DelayValidation to True on this package then will it get validated during validation when it is installed in the catalog or will it only get validated at run time?
I think I was barking up the wrong tree and will dig deeper into the state of the databases that are being referred to in my packages.
My account doesn't seem to allow me to answer questions :-(
If someone else can put an answer in (any upvotes gratefully received and offered) then the question can be closed.
Connection Manager properties get fully resolved before the individual package gets validated. As #Ferdipux says, you can see the property values at execution (if you filter on Context Depth = 2 in the Error Context report) and you will see the value for the ConnectionString property correctly resolved from the ServerName and InitialCatalog variables set in the environment variables for the project. I don't think there is an issue with SSIS, it is likely that the database, referred to in the package, is still out of sync with your package metadata.
This is my first time trying to extract data from an Oracle database and push it into a Microsoft SQL database, and I'm running into an issue I cannot find a way around. I've installed ODAC, Oracle Client, on the SSIS machine and am able to see and use the Oracle Provider for OLE DB Connection manager just fine. I put in the Oracle server name, login, PW, test the connection, works fine. I can even run the SOURCE query and load it into our SQL database just fine. But if I try and deploy the package, or open up the connection manager again, everything fails because the password isn't saved, despite the obvious "save password" checkbox being checked.
After some searching, it appears that checkbox just doesn't do anything, and I've tried the following workarounds with no success:
Configuration Files. Set up the package to use a configuration file, tried to hard-code the password into that, both on its own and/or included in the connection string line. The package just ignores the password in either case.
Expressions. I've tried using the expression in this format: "Data source=SERVER;"+ "user id = USERID; " + "password = PASSWORD; " + "Provider = OraOLEDBOracle.1; " + "persisit security info = true;", which fails (connection manager always switches to "offline" mode and doesn't seem to process the expression), and also tried to just use the expression for "password" which... sort of worked one time, inside SSIS (it seems to run and looks OK inside the data flow but on the control flow the Connection Manager shows as offline again) but fails when deployed.
In all cases, I've tried using various ProtectionLevel settings: DontSaveSensitive, EncryptSensitiveWithUserKey, EncryptSensitiveWithPassword (and then adding the password to the execution of the package in the SSMS job manager). None have worked any different than the others.
I've tried this both on our actual server where our SQL DBs and SSIS server live, and on my local machine to test out the same settings to ensure it's not machine-specific. I've uninstalled and reinstalled Oracle and the ODAC on both at least once now.
At this point I can't find any other suggestions, it seems like one of those setups has worked for everyone eventually after installing everything properly. And again, the package WORKS as long as I set the password IN the connection manager and run the package manually inside SSIS, but not under any circumstances outside SSIS.
I've spent more than 2 straight days trying to troubleshoot this and am beginning to lose my mind. Aside from the obvious complaint of "What the hell is that save password checkbox even for then" I dunno what to try at this point. I really really want to use the Oracle Provider because some of this data is using complicated queries and even the smallest one is 200K+ rows, and hitting the corporate "warehouse" so it's essential to be as efficient as possible. If there's no other alternative I could try the older OLE DB Connections but that's an absolute last resort at this point.
Other info I can think of: Using Visual Studio 2010 (tried both free and professional versions), tried running both 32bit and 64bit runtime on packages (runs fine either way in SSIS but neither way in SSMS because... no password), and I have no control over the Oracle server/DBs and there's 0 chance of getting a no-password account for access (nor should I need to frankly)
Thank you for any assistance or tips!
I'm getting this error when running an SSIS package through SQL Agent
Failed to acquire connection "ORACLE ADO.NET". Connection may not be configured correctly or you may not have the right permissions on this connection.
When I log on as the SQL Agent User and run the ssis package directly it is fine. When I then execute it through the SQL agent job, it fails.
I've read around extensively on this topic, and it seems a lot of the advise concerns how you are logged in, configuring of proxy accounts, etc, etc, etc, none of which has been helpful.
I am logging onto an Oracle database with an ADO.NET conncetion. The connection string is as follows (datasource, userid and password have been changed):
Data Source=DATASOURCE;User ID=userid;Password=password;Persist Security Info=True;Unicode=True;
I'm loading this from a registry setting using package configuration. To check that I am getting the correct string, I am writing it into a temporary log table. I am definately getting the string I need from the correct registry setting.
I've tested the oracle login credentials though PL/SQL developer, and it lets me login just fine.
As far as I can tell, as I'm using an explicit user name and password for the Oracle connection it just shouldn't matter who the SSIs pacakge is run as. The only point of failure that Ican see would be the reading of the information from the registry, but that seems fine.
I'm really quite baffled, I must confess, and would appreciate any help some of the splendid experts here can offer.
Many thanks,
James
Ok, tracked this one down after quite a lot of pain.
It was working fine on one environment, but not another, so I fired up Process Monitor (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) and ran a package through the SQL Agent job, comparing which system entities were hit on each enviroment.
On the failing environment, at the point of the bulk transfer operation, the package attempted to get the Oracle 11 client DLL, and then hung.
I knew that this was installed, and, moreoever, the DLL path was a system environment setting. After further investigation it was revealed that the server had not been rebooted since the Oracle Client install and the SQL Server Agent process had not bee recycled.
Yes, can you believe it, the old helpdesk fix "Can you reboot your computer?" worked.
Sigh!
We had issues at a client with running packages connecting to Oracle before stored on our sql server instance. The work around we found was to change the package property, protection level, to "Dont save Sensitive Data" and for security purposes, we encrypted the username and password in the package configuration that was decrypted by a udf in sql server. Of course, before you try the whole encryption part, I would recommend putting the username and password in the package configuration without encrypting the values to see if changing the protection level setting is the solution to your specific problem. I hope this helps.
I was getting this error when tnsnames.ora file did not have a valid entry for the environment