I'm trying to use SSIS to load some data from Oracle database to MSSQL database.
I created the project and used the ADO.Net source and was able to create a connection to Oracle and run queries and view results.
However when I actually run the package I get the following error:
Error: 0xC0208449 at Data Flow Task, ADO NET Source 2: ADO NET Source has failed to acquire the connection {EECB236A-59EA-475E-AE82-52871D15952D} with the following error message: "Could not create a managed connection manager.".
It seems similar to the issue here
And I did find that I have two oracle clients version installed "11.1" and "12.2".
One is used by PL/SQL and the other by other entity framework project.
If this is the issue I just wanted a way to tell the SSIS to pick-up the correct one.
I tried adding Entry in machine.config for "oracle.manageddataaccess.client" section with the desired version.
I also tried using other types of data sources but couldn't even create a successful connection
I tried changing the Run64bitRuntime property in the project to False
Note: I don't have SSIS installed on my machine.
Eventually, I just had to remove the entries related to 11.1 in path variable then restarted my machine.
Also I switched to "dotConnectForOracle" for connection and now it seems to be working fine.
I'm expecting issues related to other applications that might still be using the 11.1 version, but that will be a problem for another day.
Always make sure to write the user (oracle schema) in uppercase and some special characters [in my case it was $] in the password needs escape character even if you're using the wizard not the cmd
I still don't understand the whole issue but I hope this helps someone some day.
Related
I've created a package that gets some information from a sql database and inserts them into Dynamics CRM.
When testing the package from Visual Studio everything goes as expected and the task finishes without any errors and the rows get inserte . However , when I publish the package to SSISDB on Sql Server the package fails with this error :
KingswaySoft.IntegrationToolkit.DynamicsCrm.CrmServiceException : CRM service call returned an error : A password is required in order to establish the connection ...
I tried changing the package protection level to EncryptSensitiveWithUserKey but it still gives the same message as above , created the package again from scratch still doesn't work . This package was working before , maybe there's something I did the last time in configuration which made it work but I cannot replicate it anymore .
Also I tried the Integrated Authentication it says this :
KingswaySoft.IntegrationToolkit.DynamicsCrm.CrmServiceException : CRM service call returned an error : The caller was not authenticated by the service .
#Drinv, this is a typical SSIS runtime deployment issue. You need to make sure that you have provided a password for your job configuration for the connection manager. What you provided to the package doesn't count as far as sensitive fields are concerned (password being one) when you are using the EncryptSensitiveWithUserKey option since user key is not transferrable between different systems or different users. An easy workaround is to change your SSIS package/project's ProtectionLevel setting to encrypt using a password instead, although it may not be the best practice. If you still have trouble getting this going, please reach out to us directly, our team can walk you through the issue.
I found out what I was doing wrong .
My SSIS project was on Project Deployment Model and I was trying to deploy only the package. After making my connections available on project level and deploying the whole project everything worked as expected .
I need to extract data from C4C and I use SSIS and Odata connector to do the job. It worked fine for a couple of month but now my SSIS job is not working any more since 2 days and I cannot find any tips on the web. I tried the registry key, update my VS to 2017 but nothing worked! I'm sure it's related to TSL1.2 but don't know how to fix it. In development tools, I can see data in preview mode but when I execute the job it gives me an error:
Cannot acquire a managed connection from the run-time connection manager
I have created a SSIS package and I am trying to run it locally. We use package configurations that point to sql tables and a XML config file. The package ran successfully for about a week, even when deployed to a SQL Server Agent Job in our STAGE environment.
Now, the only way I can get the package to run is by not using the Package Configurations and choosing EncryptSensitivewithPassword. If I change the package to DontSaveSensitive, I continuously get the error below:
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80040E4D Description: "Login failed for user 'Test_User'.".
Error: 0xC020801C at AgentCompany, Lookup [37]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Test" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
It is so strange that about a week ago, this package ran fine with the configurations and the DontSaveSensitive Option.
I have updated the config file to ensure that it is establishing the connection string to the appropriate database. I also test the connectivity on the connection managers and they all test successfully.
I also double checked the SQL Database where the user is trying to connect to ensure that it has permissions there and it does.
I am very confused. Please Help!
Updating dtsconfig file
Re-creating the connection managers
Making some DFT task DelayValidation to true
Changing the RunTime to 32 bit
EncrpytPasswordSensitive with package configs removed---This works but this is not the standard at my company and this is not how I developed and tested the package before
When you open/run a package, an OnInformation event is fired that says something like
The package is attempting to configure from the XML file "c:\ssisdata\so_56776576.dtsconfig".
When Visual Studio/SSDT opens/runs a package which says it uses configuration but for reasons, cannot get them, you should then see messages like
Warning loading so_56776576.dtsx: Failure importing configuration file: "c:\ssisdata\so_56776576.dtsconfig"
and
Warning loading so_56776576.dtsx: The configuration file "c:\ssisdata\so_56776576.dtsconfig" cannot be found. Check the directory and file name.
and
Warning loading so_56776576.dtsx: Failed to load at least one of the configuration entries for the package. Check configuration entries for "Configuration 1" and previous warnings to see descriptions of which configuration failed.
If someone has manually edited the config file and broken the XML, you'd see a warning like
Cannot load the XML configuration file. The XML configuration file may be malformed or not valid
The important thing to note with regard to configuration - if a configuration cannot be found, SSIS will continue along with the design time values. That is why it is crucial to check the warnings emitted when your package runs. If you are running manually, ensure that you have /rep ew specified so you report Errors and Warnings.
Guesses as to root cause
The package has the protection level of EncryptSensitiveWithUserKey which means the AD credentials of the package creator are used to hash things that might have sensitive information in them. I could be using AD authentication in my connection string and specify that the connection should be trusted but that entire block is still going to get encrypted against my Active Directory account. When you come along and attempt to maintain the package, it's not going to be able to decrypt the sensitive data as you are not me.
The two ways around that are to use a shared key (EncryptSensitiveWithPassword/EncryptPackageWithPassword) which is cumbersome to deal with plus it goes against the whole spirit of secrecy since everyone knows the secret. The other approach as you've identified is DontSaveSensitive and that's my go to for all of this.
The problem to be overcome is that with DontSaveSensitive is that every time you save, SSIS is going to wipe out any knowledge of user name and password from places that might be holding on to it - like a connection manager. The 2005/2008 strategy to hedge against this was to use Configuration or explicit overrides at run time to supply user name and password. My typical approach was to use configuration based on a table instead of XML as I was better at securing sensitive data in a table than I was mucking with ACL on the file system. The other challenge we had with multiple developers and file based configuration was that either everyone had to set their file systems up the same (and we developers are unique rainbow snowflakes so that's unlikely) or we need to use a network shared file which is great until someone adds their own values to it and breaks it or removes your changes or any of a host of other challenges.
I am struggling with the above - I've installed PowerQuery (64 bit Excel 2013 setup) and under the database connection options, despite following the instructions here to download the Ngpsql data provider for PostgreSQL:
https://support.office.com/en-ie/article/Connect-to-a-PostgreSQL-database-Power-Query-bf941e52-066f-4911-a41f-2493c39e69e4?ui=en-US&rs=en-IE&ad=IE
I can still only see a limited set of options under under the database list, which do not include a PostgreSQL database. Now having hunted around on the web I found this thread:
https://superuser.com/questions/950100/connect-to-postgresql-database-from-excel-2013-power-query-with-npgsql
Which seems to suggest that the reason I cannot see the Postgresql option is that I am not using an OfficePro installation (think it was home edition).
Does anyone have any pointers - any workaround for this? Or do I really have to get a different version of office to get data from a Postgresql db, short of converting the database into Access or something? Thanks
[I have Office 2013 Pro.]
I had to do this in addition to installing "PowerQuery_2.44.4675.281 (64-bit) [en-us].msi" (I ticked GAC installation on the installation dialog) and "Npgsql-3.2.3.msi". Also, rebooted the machine.
Everything then started to work connection-wise.
But when returning large amounts of data into Excel with Powerquery at times I would get "type cast" errors - I could not map it down to NULLs or anything easy to determine. Powerquery seems a fine tool for some usages and I am sure this error can be fixed with data transformation steps.
If you just want to get the postgres data - you can use VBA + ADO. I have just finished setting it up and it works.
Install "psqlodbc_x64.msi".
Add references to your vba project.
Finally I created the connection with this connecton string (no windows DSN setup required - modify the string below as per your setup):
cnn.Open "Driver={PostgreSQL Unicode(x64)};Server=127.0.0.1;Port=xxxx;UID=postgres;PWD=postgres; Database=db_name;"
I could also successfully add it as a data source using the same connection string.
This is my first time trying to extract data from an Oracle database and push it into a Microsoft SQL database, and I'm running into an issue I cannot find a way around. I've installed ODAC, Oracle Client, on the SSIS machine and am able to see and use the Oracle Provider for OLE DB Connection manager just fine. I put in the Oracle server name, login, PW, test the connection, works fine. I can even run the SOURCE query and load it into our SQL database just fine. But if I try and deploy the package, or open up the connection manager again, everything fails because the password isn't saved, despite the obvious "save password" checkbox being checked.
After some searching, it appears that checkbox just doesn't do anything, and I've tried the following workarounds with no success:
Configuration Files. Set up the package to use a configuration file, tried to hard-code the password into that, both on its own and/or included in the connection string line. The package just ignores the password in either case.
Expressions. I've tried using the expression in this format: "Data source=SERVER;"+ "user id = USERID; " + "password = PASSWORD; " + "Provider = OraOLEDBOracle.1; " + "persisit security info = true;", which fails (connection manager always switches to "offline" mode and doesn't seem to process the expression), and also tried to just use the expression for "password" which... sort of worked one time, inside SSIS (it seems to run and looks OK inside the data flow but on the control flow the Connection Manager shows as offline again) but fails when deployed.
In all cases, I've tried using various ProtectionLevel settings: DontSaveSensitive, EncryptSensitiveWithUserKey, EncryptSensitiveWithPassword (and then adding the password to the execution of the package in the SSMS job manager). None have worked any different than the others.
I've tried this both on our actual server where our SQL DBs and SSIS server live, and on my local machine to test out the same settings to ensure it's not machine-specific. I've uninstalled and reinstalled Oracle and the ODAC on both at least once now.
At this point I can't find any other suggestions, it seems like one of those setups has worked for everyone eventually after installing everything properly. And again, the package WORKS as long as I set the password IN the connection manager and run the package manually inside SSIS, but not under any circumstances outside SSIS.
I've spent more than 2 straight days trying to troubleshoot this and am beginning to lose my mind. Aside from the obvious complaint of "What the hell is that save password checkbox even for then" I dunno what to try at this point. I really really want to use the Oracle Provider because some of this data is using complicated queries and even the smallest one is 200K+ rows, and hitting the corporate "warehouse" so it's essential to be as efficient as possible. If there's no other alternative I could try the older OLE DB Connections but that's an absolute last resort at this point.
Other info I can think of: Using Visual Studio 2010 (tried both free and professional versions), tried running both 32bit and 64bit runtime on packages (runs fine either way in SSIS but neither way in SSMS because... no password), and I have no control over the Oracle server/DBs and there's 0 chance of getting a no-password account for access (nor should I need to frankly)
Thank you for any assistance or tips!