I am using SSDT 2017. There are 3 Dataflow tasks (They are connected using precedence constraints) which loads data from Excel into Database tables. When I run each task individually it is showing as success but when I run the entire package it is showing as completed with out any green tick on the tasks which means the tasks are not getting executed. After I changed the RetainSameConnection Property of Excel to True, all the tasks are getting completed successfully.
I have not seen this behavior in the earlier versions. Is this a new change in 2017 version or am I missing anything ?
Trying to figure out the issue
Based on the question and comments you are using 3 Data flow Task that contains 3 Excel source components that use the same connection manager.
At the start of the package execution in the Validation phase, each one of this components try must acquire the connection from the connection manager to read the metadata and try to keep this connection open until it must be executed and since there are many files that try to open the same connection it will cause a problem.
When using RetainSameConnection the package run the acquireconnection method once and use it multiple time, then the issue is solved.
Something to try
Try changing the Delay Validation property to True, and ValidateExternalMetadata property to False for each Excel Source on all Data Flow Tasks, it may solve the issue if the connection is only acquired for validation purposes. If it doesn't works then it means that the acquire connection method is called to lock the file for read even if the data flow task execution is not started yet.
Related
I am dealing with my problem on some Windows Server 2019 (Core) with one running SQL Server 2019 CU4 instance each.
What we try to do
We are currently building a data warehouse with distributed databases. The individual layers of the DWH are located on one database server each. The data exchange between the layers/servers takes place via SSIS ETLs, which use Linked Servers to reach the other layers and drag and drop data. Each layer also has its own SSIS service instance and executes the corresponding SSIS packets.
The SSIS packages are called by SQL Server Agent jobs. We have a job that executes the SSIS packets (#1), which in turn calls another job (#2) as the last step, which after a short wait time executes the calling job (#1). Thus, controlled by schedules, a loop is created and data is continuously transferred with ETLs.
I hope this was not too much unnecessary background
The error
Basically the job is running and there are numerous successful executions. However, we are observing interruptions at job #1 without helpful information regarding the error. This means that the job history log refers to the SSIS log, which again only contains an "unexpected termination". In the SSIS log, we only see behavior that indicates that the ETL packet active at that time stopped after validation. Depending on the log level, nothing is logged at all, not even the execution of single packages of the project. The package where this error occurs is different and not limited to a specific one.
What I have already tried
Re-create the jobs and SSIS Enviroments by hand (scripted before)
Using the 32Bit Runtime
Upgrade the SSIS project/package version to
2019
Increase the log level to "verbose"
Patching the SQL Server to CU4
Save ssis dump files (couldn't find them or they weren't created)
Search Windows and SQL Server Logfiles
Does anyone have some suggestions or some ideas how to become more error specific informations?
Thank you very much and take care :)
UPDATE We have an error message (OLE DB 0xC0202009 and 0X80004005)!
In order to exclude the use of environments as a cause, I manually set the parameters in the SSIS job step instead of overwriting them by selecting an environment.
Long story short: Today it turns out that the parameter for an OLE DB Connection String is not passed correctly.
The following is specified as a parameter in the job step:
However, the following connection string is specified in the context of the error message:
Please note that some arguments are added twice to the parameter (red).
What could have caused that?
I'm having some issues with an MDX query source in an SSIS data flow.
If I configure an OLEDB source properly, and have the data access mode as SQL Command, the MDX query works.
I need this source to be parameterized though, so I'm trying to pass in a variable that is populated at runtime as the MDX source query.
The problem is, when I set this up, it will try to use the variable (which is not correct until runtime) and throw this error.
What is the trick to getting an MDX Source to work from a variable?
I built all of the downstream transformations after first configuring the source with a hardcoded query (SQL Command). Then I went back to change the source to use the variable and it broke.
Thanks for any input.
TITLE: Microsoft Visual Studio
------------------------------
The component reported the following warnings:
Error at DFT SSAS to SQL [SRC SSASPRP01 Cube [2]]: No column information was returned by the SQL command.
Choose OK if you want to continue with the operation.
Choose Cancel if you want to stop the operation.
------------------------------
BUTTONS:
OK
Cancel
------------------------------
You want to a parameterized query and would like to build a String variable with the query. Anyway, your package needs to be validated before run. So, here you have two options:
If your query variable is populated at runtime and it has no Expressions, you might specify this variable value with a valid MDX query. The package and your DataFlow task will be validated before run (regular process)with this default query and pass, and at runtime - use correct MDX query.
You can set DelayValidation property of your DataFlow task to true. Then it will be validated immediately before running, when your variable will contain valid MDX query.
I would prefer the second method as more generic.
Set Delay Validation = True. Delay validation is a property available for all SSIS components and it basicly holds the validation back till excution. Mostly when we set connections or other components with variables it helps. As the variables dont have the true property at run time.
I am trouble shooting an error in a package.
Update MYTABLE for MYCOLUMN (REF to task name):Error: Executing the query "..." failed with the following error: "Invalid column name 'MYCOLUMN'.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I have verified the table and column exists, the length of the field is way excessive than what it needs that is 14 where it is declared as varchar(250).
I have verified the script works on the server in SSMS outside of the context of the package.
I have verified the connection and database in the package is as I expect.
Is there away to verify on the server. I did try to look at the Connection Managers tab on the package configuration itself i.e. in the Integration Services Catalogs->SSISDB->solutionfolder->..->package.dtsx->Configure context menu but it is empty.
Any ideas on how to troubleshoot?
Just to add more context the package contains 27 other tasks, 9 tasks in a row linked to this task but all set to on completion, all seem to be doing stuff independent of the other. 1 task is a loop doing stuff and the rest are single independent tasks. So I don't know at this stage if it is a cascading connection issue perhaps however; I am just reading what the log says.
I kicked off the package at 9:54am, the timestamp on the error log says 11:45am so nearly 2 hours into running is this log reported.
I would suggest the below things to troubleshoot the issue.
I would suggest you to just have this task and disable all other
tasks to troubleshoot the issue. So that you can focus on this issue
specifically. That will tell you whether connection is working fine
without issues.
I would suggest you to edit the task and see whether parameters are
set properly. Different providers have different way of setting
parameters. Again check whether parameters are proper. Execute SQL
Task
one more thing, may be you are pointing the package to different
connection than the one you used for SSMS. So, it is working in SSMS
and in the connection being used in the package is not having schema
changes yet done.
I finally figure it out before I read the previous offered suggestion so will give some credit if I can! FYI: We have a lot of dev servers. I clicked on the overview hyperlink in the All Execution logs and it said another server. Also I found the connection on the job calling the package not the package itself so I have learnt something today. Anyhow the job said one server but the overview said another so I again I was back to square one scratching my head.
Then I decided to open the connection manager on the job and select the field and make no change rather then cancelling I clicked ok not thinking about it and noticed the field changed to bold face. So I am assuming if you make a manual change on the server in SSMS to anything it shows up in bold which is kind of useful. So I can only assume this is a MS SSMS or SSIS or VS deployment bug. That it does not overwrite, the previous connection although the SSMS interface says otherwise. Perhaps somebody can share some light. Having not checked the server before I made a change and deployed it I have no idea if the previous settings were changed manually by someone or the connection in the package was changed and deployed. Anyhow checking the job history shows it had been failing for awhile so it wasn't me so whoever and whenever a change was done by a previous developer didn't figure it our either or didn't bother or did not know how, or didn't observe it. Anyhow it is pointing to the correct server now!!!
SSIS Scenario
I have a SQL variable of type object. It contains all the connection to different servers/databases. I want to connect to those databases one by one and run a query.
Expected Exception Handling
If the SSIS Connection manager(Dynamic connection manger) is unable to find the connection to the server (probably the server is down) I want to Skip that connection (Database/server) and log that into the table and move onto the next Connection (Database/Server). The SSIS package should not crash.
My Implementation
I have Successfully Configured the SSIS package to Use Connection Manager (Dynamic Connection Manager) and Foreach loop to loop through the SSIS variable of type object. but I am not able to skip the Connection if the server/database is not found. it generates error that the server/databsse not found / problem with the connection and the SSIS package fails.
my experience in SSIS is one week old
Any help will be appreciated.
How about setting --> Force execution Result property of the task to success
I was looking for the same fix too. Seems the normal OnError Eventhandling does not work for issues that arise when connecting to a source DB.
There is another workaround I wanted to mention. You can handle the error in the Data Flow Task (OnError Eventhandler, set the System-Variable "Propagate" in that Eventhandler to false). I think this is still required, but not sure. I use it also to log the exception.
Afterwards you can set MaximumErrorCount in ForEachLoop to "0" (which means unlimited). I'm not exactly sure why it works, but trying to find a way to handle the scenario you described, I found this.
==
Just as an interesting observation: For debugging purposes I added an OnError Eventhandler to the ForEachLoop and set a breakpoint there in a dummy script. It was never reached. Nonetheless the ForEachLoop failed all the time until I set MaximumErrorCount to 0.
I dont think its possible to continue the package execution once it encounters an error. You need to control this behavior through a SQL Server table (or any other table for that matter).
Once the package fails, you can set a flag in the table saying that the package failed. The next time the package runs, you can start from this point on and continue the execution. But automatically skipping the down server is kinda pulling a rabbit out of a hat.
Another way you can do this is to ping each server before hand in a separate package and store the ping results in a table. Only pick those records (servers) whose ping results were positive. Otherwise just skip the server.
I have an SSIS package which reads an Excel File (Data Flow Source) and transfer the data to SQL Server using OLEDB Destination Data Flow Item. The OLEDB Connection Manager used for the destination is configured to use Windows Authentication. The package works fine on my development machine. But when I open the same package on another machine and try to execute it gives the following error in Validation phase
Error: 0xC020801C at DFT_NSOffers, Source - 'Subscription Offers$' 1 [347]: The AcquireConnection method call to the connection manager "ExcelConnection_NSOffers" failed with error code 0xC0202009.
Error: 0xC0047017 at DFT_NSOffers, DTS.Pipeline: component "Source - 'Subscription Offers$' 1" (347) failed validation and returned error code 0xC020801C.
Error: 0xC004700C at DFT_NSOffers, DTS.Pipeline: One or more component failed validation.
Error: 0xC0024107 at DFT_NSOffers: There were errors during task validation
I'm using SQL Server 2005 (Version - 9.0.1399)
How do I fix this? Do I need to install any other component or service pack?
I don't think it's 64/32 bit error. My Dev machine and DB server are 32bit. But I could make it work. I had to set Delay Validation property of Data Flow tasks to TRUE.
Hi This can be solved by changing the prorperty of the project in the solution explorer then give false to 64bit runtime option
64/32 bit error? I found this as a problem as my dev machine was 32bit and the production server 64bit. If so, you may need to call the 32bit runtime directly from the command line.
This link says it better (No 64bit JET driver): http://social.msdn.microsoft.com/forums/en-US/sqlintegrationservices/thread/da076e51-8149-4948-add1-6192d8966ead/
I was finally able to resolve the "Excel connection issue" in my case it was not a 64 bit issue like some of them had encounterd, I noticed the package worked fine when i didnt enable the package configuration, but i wanted my package to run with the configuration file, digging further into it i noticed i had selected all the properties that were available, I unchecked all and checked only the ones that I needed to store in the package configuration file. and ta dha it works :)
For me, I was accessing my XLS file from a network share. Moving the file for my connection manager to a local folder fixed the issue.
If you are receiving preview of data in the excel source. But while executing the data flow task you receive Acquire connection error. Then move the file to local system and change the file path in excel connection manager and try executing again.
In my case the problem was the 32/64 bit driver which I solved by configuring the properties of the sql server job:
I had similar issue just that excel was the destination in my case instead of source as in the case of the original question/issue. I have spent hours to resolve this issue but looks like finally Soniya Parmar saved the day for me. I have set job and let it run for few iterations already and all is good now. As per her suggestion I set up the delay validation of the Excel connection manager to 'True. Thanks Soniya
Setting RetainSameConnection property to True for Excel manager Worked for me .
I had similar issue, trying to load data from Excel spreadsheet; and was running on WinX64. So I went VS BI`s project properties: Configuration Properties \ Dbugging
and Switch Run64BitRuntime from True to False.
It worked.
I was also getting the same error and it simply got resolved after installing the MS offices driver and Execute the job in 32 Bit DTEXEC. Now it works fine.
You can get the setup from below.
https://www.microsoft.com/en-in/download/confirmation.aspx?id=23734
In my case password i set in expression was wrong causing this error. After assigning correct password to expression of connection manager issue resolved.
In my case, none of the previous solutions here worked. Apparently Visual Studio, upon creating the Excel Source component, opens the Excel file and does not release it. Trying to then execute the SSIS package within Visual Studio leads to a AcquireConnection error with code 0xC0202009. Closing Visual Studio completely (not just the solution), reopen the solution and then run the package again without any further changes works. I found out when I tried to replace the Excel file and Windows Explorer said it couldn't because the file was open.
In order to resolve this issue make all your data flow tasks in one sequence. It means it should not execute parallel. One data flow task sequence should contain only one data flow task and for this another data flow task as sequence.
Ex:-