I am trying to process Excel files (2007 .XLSX) in a directory on the local machine. There have been a number of issues that have come up of which I will list. Before that let me explain what I want to do and how I have set things up and all my attempts so far.
Set Up
Microsoft Visual Studio 2012
Microsoft Windows 7 (32 bit)
Goal
Create a simple ETL to loop through Excel files in a folder and load data into a SQL server table.
Current Attempts
Note: I am very much familiar with programming and especially loops, however in SSIS I am struggling to get the Excel file from even being read.
I have tried and am doing the following:
Created a package
Created a FOREACH loop container task
FOREACH Loop Task
Set as Foreach File Enumerator
Expression assigns the Directory to the variable which has the value of the directory ('C:\Users\Me\Desktop\Excels\')
Under Variable Mappings I have specified a variable at index 0
Set to using File and Extension
For the moment I am looking at all files (.)
Note: I have used watch on debug to determine if the variables get populated. Now I have tried an expression task to build the full path before moving onto the data flow as well as just using the two variables assigned.
Data Flow (after FOREACH loop container)
Set up Excel File Source
Set up OLE DB Destination
Excel Source (inside Data Flow as the first task)
Set up the columns initially using one of the files (preview and input/output's are shown)
Note: Data is shown so the connection seems fine
Excel Source Connection Manager
DELAY VALIDATION = TRUE
EXPRESSIONS = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + #[User::excelFullPath] + "Extended Properties='EXCEL 12.0 XML;HDR=YES';"
ExcelFilePath = #[User::excelFullPath]
FirstRowHasColumnName = TRUE
Note: I have tried a combination of using the connection string and the excel file path property. I have tried those on their own.
Note: I have two Excel files in the folder for the moment. They are identical with different file names and YES the columns are the same.
Errors Received
The latest error upon RUN is:
SSIS package "C:\Users\biadmin\Desktop\0006AssocImport\0006AssocImport\Package.dtsx" starting.
Information: 0x4004300A at Data Flow Task, SSIS.Pipeline: Validation phase is beginning.
Error: 0xC0202009 at Data Flow Task, Excel Source 1 [2]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E37.
Error: 0xC02020E8 at Data Flow Task, Excel Source 1 [2]: Opening a rowset for "MAIN$" failed. Check that the object exists in the database.
Error: 0xC004706B at Data Flow Task, SSIS.Pipeline: "Excel Source 1" failed validation and returned validation status "VS_ISBROKEN".
Error: 0xC004700C at Data Flow Task, SSIS.Pipeline: One or more component failed validation.
Error: 0xC0024107 at Data Flow Task: There were errors during task validation.
Warning: 0x80019002 at Foreach Loop Container: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (6) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
Warning: 0x80019002 at Package: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (6) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "C:\Users\biadmin\Desktop\0006AssocImport\0006AssocImport\Package.dtsx" finished: Failure.
The program '[6272] DtsDebugHost.exe: DTS' has exited with code 0 (0x0).
Another Error Received Earlier
SSIS package "C:\Users\biadmin\Desktop\0006AssocImport\0006AssocImport\Package.dtsx" starting.
Information: 0x4004300A at Import RAW Excel Data, SSIS.Pipeline: Validation phase is beginning.
Error: 0xC0202009 at Package, Connection manager "Excel Connection Manager": SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft Access Database Engine" Hresult: 0x80004005 Description: "Could not find installable ISAM.".
Error: 0xC020801C at Import RAW Excel Data, Excel Source [2]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Excel Connection Manager" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
Error: 0xC0047017 at Import RAW Excel Data, SSIS.Pipeline: Excel Source failed validation and returned error code 0xC020801C.
Error: 0xC004700C at Import RAW Excel Data, SSIS.Pipeline: One or more component failed validation.
Error: 0xC0024107 at Import RAW Excel Data: There were errors during task validation.
Warning: 0x80019002 at Foreach Loop Container: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (5) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
Warning: 0x80019002 at Package: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (5) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "C:\Users\biadmin\Desktop\0006AssocImport\0006AssocImport\Package.dtsx" finished: Failure.
These are not all the errors received (I did not record all of them) but at the moment its just failing on one particular error at a time.
Note
I have seen quite a few YouTube tutorials and read a few outlining the exact steps. I have seen one where they set the connection manager from a Script task before getting to the Data Flow task. I saw one where the ConnectionString property was set only to the variable that holds the Excel File Path.
Questions
What am I missing in my set up
After configuring the connection manager to look at a variable for the file is there anyway of still configuring inputs and outputs on the Excel Source task because at the moment its showing an error
What is the developer preference in SSIS for this sort of situation where I prefer to see the column mappings on the configuration box (Script or Flow)
Related
I am running an SSIS package that uses OLEDB connections to fetch data from and Oracle 10.2.0.5.0 database and insert it into an SQLServer 2012 (11.0.6260)
I get the following error.
Time: 2021-02-05 01:13:55 AM
Task: Data Flow Task
Error Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
followed by
Time: 2021-02-05 01:13:55 AM
Task: Data Flow Task
Error Description: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on OLE DB Source returned error code 0xC0202009. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
There are no keys or constraints on the destination table.
Both of the dataflow tasks have CommandTimeout set to 0.
All of the other packages with the same source and destination databases run without any problem.
Edit 2
This particular package runs fine on our test/development environment.
I have sinced run the package several times and noticed that the source query returns about 4500 records but only about 500 are inserted into the destination database.
End Edit 2
Is there anything else that I can check for?
Edit 1
The Error message is as it appears in service manager.
The log files just before the error are below.
The source Query is quite along running query. It takes approximately 9 minutes.
End Edit 1
Logging into the oracle database directly with the user that is used for our integrations and running the query gave the following error.
After increasing the size of the user profile the query returned data and the integrations ran correctly.
I have an SSIS package (SQL 2017) that is importing data from an Access database to SQL Database.
It errors out with the following error:
[Source - MaintenanceDetails [140]] Error: Failed to retrieve long data for column "JobNotes".
[Source - MaintenanceDetails [140]] Error: There was an error with Source - MaintenanceDetails.Outputs[OLE DB Source Output] on Source - MaintenanceDetails. The column status returned was: "DBSTATUS_UNAVAILABLE".
[Source - MaintenanceDetails [140]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "Source - MaintenanceDetails.Outputs[OLE DB Source Output]" failed because error code 0xC0209071 occurred, and the error row disposition on "Source - MaintenanceDetails" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on Source - MaintenanceDetails returned error code 0xC0209029. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
I've tried the following to resolve this based off other Stackoverflow questions which hasn't fixed the problem:
1) Setting a path for "BLOBTempStoragePath" and "BufferTempStoragePath"
2) Setting AutoAdjustBufferSize to False
3) Creating an Error Output to put failed errors into another tab
4) Setting "TypeGuessRows" to 0 for "Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\ClickToRun\REGISTRY\MACHINE\Software\Wow6432Node\Microsoft\Office\16.0\Access Connectivity Engine\Engines\Excel"
5) I have tried using the Import/Export Wizard within Management Studio and I get the same error message.
The Connections are using:
Native OLE DB\Microsoft Office 12.0 Access Database Engine OLE DB Provider
Native OLE DB\SQL Server Native Client 11.0
Does anyone have any other suggestions for what I could try to resolve this?
What I ended up doing was as ppijnenburg described. Once I exported the Access table to a comma separated TEXT file, SSIS had no issue dealing with it at all.
I suppose the moral of the story is importing from Access to SQL Server is messy.
This is my first question here and before posting this question I have checked all the previous questions same as these or related to this one but I'm unable to solve the problem despite applying the fixes suggested by other members. Please, help.
My scenario: I have put multiple excel sheets (in MS Excel 2007 format) in a single workbook. I have created a User variable with package scope that holds the first sheet name within a for each loop container.
What I want to do
: I want to read multiple excel sheets within a single workbook and then write them to a SQL table. I have been able to accomplish reading a single excel file and writing it to a SQL table but not multiple excel files.
Software installed:
I'm using SQL Server Enterprise edition R2 with BIDS in it.
Resolution steps that I have followed so far:
Have set the DelayValidation property in excel connection manager as True
Have used a Variable with package level scope and assigned it the first sheet name which in my case is named as 'Sheet1"
My connection string is "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" +#[User::SheetName]+ ";Extended Properties=\"EXCEL 12.0 XML;HDR=YES\";" which is evaluated as correct
Have set the Package property to use 32 bit validation as suggested in this forum as I'm using a 32 bit OS
I get no errors at the design time but when i start debugging the project, I get the following errors given below,
[Excel Source [1]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E37.
[Excel Source [1]] Error: Opening a rowset for "C:\Users\XYZ\Desktop\Dataset\BasicData.xlsx" failed. Check that the object exists in the database.
[SSIS.Pipeline] Error: "component "Excel Source" (1)" failed validation and returned validation status "VS_ISBROKEN".
[SSIS.Pipeline] Error: One or more component failed validation.
Error: There were errors during task validation.
Please, can anyone let me know what am i doing wrong and how do I fix these errors.
Thank you,
I'm trying to use a foreach loop container to import several excel files into a table, I specified the folder and the files I'm going to use (*.xls), after that I created a path variable then I went to the excel source connection and in the "Expression" property I selected the variable. I created the foreach container but, the excel source's connectionString only shows Provider=Microsoft.Jet.OLEDB.4.0;, so it means it's not taking the variable value. What can I do to solve that?
Note: I'm using SQL Server 2012
ADDITIONAL INFORMATION:
Error at Data Flow Task [Excel Source [20]]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Excel Connection Manager" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
Error at Data Flow Task [SSIS.Pipeline]: Excel Source failed validation and returned error code 0xC020801C.
Error at Data Flow Task [SSIS.Pipeline]: One or more component failed validation.
Error at Data Flow Task: There were errors during task validation.
Error at Package [Connection manager "Excel Connection Manager"]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E4D.
(Microsoft.DataTransformationServices.VsIntegration)
Check the link below where I answer mainly focus on RUN64BITS TO FALSE instead of TRUE.
Execel Source ERROR
Running a SQL 2012 SSIS Package via an MVC Website fails with:
The Script Task is corrupted.
and
There were errors during task validation.
Runs perfectly from with in BIDS.
To resolve it I set the Project Property:
Run64BitRunTime = False
Security = DontSaveSensitive
In the Package I set:
DelayValidation = True
Security = DontSaveSensitive
This still failed. I came across this which indicates that Script Tasks are the problem.
So I removed the script task and it worked straight away.
When I run package directly from Visual Studio then it doesn't throw error. When I execute the package via SQL Server Agent Job then it throws error:
Source: Fact_Invoice_Item PWBConsolidation [1] Description: SSIS
Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error
code: 0x80004005. An OLE DB record is available. Source: "Microsoft
SQL Server Native Client 10.0" Hresult: 0x00040EDA Description:
"Warning: Null value is eliminated by an aggregate or other SET
operation.". End Error Error: 2012-02-14 07:17:49.01 Code:
0xC0047038 Source: Fact_Invoice_Item SSIS.Pipeline
Description: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput
method on component "PWBConsolidation" (1) returned error code
0xC0202009. The component returned a failure code when the pipeline
engine called PrimeOutput(). The meaning of the failure code is
defined by the component, but the error is fatal and the pipeline
stopped executing. There may be error messages posted before this
with more information about the failure. End Error DTExec: The
package execution returned DTSER_FAILURE (1). Started: 7:14:35 AM
Finished: 7:17:50 AM Elapsed: 195.094 seconds. The package
execution failed. NOTE: The step was retried the requested number of
times (3) without succeeding. The step failed.
I have isolated a part that is causing the problem. In Ole Db Source I use query to get the data and when I remove the following line then the package is executed successfully via Sql Server Agent Job:
Checksum = Checksum(S.BrokerID, S.TeamID, II.DatabaseName, II.INVOICE, I.Invoice_Date, P.Expiration_Date, P.Effective_Date,
P.Binder_Effective, I.AGENCY, II.MARKET, P.POLICY, CT.LINE_OF_BUSINESS, II.Coverage_Type, SUB.INSURED,
LOB.Description, CT.Description, I.Date_Due, I.Installment, P.Installments, P.ENDORSES, I.Inv_Tdate, Inv_Acctcur,
INS.INDUSTRY, SUB.SUBMISSION
)
EDIT:
I have just checked and looks like part of data gets through data flow and at some point it just throws error. I have manually set Checksum=2 and still doesn't work
Any idea why is this so?
Thanks a lot,
Ilija
This looks like you are running into a constraint issue on your table that you are inserting into or values you are operating on...like aggregate function(max,sum,avg..) exists on a null value.
Try redirecting error rows to a flat file or something of that nature and see if you still get the error.
Also try set ANSI_WARNINGS OFF in your SQL statement
Enjoy and good luck