SSIS Multiple File Load Error with Variable - sql-server

I have an SSIS package that is to load multiple Excel files using the For Each Loop Containter. As soon as I add the variable in the Excel Connection manager in expressions, under ExcelFilePath Property it throws an error in the Excel Source Package:
An OLEDB error has occurred 0X80004005. The MIcrosoft Access
Database Engine cannot write to the file "C:..."
It is already opened exclusively by another user or you need permission. I had to already download the 32bit Extension to be able to view the Excel files. I've followed many guides on how to do this, and it always ends up in the same result. Delay validations are set to True. Any idea how to fix?

Related

SSIS Foreach Loop Container to read files and load into DB getting crash during execution

I'm trying to load multiple files from a location into DB using Foreach Loop Container & DataFlow task in SSIS.
It's getting crashed while I try to execute the package. It's not giving any error message, whenever I execute the package it crashes and closes the visual studio app immediately. I have to kill the debug task in the task manager for the next execution of the package.
So I tried the below steps:
I used a FileSystem task instead of DataFlow task to just
move all the files from the source to the archive directory, which ran
fine without any issues.
Ran the DataFlow task individually to load a single file into DB,
which was also executed successfully.
I couldn't figure out what was going wrong here. Any help would be appreciated! Thanks!
Screenshots
All screenshots look fine to me. I will give some tips to try to figure out the issue.
Since the File System Task is executed without any problem, there is no problem with the ForEach Loop Container. You can try to remove the OLE DB Destination and replace it with a dummy task to check if it causing the issue. If the issue remains, it means that the Flat File Source could be the cause.
Things to try
Make sure that the TargetServerVersion is accurate. You can learn more about this property in the following article: How to change TargetServerVersion of my SSIS Project
Try running the package in 32-bit mode. You can do this by changing the Run64bitRuntime property to False. You can learn more about this property in the following article: Run64bitRunTime debugging property
Running Visual Studio in safe mode. You can use the following command devenv.exe /safemode.
Workaround - Using Bulk Insert
Since you are inserting flat files into the SQL database without performing any transformation. Why not use the SSIS Bulk Insert Task. You can refer to the following step-by-step guide for more information:
SSIS Basics: Bulk-Import various text files into a table
As mentioned in the official documentation, make sure that the following requirements are met:
The server must have permission to access both the file and the destination database.
The server runs the Bulk Insert task. Therefore, any format file that the task uses must be located on the server.
The source file that the Bulk Insert task loads can be on the same server as the SQL Server database into which data is inserted, or on a remote server. If the file is on a remote server, you must specify the file name using the Universal Naming Convention (UNC) name in the path.

SSMS - Running SSIS package via batch file - resulting table empty

Currently I am building a database in SQL Server Management Studio (SSMS). Several data sources are imported in this database, via the import/export wizard. Upon completion of the import, not only am I running the import, I also save an SSIS package. This SSIS package is to schedule a weekly refresh of the data source.
On my Windows Server 2012 R2 the Express edition is installed, therefore I have manually created several batch files that run every week (as scheduled via Task Scheduler). This works fine for most tables, however for some tables I encounter some strange (?) behaviour.
This is as follows: when creating the SSIS package via import/export wizard, and directly running the import, the table shows up correctly in the database. That is, with all column names and the thousands of rows it contains.
The strange thing is that, when executing the SSIS package (via the batch file), the table is empty (column names are correct though). For some tables, I do not encounter this behaviour. For others, this behaviour is encountered all the time.
The batch script is as follows (quite straightforward):
"C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\DTExec.exe" /F "C:\Users\username\ssispackage.dtsx"
The batch file seems to run correctly at all times, as the table 'creation_date' changes when I run the batch file. Moreover, for all the tables that do correctly 'refresh', these same batch files do the job.
Some settings of the SSIS package:
Data source: Oracle OLE DB provider
Destination: SQL Server Native Client / MS OLE DB Provider for SQL Server (tried both)
Data via query (as I am querying several tables from Oracle); query is parsing correctly
Mappings: Create destination table & Drop and re-create destination table
Dropping and re-creating is done, because the data source is rather small, and has some changes weekly/monthly (to some rows).
For most data sources imported (and refreshed weekly) via this method, the tables are correctly showing each week (simply dropping the previous table, and re-creating the source).
I hope someone can explain to me why this issue occurs. If some more information is needed from my side, please ask.
Thanks in advance!
UPDATE:
When looking at the log of the batch file, this is (part) of the output:
Source: ..... "SourceConnectionOLEDB"
Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "OraOLEDB" Hresult: 0x80004005 Description: **"ORA-01005: null password given; logon denied".**
End Error
.....next error.... "SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.
Thus, it seems that the password is not remembered/saved correctly in the SSIS package?
This is strange though, as for most tables it does correctly store the password (as those do refresh correctly).
When setting the properties of the data source, namely Oracle Provider for OLE DB, I select the option "Allow saving password". So it should store the password correctly?
Found the answer after all.. The .dtsx file that is saved (the SSIS package) contains the variables for the connection string, it shows that the Password (Sensitive="1") is there. But in the wizard, I did not select 'Save sensitive data with user key'. When selecting this option, an encryption string was added. Now the SSIS packages run well!

Getting error on creating a package in SSIS VS2017

I am using VS2017 and SSMS 2017 and imported a CSV file in FLAT FILE option in Data Flow tab in SSIS and also created anADO NET SOURCE database connection through my local server that stores the data in my database. But when am trying to create a connection between FLAT FILE and ADO NET SOURCE, by dragging the blue arrow from file option to ADO NET option, am getting this error. Can anybody provide some input on how to get rid of this error?
enter image description here
You are attempting to route the flat file source into another source component, which cannot be done. Did you mean to route the data into a destination?
Error is that you have connected Source to Source.
Fix it by connecting Source to Destination, so that the package can build successfully.
Replace your ADO.NET SOURCE with ADO.NET DESTINATION and this error will be corrected.

SSIS Package stopped working with Error code: 0x80004005

I got this SSIS package working this past December. It only runs on Friday mornings. Last Friday it failed with this error message:
Package:Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft Access Database Engine" Hresult: 0x80004005 Description: "The Microsoft Access database engine cannot open or write to the file '\\ServerShare\IT\Reports\Export Templates\YoderReport.xlsx'. It is already opened exclusively by another user, or you need permission to view and write its data.".
I've checked out a couple of other questions that were similar, but they did not answer my question. I have checked to make sure that no one has that file open.
The file in question is a template that is copied over then populated, so no one should have it open, to begin with.
I've tried changing the RetainSameConnection to True, but no difference. I have run it in debug mode, and it works fine.
Anyone got any ideas how to clear this up so it runs automatically again?
UPDATE
After some more testing, it appears that the file is getting the data, but isn't being copied. Here is what I have setup:
I have a File System Task that copies a template from my template
folder to my Export folder.
Then I have a Data Flow Task
Begins with a OLE DB Source that runs a SQL script to pull data
Runs a Data Conversion to update a couple of fields to the correct format
Excel Destination is used for the output. (This is the template that was copied to the Export folder
There is also a Flat File Destination just in case there are any errors
Then back to the Control Flow with another File System Task, this one moves the file from the Export folder to its final destination on a shared drive
When I run this from VS 2015 it works fine and creates the file. When I run this from the SQL Agent job it fails with the above error message. The only thing that I can think of is that in the Data Flow Task the Excel Destination isn't releasing the file before the final File System Task tries to copy it? But if that is the case, why did that just start happening now?
I think that there is a problem in the filepath, as you can see the file path start with one single backslash:
\ServerShare\IT\Reports\Export Templates\YoderReport.xlsx
Maybe it should start with double back slash \\ if it is on the network
\\ServerShare\IT\Reports\Export Templates\YoderReport.xlsx
Or the Partition is missed --> Incomplete path
Check the way that the file path is provided to the connection manager (if you are using expressions)
Also based on this microsoft article, there is two possible causes:
You must have permission to read data in the specified file in order to view its data. To change your permission assignments, see your system administrator or the table or query's creator.
You tried to open the indicated file exclusively, but another user already has the file open.
Looks like an access issue. Ensure that the Agent Service Account is running with full rights to the network share path. Maybe you can try with your credentials on the Agent Server.
Also, Ensure that your Excel destination connection string supports .xlsx.
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=c:\path\xxx.xlsb;Extended Properties="Excel 12.0;HDR=YES";
Changing “Excel 12.0” to “Excel 12.0 Xml” will tell the provider to output in .xslx format instead.

SSIS is overwriting excelfilepath expression with user variable on first pass through for each loop

I have searched everywhere for an answer to this so I am hoping someone out there can help.
I am trying to follow the steps laid out for importing data from multiple excel files in SSIS. I have several excel 2010 files in a directory and am trying to move them into a SQL Server 2008 r2 Database. I have followed all of the directions for doing this with a For Each (File) Loop. I have set the collections information and am using the User::Filename variable to pull back the Fully Qualified filename. The problem I have is that when I check the properties of my excel connection manager-->click on expressions-->modify excelfilepath by setting it to #[User::Filename], SSIS immediately overwrites the property with the new value which is blank (the first time). This then causes my package to fail during execution because there is no filename to go to in the connection.
I have set the delayvalidation property to True on the package, the dataflow and the excel connection and this does not fix the problem. I have also tried to put a fully qualified filename into the User::Filename variable during initialization and this does allow me to process a few files but then gives me a locking error when SSIS tries to re-read the file that I put in during initialization.
I have not found anything like this on the net yet...Hopefully, someone out there has seen this.
I would split this SSIS package into two. The master package would go as deep as the For Each File Loop, but would not include the Excel Connection Manager. The sub-package would open and process each Excel file. It would receive the Filename variable value via Configuration for a Parent Package Variable or similar.
This seems to have the effect of "closing" the excel connection after each file is finished.
I also prefer the OLE DB driver from the Access Database Engine over the native SSIS Excel connection.

Resources