I have build a SSIS package where all .xlsb files are pulled from a folder and updated to a temp file. I had built the package with the existing excel files I had, it worked fine and now I have received a new data set in xlsb only but I see few columns getting update as #NA. Also I get error
Also with the new data I got error saying
VS_NEEDSNEWMETADATA
so I turned the data validation to false
The VS_NEEDSNEWMETADATA error is probably because you changed something in your schema. just double click on your inputs/outputs, click OK and it should resolve itself.
Have you checked what columns/fields are getting updates as #NA? Most likely it are fields that are empty in the xlsb and could be replaced in your SSIS package with null by using a derived column.
If you are using Excel Connection manager, try below things:
In source task, test with both the options (true/false) of 'Validate external Metadata'
If your excel column contains combined values of characters, numbers and alphanumeric, add IMEX=1 to the extended properties of excel connection manager.
https://www.youtube.com/watch?v=s3d-XIYuB7k
Related
Currently I am building a database in SQL Server Management Studio (SSMS). Several data sources are imported in this database, via the import/export wizard. Upon completion of the import, not only am I running the import, I also save an SSIS package. This SSIS package is to schedule a weekly refresh of the data source.
On my Windows Server 2012 R2 the Express edition is installed, therefore I have manually created several batch files that run every week (as scheduled via Task Scheduler). This works fine for most tables, however for some tables I encounter some strange (?) behaviour.
This is as follows: when creating the SSIS package via import/export wizard, and directly running the import, the table shows up correctly in the database. That is, with all column names and the thousands of rows it contains.
The strange thing is that, when executing the SSIS package (via the batch file), the table is empty (column names are correct though). For some tables, I do not encounter this behaviour. For others, this behaviour is encountered all the time.
The batch script is as follows (quite straightforward):
"C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\DTExec.exe" /F "C:\Users\username\ssispackage.dtsx"
The batch file seems to run correctly at all times, as the table 'creation_date' changes when I run the batch file. Moreover, for all the tables that do correctly 'refresh', these same batch files do the job.
Some settings of the SSIS package:
Data source: Oracle OLE DB provider
Destination: SQL Server Native Client / MS OLE DB Provider for SQL Server (tried both)
Data via query (as I am querying several tables from Oracle); query is parsing correctly
Mappings: Create destination table & Drop and re-create destination table
Dropping and re-creating is done, because the data source is rather small, and has some changes weekly/monthly (to some rows).
For most data sources imported (and refreshed weekly) via this method, the tables are correctly showing each week (simply dropping the previous table, and re-creating the source).
I hope someone can explain to me why this issue occurs. If some more information is needed from my side, please ask.
Thanks in advance!
UPDATE:
When looking at the log of the batch file, this is (part) of the output:
Source: ..... "SourceConnectionOLEDB"
Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "OraOLEDB" Hresult: 0x80004005 Description: **"ORA-01005: null password given; logon denied".**
End Error
.....next error.... "SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.
Thus, it seems that the password is not remembered/saved correctly in the SSIS package?
This is strange though, as for most tables it does correctly store the password (as those do refresh correctly).
When setting the properties of the data source, namely Oracle Provider for OLE DB, I select the option "Allow saving password". So it should store the password correctly?
Found the answer after all.. The .dtsx file that is saved (the SSIS package) contains the variables for the connection string, it shows that the Password (Sensitive="1") is there. But in the wizard, I did not select 'Save sensitive data with user key'. When selecting this option, an encryption string was added. Now the SSIS packages run well!
I have an SSIS package that is to load multiple Excel files using the For Each Loop Containter. As soon as I add the variable in the Excel Connection manager in expressions, under ExcelFilePath Property it throws an error in the Excel Source Package:
An OLEDB error has occurred 0X80004005. The MIcrosoft Access
Database Engine cannot write to the file "C:..."
It is already opened exclusively by another user or you need permission. I had to already download the 32bit Extension to be able to view the Excel files. I've followed many guides on how to do this, and it always ends up in the same result. Delay validations are set to True. Any idea how to fix?
I created a Project within SSIS it would take data from a table within a db, do a data conversion and then insert into a Excel spreadsheet. it was working fine with no issues for about a week but now I'm having issues with th e connection manager. I'm getting the data fine from the DB its when it comes to the excel spreadsheet. I have tried doing the project again but still doesn't work. I create a new excel destination within SSIS > a new connection manager go to folder destination and state what I want document to be called. nothing is available from the name of excel sheet drop down and when I select NEW it just closes window. I'm not sure what has gone wrong is it a permissions thing? I have attached screenshot of this and the error when I run application
error when deploy application
i think something must have changed with access to the db. if I use the import wizard and select OLE DB for SQL server as the Datasource nothing is appearing in the server name drop down and If I manual enter it I get an error
Try this:
Step 1) Navigate to Project-> [PROJECT_NAME] Properties.
Step 2: Navigate to “Debugging” option from left panel and from Right panel change Run64BitRuntime value to false.
I am using SSIS to insert data from flat file to database.
I have created Data Flow Task for that. I am using Flat File as Source and ADO NET Destination to insert data.
Below is how my setting looks like for Flat File Source.
Below is how my "Columns" tab look like
THIS WORKS FINE WHEN I RUN THAT USING BIDS AND DATA IS INSERTED PROPERLY INTO DATABASE. IT EVEN WORKS WITH DTEXEC.EXE WHEN RUN LOCALLY.
Now, The problem is with executing the package on the server using dtexec.exe. On the server, data is inserted properly but the text qualifier (double quotes) given is totally ignored while inserting data to database. THE SAME WORKS TOTALLY FINE WHEN RUN LOCALLY. I have attached image below for how its stored in database.
I have checked SQL SERVER version and SSIS version locally and on remote server and both are same.
What can be the problem? Can anyone help?
So I found a solution for this problem. Thanks to LukeBI answer here
Create a string variable called TextQualifier and assign the value " (double quotes)
Select the connection manager, and in the Properties window select 'Expressions'. See below.
Click ..., add the property 'TextQualifier' and assign the variable #[User::TextQualifier]. See below image
Now its working fine. It will even work on 64 bit OS now.
In the flat file source, click "Columns". Make sure that when you preview the data there are no quotes in the preview. Otherwise you may have to look back at your file and make sure that BOTH the text qualifier and delimiter are correct.
If this does not work, then please take a screenshot of the "Columns" screen as well and post it. A screenshot of the actual file layout would help as well. Hope this helps!
Within your flat file connection Manager, within the "Advanced" option, you should be given a view of different parameters for each field. You will see for each field it will have a Name, ColumnDelimiter, a bunch of faded out fields and the DataType and a choice if it is text qualified or not.
In there, you should specify that the column(s) which you wish to be determined as text qualified by setting the TextQualified option to true.
I have had Packages in the past where I was looping through multiple Text files in a folder and loading into sql server tables.
Now I am asked to create a package which will loop through Multiple Excel Files in a folder and load them into sql server table.
I went through the following steps to create this package assuming it shouldn't be much different from what I have in other packages where it loops through multiple Flat file.
Added an Execute Sql Task, Truncating my staging table, A simple Truncate table statement.
Added a Foreach Loop Container. Selected Foreach File Enumerator and created a variable called File_Path with data type string.
Added a Data Flow Task.
Added an Excel Data Source. and configured the Excel Connection manager By selecting any one 'Excel' File in the destination folder. (At this point is configured correctly as it is not showing any red cross or warring messages.)
Then I selected the Excel File Connection Manager and in Properties windows Under Expressions, Selected Connection String property and Used the User Variable #User::File_Path.
At this point the Excel Data source is showing a Red Cross as it needs further configuration.
I have tried a few things Like changing the Data Access Mode from Table name to Table Name or View Name Variable, And passing variable #User::File_Path but it gives me the following error.
Can someone please have a look and advice where I am going wrong and how I can fix this? Any Advice or a pointer in the right direction is much appreciated.
Thank you.
You shouldn't use an expression on the ConnectionString property, but on the ExcelFilePath property.