I am using SQL Server 2008 BIDS. I am trying to read in an Excel file, having multiple sheets. The names are mostly alphabetical( and few with special char '&'). The data starts at row 8. I have skipped the blank rows by setting the rows and columns in the open rowset property for the Excel source. I get the exact mappings. However, I am not able to preview the data. The package runs successfully(everything turns green), but there is no data in my destination.
The error I receive while I try to preview is:
There was an error displaying the preview.
Additional Information:
Index and Length must refer to a location within the string. Parameter
name: Length(mscorlib)
Please let me know if I am doing anything wrong or I am missing any settings.
The links I have referred to:
Skipping rows when importing Excel into SQL using SSIS 2008
https://connect.microsoft.com/SQLServer/feedback/details/557049/ssis-fails-to-preview-excel-source-connector-due-to-incompatible-sheet-name
Thanks
I cracked it with the help of one of my friend .
In the properties of the Excel Source >> Custom Properties >> Open Rowset >>
SheetName$A12:J
It means skip the rows till A12. And the data is taken into account from A12 though end of J column.
Problem solved.
The way to solve this is to use q SQL command in the data access mode.
Select * FROM [Sheet1$A20:K]
This will read the data correctly and no preview errors will occur.
I was getting this same error trying to import a .xlsx file into SSIS 2008. I first saved the file as .xls (Excel 97-2003). I was still getting the above error and couldn't preview the data with the new file.
In case this link breaks in the future: https://connect.microsoft.com/SQLServer/feedback/details/557049/ssis-fails-to-preview-excel-source-connector-due-to-incompatible-sheet-name
Once I removed the space from the Sheet name, I was able to preview the data. The tab was originally called Time Data; I changed it to TimeData.
Some other potential problems with the sheet name, according to the above site:
Sheets named with all numbers (ex: 4385)
Sheet names beginning with a number
In my case I was trying to export from SQL into an Excel file and receiving "Index and Length must refer to a location within the string" while trying to preview the destination data.
Removing the space from the Excel Sheet in the Excel Destination Editor fixed it for me.
After trying all the other answers with no luck, I renamed the spreadsheet.
In the Excel connection manager, I browsed to the renamed spreadsheet and unchecked first row contains column names. I was then able to view the data in the preview window.
Related
I am updating a table in SQL Server Management Studio from an Excel doc by right clicking on the table I want to update, hitting edit top 200 rows, and trying to paste in my new line of data.
However when I do this I am coming up with couple of issues. First of all, the ID column is showing as NULL. Second, the line is read only (that by itself really isn't a problem for me, but it doesn't seem right) and finally some of my columns that do have values are showing NULL.
When I try to import the excel spreadsheet (right click on database, tasks, import data) I get the error shown here.
You must to save excel file to Excel 97-2003 Workbook (*. xls).
Then let begin with mapping your fields source to destination table.
Using SSIS for Visual Studio 2017 for some excel file imports.
I've created a package with several loop containers that call to specific packages to handle some files. I have an issue with one particular package being executed in that it seemingly randomly decides the data for columns is NULL per excel file. I was/am under the impression that this is part of the registry setting for TypeGuessRows (changed initially to 0 then to 1000 as a test) located at
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel
The reason I think this is because the various files being brought in generally have the same data, but it seems that if the first few rows of columns in the source data contains only numbers, that the data with mixed values will not be brought in correctly. All other columns aside from this seems fine.
Looking at the source files, all have the same datatype.
I've tried changing the registry TypeGuessRows value and ensured that the output column property was string-based instead of numerical.
The connection string has IMEX=1
So I fixed it. Or at least found a sufficient workaround that should help anyone in my situation. I think it has to do with the cache of SSIS.
I ended up putting a sort function on the problem column so the records getting read as NULL for having a random data type are read first, and not being considered random. I will say, I tried this initially and it didn't work.
Through a little experiment of making a new data flow in the same package I discovered that this solution actually does work, hence me thinking the cache was the issue.
If anyone has any further questions on this, let me know.
This issue is related to the OLEDB provider used to read excel files: Since excel is not a database where each column has a specific data type, OLEDB provider tries to identify the dominant data types found in each column and replace all other data types that cannot be parsed with NULLs.
There are many articles found online discussing this issue and giving several workarounds (links listed below).
But after using SSIS for years, i can say that best practice is to convert excel files to csv files and read them using Flat File components.
Or, if you don't have the choice to convert excel to flat files then you can force excel connection manager to ignore headers from the first row bu adding HDR=NO to the connection string and adding IMEX=1 to tell the OLEDB provider to specify data types from the first row (which is the header - all string most of the time), in this case all columns are imported as string and no values are replaced with NULLs but you will lose the headers and a additional row (header row is imported).
If you cannot ignore the header row, just add a dummy row that contains dummy string values (example: aaa) after the header row and add IMEX=1 to the connection string.
Helpful links
SSIS Excel Data Import - Mixed data type in Rows
Mixed data types in Excel column
Importing data from Excel having Mixed Data Types in a column (SSIS)
Why SSIS always gets Excel data types wrong, and how to fix it!
EXCEL IN SSIS: FIXING THE WRONG DATA TYPES
IMEX= 1 extended properties in ssis
Good day to you, Experts.
I'm stuck on a problem I'm having with an Excel 97-02 .xls file.
When adding it as a source in SSIS, I'm getting an External Columns Datatype of DT_IMAGE .
The column represents an ID and is numeric only. I can't extract and work with the data because of the DT_IMAGE datatype.
Setting IMEX=1 didn't help.
Thank you in advance.
Reading Excel files in SSIS is done using OLEDB provider which may not detect the appropriate Excel column type.
There are many other questions mentioning similar issues such as:
SSIS Excel Import Forcing Incorrect Column Type
SSIS Excel Data Source - Is it possible to override column data types?
SSIS keeps force changing excel source string to float
As you mentioned in the question, if you added ;Extended Properties="IMEX=1" to the connectionstring with no luck then i think there is 4 things you can try:
Sorting column data inside Excel
Change the entire column formatting manually
Go to the advanced editor on the Excel source >> into the output column list and set the type for each of the columns.
Adding IMEX=1; MAXROWSTOSCAN=0 to the connectionstring
If nothing of the above steps worked then you should save the Excel sheet as a text file and then you use Flat File Connection manager
I am taking a view in a SQL DB and placing it into a CSV file using SSIS. Before doing so, I convert everything to unicode, which gives me two of everything. I was not having this issue until I recently made the change to append a date to the end of my output file by using an expression. I am receiving duplicate rows and everything is just pasted accross twice. Any suggestions to only get them to come out once on the CSV? Image below.
I eventually figured it out. When I changed my connection string to my flat file to be a function, it reset my columns to have duplicate columns. I just had to go into my connection manager for the flat file and delete the extra columns.
I am trying to export data from SQL server 2008 to Excel file using BIDS.
One of the fields 'DESCRIPTION' coming from SQL database is VARCHAR(4000).
I can export everything to excel but the 'DESCRIPTION' field size in excel is restricted to unicode 255 and no mater what I try it does not allow me to export the data over 255 characters (exports it as blank). I tried to change SQL field as varchar(max) or ntext but none of attempts worked. I used advanced editor in BIDS on excel destination to change 'DESCRIPTION' character length manually but as soon as I hit 'OK', it resets to unicode 255.
Could anybody please help me to resolve this issue?
Thanks,
Vishal
So, I did some testing. Excel data transformation is funky but I came up with a solution. I created an excel spreadsheet with fields as needed. I then created fake, dummy data in excel with character length far greater than 255 and hid the row. I then did the SSIS data transformation to the excel spreadsheet which worked. It's a weird and not preferable option but it works.
Problem: Excel only accepts 255 chars per cell when I attempt to use Excel Destination in SSIS (2008 R2) from a sql server table. SalesForce data loader would not accept CSV (with “” text qualifiers) created by
ssis flat file connection manager. SalesForce will only accept CSV (with “” text qualifiers). SalesForce will accept CSV as exported by Excel (2010).
Solution:
1. Create your excel connection manager, set name/path of the destination EXCEL file in your “Excel Destination Data Flow Competent” and map meta-data.
2. Open a new Excel file, remove all extra “sheets”, rename “sheet1” to that was created in step#1, above, select all cells and format to “text”, add all the column header names to the first row of your template sheet. In the columns that need to hold more data than 255 limit, paste in any characters that exceed your limit by 50% (just in case). These columns are now configured to hold your large data. Save the file, naming it something like TEMPLATE_Excel_forLargeCellValues.xlsx
3. Copy this template into your DESTINATION connection: Before your “Excel Destination Data Flow Competent” in the SSIS Control Flow, create a new “File System Task”. Create an ssis pkg level variable to hold the path/filename of your template excel file. In your “File System Task” set “IsSourcePathVariable” = TRUE, set “SourceVariable” to User::Template_Excel. Set “IsDestinationPathVariable” = FALSE, and set “DestinationConnection” = from step #1 above. Set “Operation” = Copy file. “OverwriteDestination”=TRUE. This will now copy your formatted Excel workbook/sheet into your destination folder with the file name you designated in step #1 above and because you put a larger amount of sample data in the columns that require more than 255 chars, all your data will fit.
Note: It is not necessary to delay validation on any components.
You're saying that the excel field is set to 255 right? Changing the SQL field won't have an effect on excel, you'd have to modify the excel file.
I don't believe you can modify the Excel output column to write more than 255 characters. Why not simply write your output to a csv, it can be opened and later modified in Excel anyway.
SSIS excel engine recognizes datatype of first 8 rows and assigns it to excel source or destination automatically. Even defining the excel column as memo wont work. I tried to resolve the error by changing registry value TypeGuessRows of excel engine but it did not work either. So I was not left with any other option but to create a dummy row(2nd row) with more than 255 characters and hide it.Excel source then identify the column with unicode text stream. You have to write some logic in SSIS package to exclude this row if you are trying to import the data from excel. I heard that this issue is resolved in excel versions on and after 2010. But BIDS 2008 does not have option to choose any version after 2007 so this is the only solution if you are working with BIDS 2008 and excel.
You have to select Microsoft Excel 97-2003 and use the xls as file extension in your file name for destination.
I got the same issue of the excel destination not allowing more than 255 characters. After spending almost a day, I tried adding more characters (to simplify, I added spaces more than 255) in the header of the column that has the issue with more than 255 characters. And it magically worked!
You can insert dummy data (260 characters) to under head column you want in your excel (Execute SQL Task)
Script Create and insert
CREATE TABLE `YourSheet` (`myColumn260char` LongText)
GO
INSERT INTO YourSheet(myColumn260char) Values('....................................................................................................................................................................................................................................................................')
And you can delete dummy row after imported.