We have over 400 Excel files, that connect to a specific SQL server's databases. Currently we are undergoing an upgrade and the Server will change. The db names and tables will remain the same, so the only change in the connection property will be the server name.
I am looking for a quick way to do this using any text editor, and replacing the old name with the new one.
I've tried Notepad++ and EmEditor opening the .xls file as Binary (Hex view) and tried to replace the Hex equivalent of the ASCII characters with no success.
Also, I tried opening in non Binary view, but after saving many funtionalities of the excel are lost and I also get a message that the file is not a valid xls file.
It was successfully done by using UltraEdit, and replacing the string by selecting 'Replace in all files' and choosing the entire folder. So it seems that it can be done with a Text Editor, that doesn't open the file but instead it searches and replaces all instances of the string in a selected folder.
Thank you all for the contribution.
Related
I am new to SSIS, I have already designed a package and configured SSIS Log provider for Text Files.
This works fine and log files are generated successfully.
We have a monitoring team, they use this log file for monitoring. They are unable to read the log files since the file encoding is in Unicode format.
They are expecting a non unicode format for their monitoring.
I tried to change the existing log file encoding to ANSI but when I re-run the package my log file has been created again with UNICODE encoding.
Is any way we can create log files using SSIS Log provider for Text Files with non unicode encoding. Kindly suggest me any workaround. I am unable to find solution for the past two days.
Trying to figure out the issue
Since SSIS Log provider for Text Files use a File connection manager for logging purposes, you don't have the choice to edit the file encoding within the SSIS package because this type of connection manager can be used for different files format (excel, text ...).
While searching for this issue it looks like if the log is created for the first time by SSIS it will write unicode data.
why are my log files getting generated with a space between every two characters?
Why is my SSIS text logfile formatted in this way?
Possible workaround
Try to create an empty text file using notepad and save it with ANSI encoding.
Then select this file from the SSIS logging configuration.
Other helpful links
Change the default of encoding in Notepad
Add Logging with SSIS
Update 1 - Experiments
To test the workaround i provided i have run the following experiments:
I add SSIS Logging and created and a new log file
After executing the package the file is create in Unicode (to check that i opened the file using notepad and click Save As the encoding shown in the combobox is Unicode)
I create a new file using Notepad and save it using Ansi encoding as mentioned above.
In SSIS i changed the File connection manager to Use Existing instead of Create New and i selected the file i created
After executing the package the log is filled within the file and the encoding is still Ansi
I repeated executing the package several times and the undoing wont changes.
TL DR: Create a file with ANSI encoding outside the ssis package and within the package create a file connection manager, select Use Existing option and choose the created file. Use this file connection manager for logging purposes.
I have multiple packages for each location site and each site contains 5 other sub sites for which data has to be populated every month. Hence I have approx 20 excel files as a source data for which the excel file acts as a source and I need to create an excel destination. Because of so many excel file as a source I have multiple Excel connection Managers (Source + Destination) across my various packages.
The source file name and the location will change everytime and so I was thinking to automate the file path via parameter. I follow the steps mentioned in the article: https://blogs.msdn.microsoft.com/sqlgardner/2015/06/18/ssis-tip-using-a-file-path-parameter-for-multiple-flat-files/
I followed every step but I got the error when I open the excel file (source). Below is the error snapshot:
Can someone guide me what exactly I am doing wrong here?
Looks like you're using an expression on the excel connection manager "ConnectionString"
property. When that property is changed to just a file path that isn't a valid connection string for that type of connection.
With excel file connections it's a little different and you'll want to use the ExcelFilePath property, which in turn is used by the connection manager when building out the connection string.
With Flat File Connections, yes you would use the ConnectionString property, Excel Connections, slightly different, use the ExcelFilePath property.
In an SSIS package that I'm writing, I have a CSV file as a source. On the Connection Manager General page, it has 65001 as the Code page (I was testing something). Unicode is not checked.
The columns map to a SQL Server destination table with varchar (among others) columns.
There's an error at the destination: The column "columnname" cannot be processed because more than one code page (65001 and 1252) are specified for it.
My SQL columns have to be varchar, not nvarchar due to other applications that use it.
On the Connection Manager General page I then change the Code page to 1252 (ANSI - Latin I) and OK out, but when I open it again it's back to 65001. It doesn't make a difference if (just for test) I check Unicode or not.
As a note, all this started happening after the CSV file and the SQL table had columns added and removed (users, you know.) Before that, I had no issues whatsoever. Yes, I refreshed the OLE DB destination in the Advanced Editor.
This is SQL Server 2012 and whichever version of BIDS and SSIS come with it.
If it is a CSV file column text stream [DT_TEXT] to SQL varchar(max) data type that you want to convert to, change the flat file Connection Manager Editor property Code page to 1252 (ANSI - Latin I).
65001 Code page = Unicode (UTF-8)
Based on this Microsoft article (Flat File Connection Manager):
Code page
Specify the code page for non-Unicode text.
Also
You can configure the Flat File connection manager in the following ways:
Specify the file, locale, and code page to use. The locale is used to interpret locale-sensitive data such as dates, and the code page is used to convert string data to Unicode.
So when the flat file has a Unicode encoding:
Unicode, UTF-8, UTF-16, UTF-32
Then this property cannot be changed, it will always return to it original encoding.
For more infor about the Code Page identifiers, you can refer to this article:
Code Page Identifiers
I solved this in SSIS through Derived Column Transformation
If it's a csv file, you can still use code page 1252 to process it. When you open the flat file connection manager it shows you the code page for the file, but you don't need to save that setting. If you have other changes to make in the connection manager, change the code page back to 1252 before you save the changes. It will process fine if there are no unicode characters in the file.
I was running into a similar challenge, which is how I ended up on this page looking for a solution. I resolved it using a different approach.
I opened the csv in Notepad++. One of the menu options is called Encoding. If you select that, it will give you the option to "Convert to ANSI."
I knew that my file did not contain any Unicode specific characters.
When I went back to the SSIS package, I edited the flat file connection and it automatically changed it to 1252.
In my case the file was generated in Excel and (mistakenly) saved as CSV UTF-8 (Comma delimited) (*.csv) instead of simply CSV (Comma delimited) (*.csv). Once I saved the file as the correct form of CSV, the code page no longer changed from 1252 (ANSI - Latin I).
I am using SSIS to insert data from flat file to database.
I have created Data Flow Task for that. I am using Flat File as Source and ADO NET Destination to insert data.
Below is how my setting looks like for Flat File Source.
Below is how my "Columns" tab look like
THIS WORKS FINE WHEN I RUN THAT USING BIDS AND DATA IS INSERTED PROPERLY INTO DATABASE. IT EVEN WORKS WITH DTEXEC.EXE WHEN RUN LOCALLY.
Now, The problem is with executing the package on the server using dtexec.exe. On the server, data is inserted properly but the text qualifier (double quotes) given is totally ignored while inserting data to database. THE SAME WORKS TOTALLY FINE WHEN RUN LOCALLY. I have attached image below for how its stored in database.
I have checked SQL SERVER version and SSIS version locally and on remote server and both are same.
What can be the problem? Can anyone help?
So I found a solution for this problem. Thanks to LukeBI answer here
Create a string variable called TextQualifier and assign the value " (double quotes)
Select the connection manager, and in the Properties window select 'Expressions'. See below.
Click ..., add the property 'TextQualifier' and assign the variable #[User::TextQualifier]. See below image
Now its working fine. It will even work on 64 bit OS now.
In the flat file source, click "Columns". Make sure that when you preview the data there are no quotes in the preview. Otherwise you may have to look back at your file and make sure that BOTH the text qualifier and delimiter are correct.
If this does not work, then please take a screenshot of the "Columns" screen as well and post it. A screenshot of the actual file layout would help as well. Hope this helps!
Within your flat file connection Manager, within the "Advanced" option, you should be given a view of different parameters for each field. You will see for each field it will have a Name, ColumnDelimiter, a bunch of faded out fields and the DataType and a choice if it is text qualified or not.
In there, you should specify that the column(s) which you wish to be determined as text qualified by setting the TextQualified option to true.
I am facing an issue with SSIS where a customer wants a (previously delivered file in UTF-8) to be delivered in ANSI-1252. No big deal i thought. change the file connection manager and done... unfortunately it wasn't that simple. Been stuck on this for a day and clueless on what to try next.
the package itself
IN - OLE DB source with a query. Source database fields are NVARCHAR.
Next i have created a Data conversion block where i convert the incoming DT_WSTR to DT_STR using 1252 codepage.
After that is a outbound file connection destination. The flat file connection is tab delimited using codepage 1252. I have mapped the converted columns to the columns used in this flat file. Below are some screenshots of the connection manager and destination block
Now when i create a new txt file from explorer it will be ANSI (as detected by Notepad++)
When the package runs the file becomes UTF-8 w/o BOM
I have tried experimenting with the checkbox for overwriting as suggested in SSIS - Flat file always ANSI never UTF-8 encoded
as building the project from scratch and experimenting with the data conversion.
Does anyone have a suggestion on what I am missing here? The strange thing is we have a different package with exact the same blocks build previously and it does output an ANSI file (checked the package from top to bottom). However we are getting mixed results on different machines. Some machines will give an ANSI file other the UTF-8 file.
Is this solved already? My idea is to delete the whole Data Flow Task and re-create it. I suppose the metadata is stuck and overwritten at each execution.
I believe you need not to change anything in your ssis package just check your editor setting (notepad++). Go to settings --> Preferences --> new document setting
You need to uncheck the 'Apply to opened ANSI files' checkbox.
Kindly check and let me know if it works for you.