I'm trying to import some CSV files into SQL using the SQL Server Import and Export Wizard. I came across an error where some of the values contained two double quotes at the end e.g. GIG L SHAPIRO GU A"". The Text Qualifier is already be set to ", so I'm not sure how to import these files.
Related
I'm trying to import a customer csv file via SSMS Import Wizard, file contains 1 million rows and I'm having trouble importing where the field has extra quotes e.g. the file has ben populated freehand so could contain anything.
Name, Address
"John","Liverpool"
"Paul",""New York"""
"Ringo","London|,"
"George","India"""
Before I press on looking into SSMS should SSMS 2016 handle this now or do I have to do in SSIS, it is a one off load to check something?
In SSMS Import/Export Wizard, when configuring the Flat File Source you have to set:
Text Qualifier = "
Column Delimiter = ,
This will import the file as the following:
Name Address
John Liverpool
Paul "New York""
Ringo London|,
George India
The remaining double quotes must be removed after import is done using SQL, or you have to create an SSIS package manually using Visual Studio and add some transformation to clean data.
I am trying to import *.DAT file(as flat file source) into sql server using SQL server import and export wizard. It has DC4 as delimiter which is causing error while trying to separate the columns and their respective data and importing them in sql server.
Are there any setting changes to be made during the importing process?
If you don't have to use the wizard, you can script it like:
BULK INSERT [your_database].[your_schema].[your_table]
FROM 'your file location.dat'
WITH (ROWTERMINATOR='0x04' -- DC4 char
,MAXERRORS=0
,FIELDTERMINATOR='รพ'
,TABLOCK
,CodePage='RAW'
)
The wizard uses SSIS under the hood. Instead of executing it directly, chose CrLF as row delimiter, then chose to save it as file. Open the file and edit it using any text editor. It's a simple xml file.
It's not clear whether 0x04 is the column delimiter or the row delimiter. Assuming it's the row delimiter,
Replace all instances of
Delimiter="_x000D__x000A_"
with
Delimiter="_x0004_"
there're two instances: DTS:HeaderRowDelimiter and DTS:ColumnDelimiter
Save the file and execute it with a double clik or "Open with: Execute package utility". I tested the solution on my PC using an account with limited permissions.
I'm using SQL Server 2014 Management Studio - Import and Export wizard to import a .DAT file into a SQL Server table.
However, while all of these records are text qualified with " double quotes, some of the field values have double quotes within them. SQL Server aborts the package every time it hits a row like that. Any solution?
"Field A"|"Field B"|"Field C"
"Value A"|"Another value "with" for B"|"Value C"
for some reason, I can't seem to figure out why I'm missing some lines when I import a csv file that I exported using sqlplus. The ssis package gives no warnings or errors as well.
The reason I know this is because when I loaded the csv file into a separate analysis application it gets the correct totals for numeric columns and row counts.
But for some reason, SSIS doesn't seem to capture all of the lines...
there were 6313052 rows in the csv file and ssis imports 6308607...
any thoughts?
I've tried different code pages too (1250, 1252, and UTF8) but they didn't seem to have an affect
I checked out this link:
Why all the records are not being copied from CSV to SQL table in a SSIS package
and numbers I've checked on numbers 2, 3, and 4.
Although, for number 1, I'm using a for each loop container descirbed in this site:
http://help.pragmaticworks.com/dtsxchange/scr/FAQ%20-%20How%20to%20loop%20through%20files%20in%20a%20specified%20folder,%20load%20one%20by%20one%20and%20move%20to%20archive%20folder%20using%20SSIS.htm
to loop through files in a folder and import them.
I've also thought about missing delimiters, but I exported the files myself using sqlplus like this:
select
trim(to_char(t1.D_DTM, 'yyyy-mm-dd hh24:mm:ss'))||','||
trim(t1.TECHNOLOGY)||','||
trim(t1.VOICEDATA)||','||
trim(t2.MRKT_NM)||','||
trim(t2.REGION_NM)||','||
trim(t2.CLUSTER_NM)||','||
trim(t3.BSC_NM)||','||
trim(t1.BTS_ID)||','||
trim(t1.CSCD_NM)||','||
trim(t1.SECT_SEQ_ID)||','||
trim(t1.BND_ID)||','||
trim(t1.FA_ID)||','||
trim(t1.TCE_DTCT_CALL_SETUP_F_CNT)||','||
trim(t1.MS_ACQ_CALL_SETUP_F_CNT)||','||
trim(t1.SIGN_CALL_SETUP_F_CNT)||','||
trim(t1.BAD_FRM_CALL_SETUP_F_CNT)||','||
trim(t1.REORG_ATT_CNT)||','||
trim(t1.TCE_CALL_SETUP_F_CNT)||','||
trim(t1.WCD_CALL_SETUP_F_CNT)||','||
trim(t1.L_CALL_SETUP_F_CNT)||','||
trim(t1.TRAF_FRM_MISS_CNT)||','||
trim(t1.BCP_TMO_CALL_SETUP_F_CNT)
from table
I'm not sure how I could miss delimiters if I export a file like that...
Also, I tested importing these files into MySql using LOAD DATA INFILE and that seems to work fine and import all of the data...
SQL 2005
I am importing .csv file to SQL Table through SSIS Package !! On doing so, I am getting double quotes on every column header or field value !!
How to remove double quotes, while importing .csv file ???
The csv file is using the " as a text qualifier.
When importing you must specify inform SSIS of this.
See How to strip out double quotes from an import file in SQL Server Integration Services SSIS
Add " as the Text Qualifier when choosing the source (csv file)
http://twitpic.com/4k3ok8/full
Enter " double quotes as the Text Qualifier.