I'm trying to import data from a .csv file into a SQL Server table.
Using the code below, I can read from the file:
BULK INSERT #TempTable
FROM '\\Data\TestData\ImportList.csv'
WITH (FIELDTERMINATOR = ',', ROWTERMINATOR ='\n', FIRSTROW = 2, Lastrow = 3)
GO
(I added LastRow = 3 so I was just getting a subset of the data rather than dealing with all 2000 rows)
But I am getting multiple columns into a single column:
If I use the Import/Export wizard in SSMS, with the below settings, I see the expected results in the preview:
Can anyone give me some pointers as to how I need to update my query to perform correctly.
Here is a sample of what the CSV data looks like:
TIA.
You probably need to specify " as Text qualifier.
Your fields seem to be quoted and most likely contain comma's, which are currrently splitting your fields.
Or, if it works fine using <none> as Text qualifier, try to use FIELDQUOTE = '' or FIELDQUOTE = '\b' in your query. FIELDQUOTE defaults to '"'.
It's hard to tell what's really wrong without looking at some raw csv data that includes those quotes (as seen in your first screenshot).
Related
I'm new to SQL Server, so forgive me for being a bit of a noob here.
The code shown here returns the following error:
Cannot obtain the required interface ("IID_IColumnsInfo") from OLE DB provider "BULK" for linked server "(null)".
Code:
BULK INSERT testingtable
FROM 'D:\TimeLords\data\db-test-file.csv'
WITH
(FORMAT = 'CSV',
FIELDQUOTE = '"',
FIRSTROW = 2,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
TABLOCK)
I've tried using:
ROWTERMINATOR = '0x0a'
and
ROWTERMINATOR = '\r\n'
This is the CSV file: https://gyazo.com/0392b660c97e3cac27f2337993190c69
This is my SQL table: https://gyazo.com/fbbaf6204df9bb574d8887864cc95ea0
And this is the complete SQL query: https://gyazo.com/ffe020437f07524ce44420bedeebf0d4
I've scouted StackOverflow and can't find any solution which works. Any ideas would be appreciated.
Thanks
There's another potential culprit. I've been running BULK INSERTs into my SQL Server 2017 Express, and my syntax used FORMAT = 'CSV' and a ROWTERMINATOR of '\n' -- and it had been working fine for months.
I added a new column to the other system where I was routinely exporting data as a CSV, and when I went to do another BULK INSERT, it was failing because I had an extra column in my CSV that didn't line up with the columns in my SQL table. DOH! I just needed to add that same new column in my SQL db and all was well again. A stupid error on my part, but maybe it will help someone else.
Change FORMAT = 'CSV' to DATAFILETYPE = 'char'
or just remove the FORMAT = 'CSV' line as your file may not be RFC 4180 compliant.
BULK INSERT testingtable
FROM 'D:\TimeLords\data\db-test-file.csv'
WITH
(FIELDQUOTE = '"',
FIRSTROW = 2,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
TABLOCK)
this has worked for me with this error.
Old post, but hey, every bit of knowledge helps. You can also run into this issue if you use a CSV other encoding or types, e.g. if you save CSV for Macintosh or UTF-8 (as you can in Excel), this is not compliant with FORMAT = 'CSV'. You can try other options like row terminator = '\r' while removing FORMAT = 'CSV' that did it for me for non-windows-based CSV files.
For me the error was an extra space on the end of the first row.
I am using Microsoft SQL Server Management studio and I am currently importing some CSV files in a database. I am importing the CSV files using the BULK INSERT command into already existing tables, using the following query.
BULK INSERT myTable
FROM >>'D:\myfolder\file.csv'
WITH
(FIRSTROW = 2,
FIELDTERMINATOR = ';', --CSV Field Delimiter
ROWTERMINATOR = '\n', -- Used to shift to the next row
ERRORFILE = 'D:\myfolder\Error Files\myErrrorFile.csv',
TABLOCK
)
This works fine for me thus far, but I would like to automate the process of naming columns in tables. More specifically I would like to create a table and use as column names, the contents of the first row of the CSV file. Is that possible?
The easiest way I can think of is:
right-click on the database, select: Tasks -> Import Data...
After that, SQL Server Import and Export Wizard will display. There you have everything to specify and custom settings on importing data from any sources (such as getting column names from first row in a file).
In your case, your data source will be Flat file source.
I've got the simple table in CSV format:
999,"01/01/2001","01/01/2001","7777777","company","channel","01/01/2001"
990,"01/01/2001","01/01/2001","767676","hhh","tender","01/01/2001"
3838,"01/01/2001","01/01/2001","888","jhkh","jhkjh","01/01/2001"
08987,"01/01/2001","01/01/2001","888888","hkjhjkhv","jhgjh","01/01/2001"
8987,"01/01/2001","01/01/2001","9999","jghg","hjghg","01/01/2001"
jhkjhj,"01/01/2001","01/01/2001","9999","01.01.2001","hjhh","01/01/2001"
090009,"","","77777","","","01/01/2001"
980989,"01/01/2001","01/01/2001","888","","jhkh","01/01/2001"
0000,"01/01/2001","01/01/2001","99999","jhjh","","01/01/2001"
92929,"01/01/2001","01/01/2001","222","","","01/01/2001"
I'm trying to import that data into SQL Server using BULK INSERT (Transact-SQL)
set dateformat DMY;
BULK INSERT Oracleload
FROM '\\Mac\Home\Desktop\Test\T_DOGOVOR.csv'
WITH
(FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
KEEPNULLS);
On the output I get the next error:
Msg 4864, Level 16, State 1, Line 4
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 2 (date_begin)....
Something wrong with date format maybe. But what script I need to write to fix that error?
Please help.
Thanks in advance.
BULK INSERT (nor bcp) cannot (properly) handle CSV files, specially if they have (correctly) " quotes. Alternatives are SSIS or PowerShell.
I always look at the data in Notepad++ to see if there are some weird characters, or non-printable characters, like a line break or something. For this, it seems like you can open it using Notepad (if you don't have Notepad++) do a find-replace for " to nothing... Save the file, and re-do the Bulk Load.
This record:
jhkjhj,"01/01/2001","01/01/2001","9999","01.01.2001","hjhh","01/01/2001"
The first column has a numeric type of some kind. You can't put the jhkjhj value into that field.
Additionally, some records have empty values ("") in date fields. These are likely to be to interpreted as empty strings, rather than null dates, and not convert properly.
But the error refers to "row 1, column 2". That's this value:
"01/01/2001"
Again, the import is interpreting this as a string, rather than a date. I suspect it's trying to import the quotes (") instead of just using them as separators.
You might try bulk loading to a special holding table, and then re-importing from there. Alternatively, you can change how data is exported or write a program to pre-clean it — strip the quotes from fields that shouldn't have them, isolate records that have data that won't insert to an exception file and report.
I have a Persian CSV file and I need to read that with SQL bulk into the SQL server:
I wrote this bulk:
BULK INSERT TEMP
FROM 'D:\t1.csv'
WITH(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
CODEPAGE = '1256'
);
but that can not read UTF-8 encoding and read ی character as ? character.
How can I write that?
1. go to the BULK INSERT documentation on MSDN
2. find the section on the CODEPAGE
3. see the note that says:
SQL Server does not support code page 65001 (UTF-8 encoding).
4. Research further and find the Use Unicode Character Format to Import or Export Data (SQL Server) and see if that helps
This problem is still there in SQL server 2017, see here and here.
If your import is just an occasional exercise, i.e. if it's OK to import not using a script at all, what worked for me is simply importing the csv using Tasks -> Import -> Flat file.
I'm adding this here because this page is high up when you Google 'SQL Server does not support code page 65001'. Hope it helps some.
In addition to the now deprecated or obsolete earlier answers by others I want to point out that a of today in May 2022, with Release Version 15.0.2080.9 (SQL Server 2019), this works flawlessly for UTF-8.
Create a UTF-8 encoded file (I use with BOM)
then
BULK INSERT #tempTable1
FROM 'C:\....\file.csv' WITH (
CODEPAGE = '65001',
FIRSTROW = 2, --skip the first line
FIELDTERMINATOR = ';',
ROWTERMINATOR = '\n')
GO
works flawlessly for me, with many french and other characters.
I went through the documenation #marc_s linked to, and found the usage of DATAFILETYPE = widechar.
I then went ahead and tried it with my UTF-8 csv file, but it didn't work, giving me the error:
[...] the data file does not have a Unicode signature
I then re-saved my csv file with Notepad's Unicode format, retried the import, and voila, success.
Make sure all commas and line-breaks are escaped (see here how to save a valid csv).
My full script (I'm using SQL Server 2017):
BULK INSERT [my_table]
FROM 'C:\path\to\file.csv'
WITH
(
FORMAT = 'CSV',
FIRSTROW = 2, -- if you have a title row, the first data row is 2nd
FIELDTERMINATOR = ',',
KEEPIDENTITY, -- remove it if you don't want identity to be kept
ROWTERMINATOR = '\n',
DATAFILETYPE = 'widechar',
ERRORFILE = 'C:\path\to\file_err.txt',
KEEPNULLS,
TABLOCK
)
Notes:
Make sure your date fields are in valid sql format.
Regarding KEEPNULS, read this question (e.g, if you have NULLs in your file, replace them with an empty string).
How can I bulk insert a file like below?
test.txt
012341231
013212313
011312321
012312312
The text file does not contain a delimiter. I have used:
BULK INSERT tbl_import_#id#
FROM '../test.txt'
WITH
(FIELDTERMINATOR = '\t',
ROWTERMINATOR = '\n')
and I got an error for that. Appreciate any help thanks.
There is no problem. You can specify a field terminator even if your file doesn't have any field terminators like \t or ,.
Please try to post what error you have got. Check your FROM file ".../test.txt" location and table schema to import data. Better to post your error. I cannot reproduce your error. It works fine for me (I used your values).
Just run the query without FILEDTERMINATOR
BULK INSERT tbl_import_#id#
FROM '../test.txt'
WITH (ROWTERMINATOR = '\n')
The FIELDTERMINATOR argument would be helpful in case you had multiple columns in your table (more values per row). But I can see that this is not the case, so you don't need to separate values except by rows, which will be records in your table.
EDIT:
In case you can use a different table, just create a table with only 1 column(ID column) and run the import (the query above).
After that, run an ALTER command and add the other columns that you want.