My CSV has text like this:
Côté fenêtres,
carré
I'm trying to open this CSV file using openrowset in SQL Server like below:-
select * from openrowset(BULK 'C:\Import_Orders\Files\PO.csv',
FORMATFILE = 'C:\Import_Orders\Format\Cust_441211.fmt.txt') as PO
But the result is like this:
C+¦t+¬ fen+¬tres,
Carr+¬
How can I tackle this issue? Let me know if I need to add anything more to this question.
SQL Version -
Microsoft SQL Server 2017 (RTM-CU29-GDR) (KB5014553) - 14.0.3445.2 (X64)
This is the format file:-
11.0
8
1 SQLCHAR 0 250 "|" 1 PARTNO ""
2 SQLCHAR 0 250 "|" 2 CODE ""
3 SQLCHAR 0 250 "|" 3 PRICEKG ""
4 SQLCHAR 0 250 "|" 4 FOOTKG ""
5 SQLCHAR 0 250 "|" 5 LENGTH ""
6 SQLCHAR 0 250 "|" 6 QTY ""
7 SQLCHAR 0 250 "|" 7 COLOR ""
8 SQLCHAR 0 250 "\r\n" 8 TOTKG ""
(1) You can try to add an additional parameter CODEPAGE = '65001' to specify a code page to support UNICODE characters.
(2) Use may try to use SQLNCHAR data type instead of SQLCHAR in the format file. For a text file you should always specify SQLCHAR for all fields, unless you have a Unicode file in in UTF‑16 encoding in which case you should use SQLNCHAR.
SQL
SELECT * FROM openrowset(BULK 'C:\Import_Orders\Files\PO.csv',
FORMATFILE = 'C:\Import_Orders\Format\Cust_441211.fmt.txt',
CODEPAGE = '65001') as PO;
I'm using bcp to import a .csv file into a table in SQL Server like this:
bcp test1 in "./file.csv" -S server_name -U login_id -P password -d database_name -c -t
I get this message/warning:
Unexpected EOF encountered in BCP data-file.
bcp copy in failed
file.csv data:
A, B, C
A, B, C
A, B, C
A, B, C
My tables:
CREATE TABLE test2
(
a VARCHAR(8) PRIMARY KEY,
b VARCHAR(8),
c VARCHAR(8)
);
CREATE TABLE test1
(
ID INT NOT NULL IDENTITY(1,1) PRIMARY KEY,
a VARCHAR(8),
b VARCHAR(8),
c VARCHAR(8)
FOREIGN KEY (a) REFERENCES test2(a)
);
Here is what I get in my SELECT * FROM test1;:
ID | a | b | c |
1 B CA B C
A B CA B C
Here is what I expected:
ID|a |b |c
1 A B C
2 A B C
3 A B C
3 A B C
I'm having no issues with test2 which is nearly like test but without ID. So the csv files are well formatted. So why do I have a shift like this?
EDIT 1
If I add header to csv fileI have this in my SELECT * FROM test1:
ID|a|b|c |
1 b cA B,C,A,B,C,A,B,C,A,B,C
EDIT 2
I generated a File Format to guide my data.
13.0
3
1 SQLCHQR 0 40 "\t" 1 a SQL_Latin1_General_CP1_CI_AS
2 SQLCHQR 0 40 "\t" 2 a SQL_Latin1_General_CP1_CI_AS
3 SQLCHQR 0 40 "\t" 3 a SQL_Latin1_General_CP1_CI_AS
The modified one to try to "jump" over ID
13.0
4
1 SQLCHQR 0 40 "\t" 2 a SQL_Latin1_General_CP1_CI_AS
2 SQLCHQR 0 40 "\t" 3 a SQL_Latin1_General_CP1_CI_AS
3 SQLCHQR 0 40 "\t" 4 a SQL_Latin1_General_CP1_CI_AS
But I can't manage to make it work.
bcp test1 in "./file.csv" -S server_name -U login_id -P password -d database_name -f file_name -t
SQLState = S1002, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]Invalid Descriptor Index
EDIT 3
I found a way around the problem, but it's still not the good solution. What I did is:
Change my test1 table, putting ID column at the end.
Then adding with sed double commas at the end of each lines in my CSV to create a new empty column. Then I made a simple bcp in. I still want keep ID in first column, I just don't want to create an extra View for putting ID in front.
First Edit due to inverse meaning for the -E switch
You are missing missing the -E for the BCP to know that you are trying to use Identity column.
If you use this switch then the values are taken from the file and the SQL Server Identity value is ignored. You should preform DBCC CHECKIDENT('table_name', RESEED) afterwards.
Second Edit due to the more specific user whishes.
After edit #BeGreen specified that he wants to use format file. I'll post then the example how to do it. No need for the view in front which I saw in other post on Stack overflow: What will be BCP format for inserting a identity column.
The examples are being inconsistent in that CSV is mixing "," (in the example) and "\t" (in the format table). I'll be using only "," in the examples.
That is possible but you have to use correct approach.
First you have to generate the format file by this command:
bcp <db_name>.dbo.test1 format nul -c -f "generated_format_file.fmt" -t "," -S "<server>\<instance>" -T
Then you take generated .fmt file in this case (using SQL Server 2005 - as I have quick testing instance there - you can change the first number to fit your needs e.g. 13.0):
8.0
4
1 SQLCHAR 0 12 "," 1 ID ""
2 SQLCHAR 0 8 "," 2 a SQL_Latin1_General_CP1_CI_AS
3 SQLCHAR 0 8 "," 3 b SQL_Latin1_General_CP1_CI_AS
4 SQLCHAR 0 8 "\r\n" 4 c SQL_Latin1_General_CP1_CI_AS
The *.csv file looks (I have added the ID for tests and different from the Identity values that should be inserted into the table):
10,A1,B1,C1
20,A2,B2,C2
30,A3,B3,C3
40,A4,B4,C4
Then the correct command for importing (note: using trusted connection -T instead of -U -P to make the line shorter):
bcp <db_name>.<schema>.test1 IN "import_data.bcp" -f "generated_format_file.fmt" -t "," -S "<server>\<instance>" -T
Starting copy...
4 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 218 Average : (18.35 rows per sec.)
Data is imported as (test1):
ID a b c
1 A1 B1 C1
2 A2 B2 C2
3 A3 B3 C3
4 A4 B4 C4
To check identity value that is currently at test1:
select IDENT_CURRENT('[database_name].[dbo].[test1]')
Result:
(No column name)
4
I have the following csv I wish to import into my db
"LE";"SOURCE";"VAR_SCTARGET_NAME"
"B";"A/K";"A/K"
"A";"A/B";"A/B"
"A";"A/B";"A/C"
I arranged the following format file
10.0
3
1 SQLCHAR 0 0 "\";\"" 1 A ""
2 SQLCHAR 0 0 "\";\"" 2 B ""
3 SQLCHAR 0 0 "\"\r\n\"" 3 AA ""
which works just fine, if it weren't for the last line. The output in my db is the following
LE SOURCE VAR_SCTARGET_NAME
B A/K A/K
A A/B A/B
A A/B A/C"
How can I remove the quote on the last row? I'm working on a SQL Server platform, if it can be of any help.
I am trying to import raw data that I have in .csv format to my table in SQL Server. My table is called [raw].[sub_brand_channel_mapping].
The last column ImportFileId is generated by my python code. I first set all the rows of that column as the default value generated. Then I use bulk insert to import my csv data in utf-8 format to my table in SQL Server. However, during the process a lot of special characters are changing. I am using a format file like this
10.0
8
1 SQLCHAR 0 100 "\t" 1 sub_brand_id ""
2 SQLCHAR 0 1024 "\t" 2 sub_brand_name SQL_Latin1_General_CP1_CI_AS
3 SQLCHAR 0 100 "\t" 3 channel_country_id ""
4 SQLCHAR 0 1024 "\t" 4 channel_id ""
5 SQLCHAR 0 1024 "\t" 5 channel_name SQL_Latin1_General_CP1_CI_AS
6 SQLCHAR 0 256 "\t" 6 status ""
7 SQLCHAR 0 256 "\t" 7 eff_start_date ""
8 SQLCHAR 0 256 "\r\n" 8 eff_end_date ""
My bulk insert command looks like this:
bcp "{table}" in "{file}" {connect_string} -f {format_file} -F {first_row} - b 10000000 -e {error_file} -q -m {max_errors}
My csv files have "\t" as the delimiter. I need to import the exact names without any change. What should I do ?
P.S. I did try converting my utf-8 encoded csv to utf-16-le and then use "-w" in my bcp command. It is giving a lot of errors. In short it didn't work. Please advise me on how to do this.
This is the error that I am getting while executing bcp command:
SQLState = 07006, NativeError = 0 Error = [Microsoft][SQL Server
Native Client 11.0][SQL Serve owed from the given field type 0x3D for
field 9 to column 9.
command used is :
bcp "VM_Asia.dbo.dim_budget" in C:\BCP_DATA_INDIA\datafile\dim_budget.dat -f C:\BCP_DATA_INDIA\formatcorrected\Dim_budget.fmt -T
And my fmt file is like this:
11.0
12
1 SQLINT 0 4 "" 1 budget_SK ""
2 SQLINT 1 4 "" 2 country_sk ""
3 SQLINT 0 4 "" 3 budget_ID ""
4 SQLNCHAR 2 36 "" 4 budget_Type Latin1_General_CI_AI
5 SQLNCHAR 2 200 "" 5 budget_Description Latin1_General_CI_AI
6 SQLDATE 1 3 "" 6 budget_Applied_From ""
7 SQLDATE 1 3 "" 7 budget_Applied_To ""
8 SQLDATETIME 1 8 "" 8 Inserted_Date ""
9 SQLDATETIME 1 8 "" 9 Modified_Date ""
10 SQLINT 1 4 "" 10 batchno ""
11 SQLINT 1 4 "" 11 Status ""
12 SQLINT 1 4 "" 12 company_id ""
What version are you on? Possible explanation here. https://msdn.microsoft.com/en-us/library/bb677301.aspx
SQL Server Native Client (SNAC) is not supported beyond SQL Server 2012. Avoid using SNAC in new development work, and plan to modify applications that currently use it. The Microsoft ODBC Driver for SQL Server provides native connectivity from Windows to Microsoft SQL Server and Microsoft Azure SQL Database.