bcp file sometimes with double quotes - sql-server

Question

I have a csv file with 2 columns ID, CompanyName.
I want to bcp to a sqlserver table with format file. This file is comma delimited. Problem with the .csv file is that Companyname has double quotes around them only when there are multiple words.
Here's an example
CompanyID,CompanyName
1000,FirstCompanyName
2000,"Testing Comma Name"
I do not know how to write a format file for this.
This is what I tried
10.0
3
1 SQLCHAR 0 10 ",\"" 1 CompanyID SQL_Latin1_General_CP1_CI_AI
2 SQLCHAR 0 0 "\"" 0 junk1 SQL_Latin1_General_CP1_CI_AI
3 SQLCHAR 0 100 "\r\n" 2 CompanyName SQL_Latin1_General_CP1_CI_AI
There are no errors
when I type this in Command prompt
bcp "[a].b.[CompanyData]" in "C:\test.csv" -f C:\Data.fmt -t, -F2 -S "server1\prod01" -Uuername -Ppwd -e C:\Logs\error.log -o C:\Logs\outputlog.log
there are no errors but nothing in the table either.
Can someone guide me?
Thanks
MR

Answer 1

bcp command line utility cannot process a *.csv file where a column has a sporadic double quotes as a delimiter.
A possible solution would be to load the entire line into a wide single column in a staging table. After that split into three columns by using T-SQL via bcp or BULK INSERT.
Here is the 2nd part. SQL Server 2017 onwards (due to the enhanced SPLIT() function)
SQL
-- DDL and sample data population, start
DECLARE #tbl TABLE (line_from_file NVARCHAR(MAX));
INSERT INTO #tbl (line_from_file) VALUES
(N'1000,FirstCompanyName'),
(N'2000,"Testing Comma Name"');
-- DDL and sample data population, end
DECLARE #separator CHAR(1) = ',';
;WITH rs AS
(
SELECT *
, TRY_CAST('<root><r><![CDATA[' +
REPLACE(line_from_file, #separator, ']]></r><r><![CDATA[') + ']]></r></root>' AS XML) AS xmldata
FROM #tbl
)
-- INSERT INTO targetTable (CompanyID, CompanyName)
SELECT c.value('(r[1]/text())[1]', 'INT') AS CompanyID
, TRIM('"' FROM c.value('(r[2]/text())[1]', 'NVARCHAR(100)')) AS CompanyName
FROM rs CROSS APPLY xmldata.nodes('/root') AS t(c);
Output
+-----------+--------------------+
| CompanyID | CompanyName |
+-----------+--------------------+
| 1000 | FirstCompanyName |
| 2000 | Testing Comma Name |
+-----------+--------------------+