I am trying to export of table in csv file by using bcp command in microsoft sql server.
Below is the table sample data
Table name : XYZ
col1 col2 col3
abcd,inc. USD,inc 1234
pqrs,inc USD,inc 6789
stuv,inc USD,inc 0009
There is comma in column values as above.
I have written .fmt file like below:
test.fmt
13.0
3
1 SQLCHAR 0 4000 "\",\"" 1 col1 SQL_Latin1_General_CP1_CI_AS
2 SQLCHAR 0 4000 "\",\"" 2 col2 SQL_Latin1_General_CP1_CI_AS
3 SQLCHAR 0 4000 "\r\n" 3 col3 SQL_Latin1_General_CP1_CI_AS
Below is command I am using:
DECLARE
#V_BCP_QUERY VARCHAR(4000),
#V_BCP_OUTPUT_FILE VARCHAR(1500),
#V_BCP_FORMAT_FILE VARCHAR(1500),
#V_BCP_COMMAND VARCHAR(4000)
begin
SET #V_BCP_QUERY='"SELECT col1,col2,col3 FROM TABS..XYZ"'
SET #V_BCP_OUTPUT_FILE='"D:\OUTPUT.csv"'
SET #V_BCP_FORMAT_FILE='"D:\test.fmt"'
SET #V_BCP_COMMAND='bcp '+#V_BCP_QUERY+' queryout '+#V_BCP_OUTPUT_FILE+' -f '+#V_BCP_FORMAT_FILE+' -T -S "DEV-CR"'
EXECUTE Master.dbo.xp_CmdShell #V_BCP_COMMAND
end
I am getting below data in OUTPUT.csv file:
abcd,inc.","USD,inc","1234
pqrs,inc","USD,inc","6789
stuv,inc","USD,inc","0009
there is no " at start of line and end of line.
Also when I open this in excel then all rows are coming in a single line
my requirement is to export file in csv file.
Kindly help
You could hack a solution together - but using the correct tool for the job would be much easier and better in the long run.
Instead of using BCP to output individual columns, create a single column formatted with your desired result:
SELECT quotename(concat_ws(',', quotename(col1, char(34)), quotename(col2, char(34)), quotename(col3, char(34)), char(34))
FROM yourTable
This will give you a single column in your output - with double-quotes around the whole string, double-quotes around each column, concatenated with the '.' separator.
Sure it's ugly, but it's simple, quick and gets it done.
SELECT '"' + col1 + '",' +
'"' + col2 + '",' +
'"' + col3 + '"'
FROM Table
I have an sqlcmd command which generates a csv file from a view. Is it possible to replace the NULL values to empty string using the findstr command?
Here is what I tried.
sqlcmd -S . -d SAMPLEDB -U sa -P pass -s"|" -W -Q "SET NOCOUNT ON SET ANSI_WARNINGS OFF select * from view_Table" > Sample.csv -h -1 | findstr /v /c:"NULL"
You can easily build the extraction SQL for each view using the system management views. This simple query:
SELECT v.[name]
,c.[name]
,c.[column_id]
,c.[is_nullable]
FROM sys.views V
INNER JOIN sys.columns C
ON V.[object_id] = C.[object_id];
will return everything we need to perform the task:
the view name
the column name
the column order
if the column is nullable
So, we need only to build the extraction SQL statements:
SELECT v.[name]
,'SELECT ' + DS.[definition] + ' FROM ' + v.[name]
FROM sys.views V
CROSS APPLY
(
SELECT STUFF
(
(
SELECT ',' + CASE WHEN c.[is_nullable] = 1 THEN 'ISNULL(' + c.[name] + ','''')' ELSE c.[name] END
FROM sys.columns C
WHERE V.[object_id] = C.[object_id]
ORDER BY c.[column_id]
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1
,1
,''
)
) DS ([definition]);
Depending on your SQL version you can reduce the code above - for example using IIF or STRING_AGG.
Also, you can add WHERE clause to filter the query for specific views.
Change the sql in the command string to select fields individually and do an ISNULL on them
select * from view_Table
To
select ISNULL([FIELD_1],''''),ISNULL([FIELD_2],'''') from view_Table
in you calling string
use the following code to build it and check to see the final statement runs
DECLARE #Temp VARCHAR(MAX)
SET #Temp = 'SELECT ISNULL([FIELD_1],''''),ISNULL([FIELD_2],'''') FROM view_Table'
SELECT #Temp
I am trying to export a .txt file with bcp:
EXEC xp_CMDSHELL 'BCP "SELECT v.ItemId
, v.InventLocationId
, CAST(v.AvailPhysical AS int)
FROM D_R1.dbo.vwStockOnHand AS v
JOIN D_R1.DBO.vwProducts AS m
ON v.ItemId = m.ITEMID
WHERE NOT EXISTS
(SELECT 1
FROM D_R1.DBO.ExportExcludedFamilies AS magExport
WHERE magExport.REFRECID = m.FamilyRecId)
AND AVAILPHYSICAL > 0 AND v.PICKFROMZ = 1
ORDER BY v.InventLocationId, v.ITEMID"
queryout "C:\temp\1.txt" -c -t"|" -T -S D_R1'
All good, but i would need a last line in the file like "END OF FILE" and i am not able to figure it out ..
Can someone give me a hint on this?
1) First solution
SELECT CONVERT(VARCHAR(11), v.ItemId) AS ItemId
, v.InventLocationId
, CAST(v.AvailPhysical AS int)
, 1 AS Priority
FROM ...
UNION ALL
SELECT 'End of export file', NULL, NULL, 2 AS Priority
ORDER BY Priority, InventLocationId, ItemID
or
SELECT... ;
SELECT 'End of report' ;
or
SELECT... ;
PRINT 'End of report' ;
2) Second solution
bcp "the same SQL query", ...
echo End of export
echo command
I want to extract the CSV file from SQL Server and I manage to do that with the following code:
DECLARE #bcpCommand1 VARCHAR(2000)
SET #bcpCommand1 = 'bcp " SELECT * from ##tblUser " queryout'
SET #bcpCommand1 = #bcpCommand1 + ' C:\OutPut\CSV\My_Output.csv -c -w -T -S197.1.1.1 -Umyuser -Pmypwd","-CRAW'
EXEC CPS..xp_cmdshell #bcpCommand1
But the problem is ...
The big integer values are automatically coverted to scientific notation in CSV. Instead of this I want to display all the digits as it is.
I want to display the date as '01-Dec-2015 11:20 AM'.
If any data contains a comma, then the data is split to the next column. For example name is "mohan,Raj" then in the .csv data Mohan is in one column and raj in another column. How to avoid that?
Is there any way extract .csv using format file. How to do that?
Is there any way to do that? Please help me on this and post the code for that.
The BCP is used to copy whatever from a table to csv or txt or xml format.
If you want to get the comma separated values in one column as multiple column in csv. Then you have to modify your temp table accordingly.
I want to display the date as '01-Dec-2015 11:20 AM'.
There is not direct conversion to your exact format
you can use this code to do the same
SELECT REPLACE(CONVERT(VARCHAR(11), GETDATE(), 106), ' ', '-') +' '+STUFF(RIGHT( CONVERT(VARCHAR,GETDATE(),100 ) ,7), 6, 0, ' ')
4) Is there any way extract csv using format file.How to do that?
sqlcmd -S MyServer -d myDB -E -Q "select col1, col2, col3 from SomeTable"
-o "MyData.csv" -h-1 -s"," -w 700
-h-1 removes column name headers from the result
-s"," sets the column seperator to ,
-w 700 sets the row width to 700 chars
(this will need to be as wide as the longest row or it will wrap to the next line)
The answer on your 4th question (and it helps with questions 1 and 2). To load csv-data on server you can use:
INSERT INTO dbo.YourTable
SELECT a.* FROM OPENROWSET( BULK 'D:\our.csv', FORMATFILE = 'D:\our.fmt') AS a;
The sample of our.fmt (it's file that describes the fields in csv)
9.0
4
1 SQLCHAR 0 50 ";" 1 Field1 SQL_Latin1_General_Cp437_BIN
2 SQLCHAR 0 50 ";" 2 Field2 SQL_Latin1_General_Cp437_BIN
3 SQLCHAR 0 50 ";" 3 Field3 SQL_Latin1_General_Cp437_BIN
4 SQLCHAR 0 500 "\r\n" 4 Field4 SQL_Latin1_General_Cp437_BIN
You can find description of *.fmt here.
I have a basic Select query which brings me back a set of results (roughly around 100,000 records) which I currently have to export to .CSV Format , issue is I then have to remove the commas out of the results and replace it with a (~). Which I do using a file format application that I got off the web.
But I'm trying to go about making this automated (if possible) to save time. Like e.g run off a stored procedure that can do this for me export the file in a (~) format.
Does anyone have any tips how this stored procedure can be written or an pointers would be appreciated.
p.s I have tried to use the export wizard but it just just crashes due to too many records.
Expected Result
Test1~Test2~Test3
5~6~7
(sql Script which I am running)
select
'SPK' as [AGENCY_CODE], -- should be set to SPK
'OBCALL' as [MEDIA_CODE], -- should be set to OBCALL
isnull(c.salutation,'') as [TITLE],
isnull(c.otherName,'') as [FORENAME],
isnull(c.name,'') as [SURNAME],
isnull(c.attTXT64,'') as [STANDARDISED_NAME],
replace(isnull(c.addr1, ''), ',', '.') AS [BEST_ADDRESS_LINE_1],
replace(isnull(c.addr2, ''), ',', '.') AS [BEST_ADDRESS_LINE_2],
replace(isnull(c.addr3, ''), ',', '.') AS [BEST_ADDRESS_LINE_3],
replace(isnull(c.addr4, ''), ',', '.') AS [BEST_ADDRESS_LINE_4],
isnull(c.postCode,'') as [BEST_POSTCODE],
--'0' + isnull(c.phone1,'') as [TELEPHONE_N2O],-- should be populated with the spare field ORIG_TEL .
RIGHT('0' + CONVERT(VARCHAR(11), c.phone1), 11) as [TELEPHONE_NO],-- should be populated with the spare field ORIG_TEL .
convert(varchar(100),c.attDT03,120) as [DATE_TIMESTAMP],
isnull(c.attTXT10,'') as [SM_CONTACT_KEY],
isnull(c.attTXT89,'') as [ SM_ADDRESS_KEY],
isnull(c.attTXT11,'') as [ CAMPAIGN_IDENTIFIER],
isnull(c.attTXT12,'') as [ WAVE_ID],
isnull(c.attTXT13,'') as [OLDSTACK_NEWSTACK_FLAG],
isnull(c.attTXT14,'') as [MARKET_3_FLAG],
isnull(c.attTXT15,'') as [ADSL_2_FLAG],
isnull(c.attTXT16,'') as [FIBRE_FLAG],
isnull(c.attTXT17,'') as [LOAD_ID],
isnull(c.attTXT18,'') as [CONTACT_POINT_KEY],
isnull(c.attTXT19,'') as [DATA_POOL_URN],
isnull(c.attTXT20,'') as [EVENT_KEY],
isnull(c.attTXT21,'') as [BILLING_ACCOUNT_KEY] ,
isnull(c.attTXT22,'') as [CAMPAIGN_SOURCE] ,
isnull(c.attTXT23,'') as [CAMPAIGN_CODE] ,
isnull(c.attTXT24,'') as [ CMT_ROLE_KEY],
isnull(c.attTXT25,'') as [ CMT_LOCATION_KEY],
isnull(c.attTXT26,'') as [BILL_ACCNT_NUM],
isnull(c.attTXT27,'') as [BILLING_ACCOUNT_TYPE],
--All other fields are as per the import record values
--DATE/TIME_OF_CONTRACT to CONTRACT_END_DATE_SUPPLIER3
CASE WHEN dx.datetime IS NULL THEN convert(varchar,getdate(),120) ELSE CONVERT(varchar, dx.datetime, 120) END
as [DATE/TIMEOF CONTACT],
Case when uc.campaignid = 3 then 'CT001' when uc.campaignid = 22 then 'CT001' when uc.campaignid = 18 then 'CT011'
when uc.campaignid = 26 then 'CT013'
end as [CAMPAIGN_TYPE], -- map to BT/DATA/10.CAMPAIGN_Code (return CAMPAIGN_TYPE)
ISNULL(( CASE
WHEN dx.[Abandon] = 1 THEN 'OC039'
ELSE d.code END),'OC042')
AS [OUTCOME_CODE], -- populate with (OC001-OCxxx)
'TM' as [CHANNEL_MEDIA_CODE], --= (TBC)
isnull(c.email,'') as [EMAIL_ADDRESS], -- populate with EMAIL_ADDRESS
'' as [EMAIL_CONSENT], --= populate with EMAIL_CONSENT ***************
'' as [INBOUND_TELEPHONE_NUMBER], --*****************
'' as [COMPETITOR_SUPPLIER_1], -- CONT CONTRACT_END_DATE_SUPPLIER3 – poplulate ***********
'' as [PRODUCT_FROM_SUPPLIER1], -- populate ************
'' as [CONTRACT_START_DATE_SUPPLIER1], --*****
'' as [CONTRACT_END_DATE_SUPPLIER1], --*****
'' as [COMPETITOR_SUPPLIER2], --****
'' as [PRODUCT_FROM_SUPPLIER2], --****
'' as [CONTRACT_START_DATE_SUPPLIER2],-- *****
'' as [CONTRACT_END_DATE_SUPPLIER2], --*****
'' as [COMPETITOR_SUPPLIER3],-- ****
'' as [PRODUCT_FROM_SUPPLIER3],--****
'' as [CONTRACT_START_DATE_SUPPLIER3],-- *****
'' as [CONTRACT_END_DATE_SUPPLIER3],-- *****
--ORDER_NUM to NUMBER_OF_CALLS_MADE
isnull(c.attTXT02,'') as [ORDER_NUM], -- Captured by Operator
isnull(dx.duration,0) as [CALL_DURATION],
-------------dxi.talk as [CALL_DURATION], -- populate difference start/end time (seconds)******
isnull(c.attTXT09,'') as [WARMTH_RATING_NOW], -- captured by agent
isnull(c.attTXT80,'') as [WARMTH_RATING_FUTURE_CAMPAIGNS], -- captured by agent
isnull(dx.callid,'') [SOURCE_INTERACTION_ID],
-------------isnull(a.id,'') as [SOURCE_INTERACTION_ID], -- Tpoints unique call Identifier --- Activity ID *******
isnull(uc.callcount,'') as [NUMBER_OF_CALLS_MADE],
(select left(ethnicOrigin, 1)) as [CALL_CONSENT_VALIDATE_FLG], -- ******** gift aid status - first byte only
(select left(nationality, 5)) as [CALL_CONSENT_OUTCOME_CD], -- ******** nationality - first 5 bytes only
--QUESTION _1 to ANSWER_3
isnull(c.attTXT56,'') as [QUESTION_1], -- populate from SPARE_FIELD_28
isnull(c.attTXT51,'') as [ANSWER_1], -- captured by the agent if SPARE_FIELD_1 populated
isnull(c.attTXT29,'') as [QUESTION_2], -- poaddingpulate from SPARE_FIELD_28
isnull(c.attTXT67,'') as [ANSWER_2], -- captured by the agent if SPARE_FIELD_2 populated
isnull(c.attTXT37,'') as [QUESTION_3], -- populate from SPARE_FIELD_28
isnull(c.attTXT58,'') as [ANSWER_3] -- captured by the agent if SPARE_FIELD_3 populated
--isnull(c.attdt18,GETDATE()) as [Export_Date]
from
u_contact c with (nolock)
inner join u_campaigncontact uc with (nolock) on uc.contactid = c.id
inner join u_dispcode d with (nolock) on d.id = uc.resultcodeid
outer apply (select top 1 duration, callid,
case when outcome = 113 then 1 else 0 end [Abandon], [datetime] from dxi_cdrlog cdr where cdr.urn = c.id order by callid desc) dx
where
uc.campaignid in (3, 18, 22, 26) and d.dmc = 1 and c.created between DATEADD(week, -1, getdate()) AND getdate()
and c.importid > 0
Since you are using SQL-Server you could use the command line tool sqlcmd which comes with the package. You might have to re-run the installation of the SSMS to acticvate the option.
Open cmd.exe. When you use the sqlcmd command with the options
sqlcmd -S host -d dbname -U username -P password -W -s ~ -h -1 -Q "SET NOCOUNT ON;SELECT 1,'hello world'"
It should get you
1~hello world
-W removes blanks between the columns, -s ~ sets ~ as the column separator, -h -1 removes the header line at the top and -Q then expects the actual query statement as the next argument.
Obviously for the actual job you need to call your SELECt statement. It might be a good idea to define a view (myview) for the job and in the sqlcmd just do a SELECT * FROM myview since writing a multiline SQL command within a cmd environment is not so much fun .... Of course it is possible but it is not really worth the hassle.
The command should then redirect its output directly into a file like
set sql=SET NOCOUNT ON;SELECT * FROM myview
set scmd=sqlcmd -S host -d dbname -U username -P password -W -s ~ -h -1 -Q
%scmd% "%sql" > exportfile.csv