I have a query that executes a stored procedure and exports the results to an XML file via BCP. The process works, except that the encoding of the XML file is showing as UCS-2 and I am told that the system I will be importing it into needs it to be UTF-8. Various threads suggest this should be possible, but I can't seem to find the right syntax. My code below...
ALTER PROCEDURE [dbo].[cli_sel_c4t_supplier_report]
#Consignment_id decimal(18,0),
#Supplier decimal(18,0),
#Username varchar(50)
AS
BEGIN
DECLARE #fileName VARCHAR(50),
#sqlCmd VARCHAR(8000),
#sqlStr Varchar(8000)
SET #fileName = 'C:\C4T\' + CONVERT(VARCHAR(10), #Consignment_id) + CONVERT(VARCHAR(20), #Supplier) + '.xml';
SET #sqlStr='EXEC [Company Database].[dbo].[cli_sel_c4t_xml_data] #Consignment_id = ' + CONVERT(VARCHAR(10), #Consignment_id) + ', #Supplier = ' + CONVERT(VARCHAR(12), #Supplier) + ';';
SET #sqlCmd = 'bcp "' + #sqlStr + '" queryout ' + #fileName + ' -c -C65001 -T -S "SVRNAME\INSTANCE"';
EXEC cli_ins_audit_c4t_file #Consignment_id=#Consignment_id, #Supplier_id=#Supplier, #Path=#fileName, #Username=#Username
EXEC xp_cmdshell #sqlCmd;
END
Any assistance would be appreciated.
Thanks
Steve
To generate UTF-8 instead of UTF-16, use -c instead of -w parameter. The -c option in bcp export characters by single byte.
Explicitly specifying code page 1252, and UTF-8: -c -C 1252
Related
I have a procedure which is for exporting data from table to .DAT file. There's a job running on it which calls SP every 10 Min. .DAT file contains H= column names, B= table data, T= No of rows processed(.SIG file appends this to .DAT file).
Following is the stored procedure :
CREATE PROCEDURE [dbo].[Test_Export] #count2 VARCHAR(10)
AS
BEGIN
SET NOCOUNT ON;
--Alerts_Balance
DECLARE #bcp_sql VARCHAR(1000)
DECLARE #filename VARCHAR(200), #count1 VARCHAR(10)=20000
SET #filename = 'C:\Test'+'.DAT';
SELECT #bcp_sql = 'bcp "SELECT ''H'',''Name'',''City'',''State'',''Country'' union all SELECT top 10 ''B'', [Name],[City],[State],[Country] FROM Test_Table" queryout ' + #filename + ' -c -t, -T -S '+ ##servername
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'bcp "SELECT ''T'', '+cast(IIF(cast(#count1 as int)>cast(#count2 as int), #count2+2, #count1+2) as varchar(10))+'" queryout '+ #filename + '.sig -c -t, -T -S '+ ##servername
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'type ' + #filename + '.sig >> "' + #filename + '"'
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'copy nul: ' + #filename + '.sig'
EXEC master.sys.xp_cmdshell #bcp_sql
EXEC (#bcp_sql);
END
It was working fine before with proper data format in .dat.
From couple of days, sometimes .dat file contain no "B" (body) data. Sometimes SIG file appends T 2 times to .DAT file on PROD environment. Sometimes it gives correct output also. This issue is coming only on PROD environment
Any idea why it is behaving like this?
I´m trying to export to XML format using BCP and the XML file is generated correctly but the actual content seems wrong. Can anyone help please?
When I try to open the XML in a browser I get the following error message:
This page contains the following errors: error on line 1 at column 62:
Extra content at the end of the document
The SQL select that I'm using is:
DECLARE #fileName VARCHAR(50)
DECLARE #sqlStr VARCHAR(1000)
DECLARE #sqlCmd VARCHAR(1000)
SET #fileName = 'c:\fund_lib\test.xml'
USE PORT_APP_SQL
DROP TABLE ##temp;
WITH cte1
AS (SELECT LTRIM(RTRIM(codigo)) AS code,
CONVERT(VARCHAR(10), fecha, 120) AS date,
precio AS NAV
FROM mpr_price_history
WHERE codigo IN( 'LU0038743380', 'LU0086913042', 'LU0265291665', 'LU0098860363',
'LU0128525689', 'LU0121204944', 'CZ0008474780', 'LU0363630376',
'LU0372180066', 'LU0271663857', 'LU0271663774', 'LU0363630707', 'LU0313643024' ))
SELECT *
INTO ##temp
FROM cte1
SET #sqlStr = 'select * from ##temp order by code, date desc FOR XML RAW (''code'');'
-- Save XML records to a file:
SET #sqlCmd = 'bcp "' + #sqlStr + '" queryout ' + #fileName
+ ' -S "MPR01\SQLEXPRESS" -T -w'
EXEC xp_cmdshell #sqlCmd
And this is the error message if I open it in Firefox ( sorry is in Spanish).
error message seems to be at the end of each line
I think that your whole query can be simplified... No need for a CTE or a temp table...
attention The solution for your problem is - probably! - the missing root node, as mentioned by #TT.. If adding a root node solves your problem, please do not accept my answer (although you might vote it up, if you like it :-) ).
Your problem might be bound to a mix of encodings. If your output includes special characters, there could be some problems when you mix 8-byte encoding (VARCHAR) and the output with option -w. Therefore I put this all in NVARCHAR(MAX)
My suggestion to get things slim:
USE PORT_APP_SQL;
DECLARE #fileName NVARCHAR(50) = 'c:\fund_lib\test.xml';
DECLARE #cmd NVARCHAR(MAX);
SET #cmd =
N'SELECT LTRIM(RTRIM(codigo)) AS code
,CONVERT(VARCHAR(10), fecha, 120) AS [date]
,precio AS NAV
FROM mpr_price_history
WHERE codigo IN( ''LU0038743380'', ''LU0086913042'', ''LU0265291665'', ''LU0098860363'',
''LU0128525689'', ''LU0121204944'', ''CZ0008474780'', ''LU0363630376'',
''LU0372180066'', ''LU0271663857'', ''LU0271663774'', ''LU0363630707'', ''LU0313643024'' )
ORDER BY code,[date] DESC
FOR XML RAW(''code''),ROOT(''root'');'
-- Save XML records to a file:
SET #cmd = N'bcp "' + #cmd + N'" queryout ' + #fileName
+ N' -S "MPR01\SQLEXPRESS" -T -w'
EXEC xp_cmdshell #cmd;
The reason is because the XML doesn't have a root path. This example based on your script should produce XML for which the browser doesn't complain:
DECLARE #fileName VARCHAR(50);
DECLARE #sqlStr VARCHAR(1000);
DECLARE #sqlCmd VARCHAR(1000);
SET #fileName = 'c:\temp\test.xml';
SELECT *
INTO ##temp
FROM (VALUES('LU0038743380',GETDATE(),1),
('LU0086913042',GETDATE(),2),
('LU0265291665',GETDATE(),3),
('LU0098860363',GETDATE(),4)) AS cte1(fecha,[date],nav);
SET #sqlStr = 'select (select * from ##temp FOR XML RAW(''code''),TYPE) FOR XML PATH(''data'');'
-- Save XML records to a file:
SET #sqlCmd = 'bcp "' + #sqlStr + '" queryout ' + #fileName
+ ' -S '+##SERVERNAME+' -T -w';
EXEC xp_cmdshell #sqlCmd;
DROP TABLE ##temp;
I have ran your code with small alterations with no errors.
It seems the issue is with your data.
Look for special XML characters.
I'm having issues with SQL Server's BCP. I'm not getting an error message but the command is also not writing anything to the path specified.
Here is my code:
In this scenario, the only variable not accounted for in the code below is #MESSAGE which is a NVARCHAR(MAX) and will take up most of that space so VARCHAR(8000) won't be nearly big enough.
DECLARE #OUTPUT_TABLE VARCHAR(255) = '##temp' + CONVERT(VARCHAR(12), CONVERT(INT, RAND() * 1000000))
DECLARE #RESULT INTEGER
DECLARE #OUTPUT_EXECUTION_COMMAND NVARCHAR(MAX) = 'CREATE TABLE [' + #OUTPUT_TABLE + '] ( CONTENT NVARCHAR(MAX) )
INSERT INTO [' + #OUTPUT_TABLE + ']
SELECT #MESSAGE'
EXECUTE SP_EXECUTESQL #OUTPUT_EXECUTION_COMMAND, N'#MESSAGE NVARCHAR(MAX)', #MESSAGE
DECLARE #OUTPUT_FILE_WRITE_COMMAND VARCHAR(1000) = 'BCP ”SELECT CONTENT FROM [' + #OUTPUT_TABLE + ']"
QUERYOUT "' + #FULL_PATH + '\' + #FILE_NAME + '" -w -T -S ' + ##servername
PRINT #OUTPUT_FILE_WRITE_COMMAND
EXECUTE #RESULT = master.sys.xp_cmdshell #OUTPUT_FILE_WRITE_COMMAND, NO_OUTPUT
PRINT #RESULT
EXECUTE ( 'DROP TABLE [' + #OUTPUT_TABLE + ']' )
So, the frustrating part is that I'm receiving an error, yet my file is not being written.
The output of PRINT #OUTPUT_FILE_WRITE_COMMAND is:
BCP ”SELECT CONTENT FROM [##temp878274]"
QUERYOUT "\\TXPDC-FS01\Profiles\cofarmer\My Sandbox\THCIC\Q2_2014\Burleson\PIPSUB2938718184092014251607.txt" -w -T -S TXPDC-STKSQL01
While the output of PRINT #RESULT is: 1
Yet nothing is being written. What am I doing wrong?
Looks like the issue is the line break in the#OUTPUT_FILE_WRITE_COMMAND (and the erroneous double quote”). Removing the line break worked for me:
DECLARE #OUTPUT_FILE_WRITE_COMMAND VARCHAR(1000) = 'BCP "SELECT CONTENT FROM [' + #OUTPUT_TABLE + ']" QUERYOUT "' + #FULL_PATH + '\' + #FILE_NAME + '" -w -T -S ' + ##servername
Result: 0 and the file was created.
I want to send SQL Server output to a file which is working as expected. But if I pass the file path in a variable, It is not creating the file.
Working
:out 'C:\Temp.txt'
Not Working
DECLARE #BCVFileName VARCHAR(500)
SET #BCVFileName= 'C:\Temp.txt'
:out #BCVFileName
Could anyone please help me on this??
Thanks,
Mahesh
For this you need to store your query in Query file and then you can execute from command to store result to a text file as shown below.
>sqlcmd -E -S(Local)\SQLExpress -YourDBName -iC:\myQuery -oC:\Output.txt
Another way to use T-SQL and create a batch statement to execute via XP_CMDSHELL. Following script will help you to do the same just replace your query, ServerName with used variables.
SET NOCOUNT ON;
GO
DECLARE #sqlcmd varchar(1000);
PRINT 'using SqlCMD.exe';
SET #sqlcmd = 'sqlcmd -S' + ##SERVERNAME + ' -E -oc:\outfile.txt '
DECLARE #cmd varchar(1000);
SET #cmd = '-Q"SELECT RIGHT(REPLICATE('' '' , 6) + CONVERT(varchar, AddressID), 6 ) AS AddressID, CONVERT(varchar(10), ModifiedDate, 121) AS ModifiedDate FROM AdventureWorks.Person.Address ORDER BY ModifiedDate;"';
SET #sqlcmd = #sqlcmd + #cmd;
--SELECT #sqlcmd;
EXEC xp_cmdshell #sqlcmd, no_output;
PRINT 'using BCP.exe';
SET #cmd = '"SELECT RIGHT(REPLICATE('' '' , 6) + CONVERT(varchar, AddressID), 6 ) AS AddressID, CONVERT(varchar(10), ModifiedDate, 121) AS ModifiedDate FROM AdventureWorks.Person.Address ORDER BY ModifiedDate;"';
SET #sqlcmd = 'bcp ' + #cmd + ' queryout "c:\outfile2.txt" -c -T -S' + ##SERVERNAME
--SELECT #sqlcmd;
EXEC xp_cmdshell #sqlcmd, no_output;
regards
I have attached the code for storing the stored procedure output to an Excel sheet..
but I have few issues, they are
(1) all the columns are displayed in the first column of the excel sheet..which I don't need, I need to show the report in diff columns.. how to do that..
(2) if I use inner join with the main, how can I get the column names of the other table
Can anyone please help me out to solve the above issues?
alter procedure proc_generate_excel_with_columns
(
#db_name varchar(100),
#schm_name varchar(100),
#table_name varchar(100),
#file_name varchar(100)
)
as
--Generate column names as a recordset
declare #columns varchar(8000), #sql varchar(8000), #data_file varchar(100)
select
#columns = coalesce(#columns+',', '') + column_name + ' as ' + column_name
from
information_schema.columns
where
table_name = #table_name
select
#columns = '''' + replace(replace(#columns, ' as ', ''' as '),',',',''')
--Create a dummy file to have actual data
select
#data_file = substring(#file_name, 1, len(#file_name) -
charindex('\', reverse(#file_name))) + '\data_file.xls'
--Generate column names in the passed EXCEL file
set #sql='bcp " select * from (select ' + #columns + ') as t" queryout
c:\test.xls -c -t, -T -S ' + ##servername
exec master..xp_cmdshell #sql
--Generate data in the dummy file
set #sql = 'bcp "select * from st..VEN_FULLREPORTMASTER
where entry_date = convert(varchar, getdate() - 3, 105) "
queryout c:\data_file.xls -c -t, -T -S' + ##servername
-- exec master..xp_cmdshell #sql
-- Copy dummy file to passed EXCEL file
set #sql = 'exec master..xp_cmdshell ''type '+#data_file+' >> "'+#file_name+'"'''
exec(#sql)
----Delete dummy file
set #sql = 'exec master..xp_cmdshell ''del ' + #data_file + ''''
exec(#sql)
If using SSIS is an option this would be a trivial task to complete using a data flow task.
Can you use SSIS or do you need a pure tSQL solution?