I have attached the code for storing the stored procedure output to an Excel sheet..
but I have few issues, they are
(1) all the columns are displayed in the first column of the excel sheet..which I don't need, I need to show the report in diff columns.. how to do that..
(2) if I use inner join with the main, how can I get the column names of the other table
Can anyone please help me out to solve the above issues?
alter procedure proc_generate_excel_with_columns
(
#db_name varchar(100),
#schm_name varchar(100),
#table_name varchar(100),
#file_name varchar(100)
)
as
--Generate column names as a recordset
declare #columns varchar(8000), #sql varchar(8000), #data_file varchar(100)
select
#columns = coalesce(#columns+',', '') + column_name + ' as ' + column_name
from
information_schema.columns
where
table_name = #table_name
select
#columns = '''' + replace(replace(#columns, ' as ', ''' as '),',',',''')
--Create a dummy file to have actual data
select
#data_file = substring(#file_name, 1, len(#file_name) -
charindex('\', reverse(#file_name))) + '\data_file.xls'
--Generate column names in the passed EXCEL file
set #sql='bcp " select * from (select ' + #columns + ') as t" queryout
c:\test.xls -c -t, -T -S ' + ##servername
exec master..xp_cmdshell #sql
--Generate data in the dummy file
set #sql = 'bcp "select * from st..VEN_FULLREPORTMASTER
where entry_date = convert(varchar, getdate() - 3, 105) "
queryout c:\data_file.xls -c -t, -T -S' + ##servername
-- exec master..xp_cmdshell #sql
-- Copy dummy file to passed EXCEL file
set #sql = 'exec master..xp_cmdshell ''type '+#data_file+' >> "'+#file_name+'"'''
exec(#sql)
----Delete dummy file
set #sql = 'exec master..xp_cmdshell ''del ' + #data_file + ''''
exec(#sql)
If using SSIS is an option this would be a trivial task to complete using a data flow task.
Can you use SSIS or do you need a pure tSQL solution?
Related
How can I ignore null's in my exported .csv file, I used bcp utility
within sql server, here it is how this looks like altogether.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
IF OBJECT_ID ('[dbo].[generateCSV]') IS NOT NULL
DROP PROCEDURE [dbo].[generateCSV]
GO
CREATE PROCEDURE [dbo].[generateCSV]
(
#table varchar(100),
#output varchar(100),
#date varchar(12),
#server varchar(30)
)
AS
DECLARE #sql varchar(8000)
SELECT #sql = 'bcp "select * from ' + DB_NAME() + '.dbo.' + #table + '
where reportingdate = ''' + #date + '''"' + ' queryout ' + #output + ' -c
-C65001 -t";" -r"\n" -T -S' + #server
exec master..xp_cmdshell #sql
-- Main EXEC
EXEC dbo.generateCSV #table = 'Clients', #date = '2017-10-31', #output =
'//172.18.16.109/share/Test.csv (server with export target location ) ',
#server = '172.18.16.108(server we are connected to and from which we are
taking the data)'
After exporting and opening my file in notepad++ empty columns end up filled with NULL's.
What I want it to look like is...
Edit:
I included ISNULL() in my select clause but still notepad will read those columns as null valued.
SELECT #sql = 'bcp "select
ReportingDate,uniqClientID,registrationNumber,name,ISNULL(vatNumber,'''') as
vatNumber,ISNULL(entityStatusCode,'''') as entityStatusCode,
maximumLifetimeDPD from ' + DB_NAME() + '.dbo.' +
#table + ' where ReportingDate = ''' + #date + '''"' + ' queryout ' +
#output + ' -c -C65001 -t";" -r"\n" -T -S' + #server
SELECT #sql = 'bcp "select * from ' + DB_NAME() + '.dbo.' + #table +
' where reportingdate = ''' + #date + '''"' + ' queryout ' + #output
+ ' -c
-C65001 -t";" -r"\n" -T -S' + #server
Instead of SELECT * write the specific column names.
Wherever you feel NULL can appear you can write like following.
....
SELECT
Column1,
ISNULL(Column2,'') AS Column2,
Column3,
....
from ' + DB_NAME() + '.dbo.' + #table
.....
Note: If you have INT as datatype, you need to slightly change your select according to following query.
DECLARE #INTVAL INT
SET #INTVAL = NULL
SELECT
ISNULL(LTRIM(NULLIF(#INTVAL, 0)), '') AS ColumnName
I have a procedure which is for exporting data from table to .DAT file. There's a job running on it which calls SP every 10 Min. .DAT file contains H= column names, B= table data, T= No of rows processed(.SIG file appends this to .DAT file).
Following is the stored procedure :
CREATE PROCEDURE [dbo].[Test_Export] #count2 VARCHAR(10)
AS
BEGIN
SET NOCOUNT ON;
--Alerts_Balance
DECLARE #bcp_sql VARCHAR(1000)
DECLARE #filename VARCHAR(200), #count1 VARCHAR(10)=20000
SET #filename = 'C:\Test'+'.DAT';
SELECT #bcp_sql = 'bcp "SELECT ''H'',''Name'',''City'',''State'',''Country'' union all SELECT top 10 ''B'', [Name],[City],[State],[Country] FROM Test_Table" queryout ' + #filename + ' -c -t, -T -S '+ ##servername
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'bcp "SELECT ''T'', '+cast(IIF(cast(#count1 as int)>cast(#count2 as int), #count2+2, #count1+2) as varchar(10))+'" queryout '+ #filename + '.sig -c -t, -T -S '+ ##servername
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'type ' + #filename + '.sig >> "' + #filename + '"'
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'copy nul: ' + #filename + '.sig'
EXEC master.sys.xp_cmdshell #bcp_sql
EXEC (#bcp_sql);
END
It was working fine before with proper data format in .dat.
From couple of days, sometimes .dat file contain no "B" (body) data. Sometimes SIG file appends T 2 times to .DAT file on PROD environment. Sometimes it gives correct output also. This issue is coming only on PROD environment
Any idea why it is behaving like this?
I´m trying to export to XML format using BCP and the XML file is generated correctly but the actual content seems wrong. Can anyone help please?
When I try to open the XML in a browser I get the following error message:
This page contains the following errors: error on line 1 at column 62:
Extra content at the end of the document
The SQL select that I'm using is:
DECLARE #fileName VARCHAR(50)
DECLARE #sqlStr VARCHAR(1000)
DECLARE #sqlCmd VARCHAR(1000)
SET #fileName = 'c:\fund_lib\test.xml'
USE PORT_APP_SQL
DROP TABLE ##temp;
WITH cte1
AS (SELECT LTRIM(RTRIM(codigo)) AS code,
CONVERT(VARCHAR(10), fecha, 120) AS date,
precio AS NAV
FROM mpr_price_history
WHERE codigo IN( 'LU0038743380', 'LU0086913042', 'LU0265291665', 'LU0098860363',
'LU0128525689', 'LU0121204944', 'CZ0008474780', 'LU0363630376',
'LU0372180066', 'LU0271663857', 'LU0271663774', 'LU0363630707', 'LU0313643024' ))
SELECT *
INTO ##temp
FROM cte1
SET #sqlStr = 'select * from ##temp order by code, date desc FOR XML RAW (''code'');'
-- Save XML records to a file:
SET #sqlCmd = 'bcp "' + #sqlStr + '" queryout ' + #fileName
+ ' -S "MPR01\SQLEXPRESS" -T -w'
EXEC xp_cmdshell #sqlCmd
And this is the error message if I open it in Firefox ( sorry is in Spanish).
error message seems to be at the end of each line
I think that your whole query can be simplified... No need for a CTE or a temp table...
attention The solution for your problem is - probably! - the missing root node, as mentioned by #TT.. If adding a root node solves your problem, please do not accept my answer (although you might vote it up, if you like it :-) ).
Your problem might be bound to a mix of encodings. If your output includes special characters, there could be some problems when you mix 8-byte encoding (VARCHAR) and the output with option -w. Therefore I put this all in NVARCHAR(MAX)
My suggestion to get things slim:
USE PORT_APP_SQL;
DECLARE #fileName NVARCHAR(50) = 'c:\fund_lib\test.xml';
DECLARE #cmd NVARCHAR(MAX);
SET #cmd =
N'SELECT LTRIM(RTRIM(codigo)) AS code
,CONVERT(VARCHAR(10), fecha, 120) AS [date]
,precio AS NAV
FROM mpr_price_history
WHERE codigo IN( ''LU0038743380'', ''LU0086913042'', ''LU0265291665'', ''LU0098860363'',
''LU0128525689'', ''LU0121204944'', ''CZ0008474780'', ''LU0363630376'',
''LU0372180066'', ''LU0271663857'', ''LU0271663774'', ''LU0363630707'', ''LU0313643024'' )
ORDER BY code,[date] DESC
FOR XML RAW(''code''),ROOT(''root'');'
-- Save XML records to a file:
SET #cmd = N'bcp "' + #cmd + N'" queryout ' + #fileName
+ N' -S "MPR01\SQLEXPRESS" -T -w'
EXEC xp_cmdshell #cmd;
The reason is because the XML doesn't have a root path. This example based on your script should produce XML for which the browser doesn't complain:
DECLARE #fileName VARCHAR(50);
DECLARE #sqlStr VARCHAR(1000);
DECLARE #sqlCmd VARCHAR(1000);
SET #fileName = 'c:\temp\test.xml';
SELECT *
INTO ##temp
FROM (VALUES('LU0038743380',GETDATE(),1),
('LU0086913042',GETDATE(),2),
('LU0265291665',GETDATE(),3),
('LU0098860363',GETDATE(),4)) AS cte1(fecha,[date],nav);
SET #sqlStr = 'select (select * from ##temp FOR XML RAW(''code''),TYPE) FOR XML PATH(''data'');'
-- Save XML records to a file:
SET #sqlCmd = 'bcp "' + #sqlStr + '" queryout ' + #fileName
+ ' -S '+##SERVERNAME+' -T -w';
EXEC xp_cmdshell #sqlCmd;
DROP TABLE ##temp;
I have ran your code with small alterations with no errors.
It seems the issue is with your data.
Look for special XML characters.
I have code like this run with no error and I got result.
DECLARE #fileName VARCHAR(50)
DECLARE #sqlStr VARCHAR(1000)
DECLARE #sqlCmd VARCHAR(1000)
SET #fileName = 'C:\SQL_Queries\test.xml'
SET #sqlStr = 'select * from ##tmpEmployeeJob FOR XML PATH(''EmployeeJob''), ROOT(''Pos.EmployeeJob'')'
SET #sqlCmd = 'bcp "' + #sqlStr + '" queryout ' + #fileName + ' -w -T'
EXEC xp_cmdshell #sqlCmd
But when I want to do multiple select in sqlstr like this
SET #sqlStr = 'select(select * from ##tmpEmployeeJob FOR XML PATH(''EmployeeJob''), ROOT(''Pos.EmployeeJob''),
select * from ##tmpCity FOR XML PATH(''City''), ROOT(''Pos.City''))'
I got this error
Incorrect syntax near 'EmployeeJob'
How can I do the multiple select like above?
When you exporting xml you can easily compose xml strings from two or more queries. Or you can export table with more rows (where rows can be xml data type).
Below script suppose to be executed on multiple DBs, however on parsing gives below error-
Msg 102, Level 15, State 1, Line 40
Incorrect syntax near '+'.
declare #list table (name varchar(128) not null);
fill the list with your custom subset
insert into #list
select name from sys.databases where name like '%connect%';
--select * from #list
--select min(name) from #list
--read the script
declare
#DBServerName varchar(128),
#FilePathName varchar(128),
#DBName varchar(128)
;
set #DBServerName = 'peeyushp-w8';
set #FilePathName = 'D:\script.sql';
set #DBName = '';
--EXEC xp_cmdshell 'sqlcmd -S ' + #DBServerName + ' -d ' + #DBName + ' -i ' + #FilePathName
declare
#db varchar(128),
#t varchar(max),
#s varchar(max)
;
--set #t = 'use {db}; exec sp_spaceused';
set #db = (select min(name) from #list);
while #db is not null
begin
--set #s = replace(#t, '{db}', ' -d' + #db);
--exec (#s);
EXEC xp_cmdshell 'sqlcmd -S ' + #DBServerName + #s + ' -i ' + replace(#FilePathName, '{db}', ' -d' + #db);
set #db = (select min(name) from #list where name > #db);
end
GO
Error is coming on 'xp_cmdshell' exec statement. Am i missing something which to be included in my SQL script. Will appreciate any help on the same.
According to PROCEDURE's Limitations and Restrictions in T-SQL :
You cannot specify a function name as a parameter default value or as the value passed to a parameter when executing a procedure. However, you can pass a function as a variable as shown in the following example.
So, it is impossible to use directly the string concatenation as procedure parameter. It must be declared before :
DECLARE #cmd VARCHAR(400)
SET #cmd = 'sqlcmd -S ' + #DBServerName + ' -d ' + #DBName + ' -i ' + #FilePathName
-- Now you can use it
EXEC xp_cmdshell #cmd