I'm having issues with SQL Server's BCP. I'm not getting an error message but the command is also not writing anything to the path specified.
Here is my code:
In this scenario, the only variable not accounted for in the code below is #MESSAGE which is a NVARCHAR(MAX) and will take up most of that space so VARCHAR(8000) won't be nearly big enough.
DECLARE #OUTPUT_TABLE VARCHAR(255) = '##temp' + CONVERT(VARCHAR(12), CONVERT(INT, RAND() * 1000000))
DECLARE #RESULT INTEGER
DECLARE #OUTPUT_EXECUTION_COMMAND NVARCHAR(MAX) = 'CREATE TABLE [' + #OUTPUT_TABLE + '] ( CONTENT NVARCHAR(MAX) )
INSERT INTO [' + #OUTPUT_TABLE + ']
SELECT #MESSAGE'
EXECUTE SP_EXECUTESQL #OUTPUT_EXECUTION_COMMAND, N'#MESSAGE NVARCHAR(MAX)', #MESSAGE
DECLARE #OUTPUT_FILE_WRITE_COMMAND VARCHAR(1000) = 'BCP ”SELECT CONTENT FROM [' + #OUTPUT_TABLE + ']"
QUERYOUT "' + #FULL_PATH + '\' + #FILE_NAME + '" -w -T -S ' + ##servername
PRINT #OUTPUT_FILE_WRITE_COMMAND
EXECUTE #RESULT = master.sys.xp_cmdshell #OUTPUT_FILE_WRITE_COMMAND, NO_OUTPUT
PRINT #RESULT
EXECUTE ( 'DROP TABLE [' + #OUTPUT_TABLE + ']' )
So, the frustrating part is that I'm receiving an error, yet my file is not being written.
The output of PRINT #OUTPUT_FILE_WRITE_COMMAND is:
BCP ”SELECT CONTENT FROM [##temp878274]"
QUERYOUT "\\TXPDC-FS01\Profiles\cofarmer\My Sandbox\THCIC\Q2_2014\Burleson\PIPSUB2938718184092014251607.txt" -w -T -S TXPDC-STKSQL01
While the output of PRINT #RESULT is: 1
Yet nothing is being written. What am I doing wrong?
Looks like the issue is the line break in the#OUTPUT_FILE_WRITE_COMMAND (and the erroneous double quote”). Removing the line break worked for me:
DECLARE #OUTPUT_FILE_WRITE_COMMAND VARCHAR(1000) = 'BCP "SELECT CONTENT FROM [' + #OUTPUT_TABLE + ']" QUERYOUT "' + #FULL_PATH + '\' + #FILE_NAME + '" -w -T -S ' + ##servername
Result: 0 and the file was created.
Related
How can I ignore null's in my exported .csv file, I used bcp utility
within sql server, here it is how this looks like altogether.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
IF OBJECT_ID ('[dbo].[generateCSV]') IS NOT NULL
DROP PROCEDURE [dbo].[generateCSV]
GO
CREATE PROCEDURE [dbo].[generateCSV]
(
#table varchar(100),
#output varchar(100),
#date varchar(12),
#server varchar(30)
)
AS
DECLARE #sql varchar(8000)
SELECT #sql = 'bcp "select * from ' + DB_NAME() + '.dbo.' + #table + '
where reportingdate = ''' + #date + '''"' + ' queryout ' + #output + ' -c
-C65001 -t";" -r"\n" -T -S' + #server
exec master..xp_cmdshell #sql
-- Main EXEC
EXEC dbo.generateCSV #table = 'Clients', #date = '2017-10-31', #output =
'//172.18.16.109/share/Test.csv (server with export target location ) ',
#server = '172.18.16.108(server we are connected to and from which we are
taking the data)'
After exporting and opening my file in notepad++ empty columns end up filled with NULL's.
What I want it to look like is...
Edit:
I included ISNULL() in my select clause but still notepad will read those columns as null valued.
SELECT #sql = 'bcp "select
ReportingDate,uniqClientID,registrationNumber,name,ISNULL(vatNumber,'''') as
vatNumber,ISNULL(entityStatusCode,'''') as entityStatusCode,
maximumLifetimeDPD from ' + DB_NAME() + '.dbo.' +
#table + ' where ReportingDate = ''' + #date + '''"' + ' queryout ' +
#output + ' -c -C65001 -t";" -r"\n" -T -S' + #server
SELECT #sql = 'bcp "select * from ' + DB_NAME() + '.dbo.' + #table +
' where reportingdate = ''' + #date + '''"' + ' queryout ' + #output
+ ' -c
-C65001 -t";" -r"\n" -T -S' + #server
Instead of SELECT * write the specific column names.
Wherever you feel NULL can appear you can write like following.
....
SELECT
Column1,
ISNULL(Column2,'') AS Column2,
Column3,
....
from ' + DB_NAME() + '.dbo.' + #table
.....
Note: If you have INT as datatype, you need to slightly change your select according to following query.
DECLARE #INTVAL INT
SET #INTVAL = NULL
SELECT
ISNULL(LTRIM(NULLIF(#INTVAL, 0)), '') AS ColumnName
I have a procedure which is for exporting data from table to .DAT file. There's a job running on it which calls SP every 10 Min. .DAT file contains H= column names, B= table data, T= No of rows processed(.SIG file appends this to .DAT file).
Following is the stored procedure :
CREATE PROCEDURE [dbo].[Test_Export] #count2 VARCHAR(10)
AS
BEGIN
SET NOCOUNT ON;
--Alerts_Balance
DECLARE #bcp_sql VARCHAR(1000)
DECLARE #filename VARCHAR(200), #count1 VARCHAR(10)=20000
SET #filename = 'C:\Test'+'.DAT';
SELECT #bcp_sql = 'bcp "SELECT ''H'',''Name'',''City'',''State'',''Country'' union all SELECT top 10 ''B'', [Name],[City],[State],[Country] FROM Test_Table" queryout ' + #filename + ' -c -t, -T -S '+ ##servername
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'bcp "SELECT ''T'', '+cast(IIF(cast(#count1 as int)>cast(#count2 as int), #count2+2, #count1+2) as varchar(10))+'" queryout '+ #filename + '.sig -c -t, -T -S '+ ##servername
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'type ' + #filename + '.sig >> "' + #filename + '"'
EXEC master.sys.xp_cmdshell #bcp_sql
SELECT #bcp_sql = 'copy nul: ' + #filename + '.sig'
EXEC master.sys.xp_cmdshell #bcp_sql
EXEC (#bcp_sql);
END
It was working fine before with proper data format in .dat.
From couple of days, sometimes .dat file contain no "B" (body) data. Sometimes SIG file appends T 2 times to .DAT file on PROD environment. Sometimes it gives correct output also. This issue is coming only on PROD environment
Any idea why it is behaving like this?
Below script suppose to be executed on multiple DBs, however on parsing gives below error-
Msg 102, Level 15, State 1, Line 40
Incorrect syntax near '+'.
declare #list table (name varchar(128) not null);
fill the list with your custom subset
insert into #list
select name from sys.databases where name like '%connect%';
--select * from #list
--select min(name) from #list
--read the script
declare
#DBServerName varchar(128),
#FilePathName varchar(128),
#DBName varchar(128)
;
set #DBServerName = 'peeyushp-w8';
set #FilePathName = 'D:\script.sql';
set #DBName = '';
--EXEC xp_cmdshell 'sqlcmd -S ' + #DBServerName + ' -d ' + #DBName + ' -i ' + #FilePathName
declare
#db varchar(128),
#t varchar(max),
#s varchar(max)
;
--set #t = 'use {db}; exec sp_spaceused';
set #db = (select min(name) from #list);
while #db is not null
begin
--set #s = replace(#t, '{db}', ' -d' + #db);
--exec (#s);
EXEC xp_cmdshell 'sqlcmd -S ' + #DBServerName + #s + ' -i ' + replace(#FilePathName, '{db}', ' -d' + #db);
set #db = (select min(name) from #list where name > #db);
end
GO
Error is coming on 'xp_cmdshell' exec statement. Am i missing something which to be included in my SQL script. Will appreciate any help on the same.
According to PROCEDURE's Limitations and Restrictions in T-SQL :
You cannot specify a function name as a parameter default value or as the value passed to a parameter when executing a procedure. However, you can pass a function as a variable as shown in the following example.
So, it is impossible to use directly the string concatenation as procedure parameter. It must be declared before :
DECLARE #cmd VARCHAR(400)
SET #cmd = 'sqlcmd -S ' + #DBServerName + ' -d ' + #DBName + ' -i ' + #FilePathName
-- Now you can use it
EXEC xp_cmdshell #cmd
I'm building a stored procedure to transfer data from SQL Server to a remote FTP server.
I wrote the table data to a file with the following query
exec xp_cmdshell "bcp master.dbo.table out c:\temp\bcp_outputFile.txt -S(local)\SQLEXPRESS -T -c"
However, I'm struggling with building a script that would take that file and send it across to the FTP Server.
Using SSIS is not an option. It must be done with the xp_cmdshell command.
Any ideas?
Thanks
Use the below mention procedure for uploading the file in the ftp server.
here in the procedure i have hard coded the ftp details you can change it.
create procedure [dbo].[uploadfile_ftp]
#SourcePath varchar(128),
#SourceFiles varchar(128),
#DestPath varchar(128)
AS
Begin
DECLARE #FTPServer varchar(128)
DECLARE #FTPUser varchar(128)
DECLARE #FTPPwd varchar(128)
DECLARE #FTPMode varchar(10)
-- FTP attributes.
SET #FTPServer = 'ftp url'
SET #FTPUser = 'ftp username'
SET #FTPPwd = 'ftp password'
SET #FTPMode = 'binary' -- ascii, binary or blank for default.
DECLARE #cmd varchar(1000)
DECLARE #workfile varchar(128)
DECLARE #nowstr varchar(25)
-- Get the %TEMP% environment variable.
DECLARE #tempdir varchar(128)
CREATE TABLE #tempvartable(info VARCHAR(1000))
INSERT #tempvartable EXEC master..xp_cmdshell 'echo %temp%'
SET #tempdir = (SELECT top 1 info FROM #tempvartable)
IF RIGHT(#tempdir, 1) <> '\' SET #tempdir = #tempdir + '\'
DROP TABLE #tempvartable
-- Generate #workfile
SET #nowstr = replace(replace(convert(varchar(30), GETDATE(), 121), ' ', '_'), ':', '-')
SET #workfile = 'FTP_SPID' + convert(varchar(128), ##spid) + '_' + #nowstr + '.txt'
-- Deal with special chars for echo commands.
select #FTPServer = replace(replace(replace(#FTPServer, '|', '^|'),'<','^<'),'>','^>')
select #FTPUser = replace(replace(replace(#FTPUser, '|', '^|'),'<','^<'),'>','^>')
select #FTPPwd = replace(replace(replace(#FTPPwd, '&', '^&'),'<','^<'),'>','^>')
select #DestPath = replace(replace(replace(#DestPath, '|', '^|'),'<','^<'),'>','^>')
IF RIGHT(#SourcePath, 1) <> '\' SET #SourcePath = #SourcePath + '\'
-- Build the FTP script file.
select #cmd = 'echo ' + 'open ' + #FTPServer + ' > ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
print #cmd;
select #cmd = 'echo ' + #FTPUser + '>> ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
print #cmd;
select #cmd = 'echo ' + #FTPPwd + '>> ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
print #cmd;
select #cmd = 'echo ' + 'prompt ' + ' >> ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
print #cmd;
IF LEN(#FTPMode) > 0
BEGIN
select #cmd = 'echo ' + #FTPMode + ' >> ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
END
IF LEN(#DestPath) > 0
BEGIN
select #cmd = 'echo ' + 'cd ' + #DestPath + ' >> ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
END
select #cmd = 'echo ' + 'mput ' + #SourcePath + #SourceFiles + ' >> ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
select #cmd = 'echo ' + 'quit' + ' >> ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
-- Execute the FTP command via script file.
select #cmd = 'ftp -s:' + #tempdir + #workfile
print #cmd;
create table #a (id int identity(1,1), s varchar(1000))
insert #a
EXEC master..xp_cmdshell #cmd
select id, ouputtmp = s from #a
-- Clean up.
drop table #a
select #cmd = 'del ' + #tempdir + #workfile
EXEC master..xp_cmdshell #cmd
End
I have attached the code for storing the stored procedure output to an Excel sheet..
but I have few issues, they are
(1) all the columns are displayed in the first column of the excel sheet..which I don't need, I need to show the report in diff columns.. how to do that..
(2) if I use inner join with the main, how can I get the column names of the other table
Can anyone please help me out to solve the above issues?
alter procedure proc_generate_excel_with_columns
(
#db_name varchar(100),
#schm_name varchar(100),
#table_name varchar(100),
#file_name varchar(100)
)
as
--Generate column names as a recordset
declare #columns varchar(8000), #sql varchar(8000), #data_file varchar(100)
select
#columns = coalesce(#columns+',', '') + column_name + ' as ' + column_name
from
information_schema.columns
where
table_name = #table_name
select
#columns = '''' + replace(replace(#columns, ' as ', ''' as '),',',',''')
--Create a dummy file to have actual data
select
#data_file = substring(#file_name, 1, len(#file_name) -
charindex('\', reverse(#file_name))) + '\data_file.xls'
--Generate column names in the passed EXCEL file
set #sql='bcp " select * from (select ' + #columns + ') as t" queryout
c:\test.xls -c -t, -T -S ' + ##servername
exec master..xp_cmdshell #sql
--Generate data in the dummy file
set #sql = 'bcp "select * from st..VEN_FULLREPORTMASTER
where entry_date = convert(varchar, getdate() - 3, 105) "
queryout c:\data_file.xls -c -t, -T -S' + ##servername
-- exec master..xp_cmdshell #sql
-- Copy dummy file to passed EXCEL file
set #sql = 'exec master..xp_cmdshell ''type '+#data_file+' >> "'+#file_name+'"'''
exec(#sql)
----Delete dummy file
set #sql = 'exec master..xp_cmdshell ''del ' + #data_file + ''''
exec(#sql)
If using SSIS is an option this would be a trivial task to complete using a data flow task.
Can you use SSIS or do you need a pure tSQL solution?