Send Query result into a Text file in SqlServer - sql-server

I want to send the result from a scalar variable into a text file,
My Code looks like this
DECLARE #test varchar(10)
SET #test='This is sample text'
EXEC master..xp_cmdshell'bcp ' + #test + ' queryout "D:\sample.txt" -S LocalHost -U
sa -P 123 -c -T -t'
But this is showing the following errors
output
usage: bcp {dbtable | query} {in | out | queryout | format} datafile
[-m maxerrors] [-f formatfile] [-e errfile]
[-F firstrow] [-L lastrow] [-b batchsize]
[-n native type] [-c character type] [-w wide character type]
[-N keep non-text native] [-V file format version] [-q quoted identifier]
[-C code page specifier] [-t field terminator] [-r row terminator]
[-i inputfile] [-o outfile] [-a packetsize]
[-S server name] [-U username] [-P password]
[-T trusted connection] [-v version] [-R regional enable]
[-k keep null values] [-E keep identity values]
[-h "load hints"] [-x generate xml format file]
NULL
Dont know how to give the format assigning a scalar variable value in bcp command
Please any one help.
And tried in this way also
Create table #sample
(
productid int
)
Insert into #sample(productid) values(1001098)
EXEC master..xp_cmdshell'bcp "select * from #sample" queryout "D:\sample.txt" -S
LocalHost -U sa -P 123 -c -T -t'
It gives as Error that
#sample does not exist (in bcp command line)
Can any one please solve this.
Thanks in advance.

First thing, a temp table is bound to a scope, and thus cannot be used in a different process ID.
But, you can declare a global temp table (be sure to delete it afterward and make sure to use a very specific name to make sure you don't interfere with some other code).
To do so, you just have to double the '#' in the table name.
if object_ID('tempdb..##sample') is not null
drop table ##sample
Create table ##sample
(
productid int
)
Insert into ##sample(productid) values(1001098)
Then all you have to do is your output.
EXEC master..xp_cmdshell'bcp "select * from ##sample" queryout "d:\sample.txt" -w -U sa -P 123 -S server_name\instance_name'
As you can see I changed a couple of switches too.
I found it's preferable to use -w to generate ansi characters, so I removed -c.
-T is for trusted connection, but since you provide a username and password, you don't need it.
Then you should be fine.

Test this:
EXEC xp_cmdshell 'bcp "SELECT *FROM [YOURDATABASE].[dbo].[YOURTABLE]" queryout "f:\newOUTPUT.txt" -S DESKTOP-A5CFJSH\MSSQLSERVER1 -UYOURUSERNAME -PYOURPASSWORD -n '

Related

BCP utility SQL Server Export to CSV - destination CSV file never gets created

I'm experiencing the following error when using the bcp utility in SQL Server 2017.
I need to export data from a view in SQL Server to a .CSV file using a comma as the column delimiter. My data contains comma in certain columns. So I used " as a column qualifier when created a view using [+'"' column_name + '"'].
I tried 3 different ways:
Option 1:
declare #sql varchar(8000)
select #sql = 'bcp "SELECT * FROM MyDB.dbo.MyTable
WHERE Rn BETWEEN 1 AND 100 ORDER BY Rn"
queryout "E:\MyFolder\MyFileName.txt" -c -t, -T -S' + ##servername
exec master..xp_cmdshell #sql
Option 2:
declare #sql varchar(8000)
select #sql = 'bcp "SELECT * FROM MyDB.dbo.MyTable
WHERE Rn BETWEEN 1 AND 100 ORDER BY Rn"
queryout "E:\MyFolder\MyFileName.txt" -c -t, [MyServer_Name] -T -S'
exec master..xp_cmdshell #sql
Option 3
declare #sql varchar(8000)
select #sql = 'bcp "SELECT * FROM MyDB.dbo.MyTable
WHERE Rn BETWEEN 1 AND 100 ORDER BY Rn"
queryout "E:\MyFolder\MyFileName.txt" -c -t, [MyServer_Name] -T -S ' + '##servername'
exec master..xp_cmdshell #sql
In every case I'm getting the same output but the CSV file never gets created:
usage: bcp {dbtable | query} {in | out | queryout | format} datafile
[-m maxerrors] [-f formatfile] [-e errfile]
[-F firstrow] [-L lastrow] [-b batchsize]
[-n native type] [-c character type] [-w wide character type]
[-N keep non-text native] [-V file format version] [-q quoted identifier]
[-C code page specifier] [-t field terminator] [-r row terminator]
[-i inputfile] [-o outfile] [-a packetsize]
[-S server name] [-U username] [-P password]
[-T trusted connection] [-v version] [-R regional enable]
[-k keep null values] [-E keep identity values]
[-h "load hints"] [-x generate xml format file]
[-d database name] [-K application intent] [-l login timeout]
NULL
PS: I found alternative solution to import data into .CSV from SQL Server. But the problem here is that sqlcmd - unlike bcp - does not accept two characters as text qualifier. I decided to use a text qualifier of 2 chars ~^ following comma as a field separator.
exec master..xp_cmdshell 'sqlcmd -s -W -Q "set nocount on;select * from MyTable" | findstr /v /c:"-" /b > "E:\MyFile.csv"'
where -s is col_separator
Specifies the column-separator character. The default is a blank space. This option sets the sqlcmd scripting variable SQLCMDCOLSEP. To use characters that have special meaning to the operating system such as the ampersand (&), or semicolon (;), enclose the character in quotation marks ("). The column separator can be any 8-bit character.( https://learn.microsoft.com/en-us/sql/tools/sqlcmd-utility?view=sql-server-ver15 )
Command lines supplied to xp_cmdshell cannot span multiple lines of text, they need to be specified completely on a single line. Try building the command as a series of concatenated strings like this:
declare #sql varchar(8000) =
'bcp' +
' "select * from MyDB.dbo.MyTable WHERE Rn BETWEEN 1 AND 100 order by Rn"' +
' queryout' +
' "E:\MyFolder\MyFileName.txt"' +
' -c -t, -T -S' + ##servername
exec master..xp_cmdshell #sql
When concatenating the strings be mindful of where you need to include white space between each segment so that command line parameters don't run into each other.
P.S.
I found alternative solution to Import data into CSV fro SQL Server. But the problem here is that sqlcmd unlike bcp does not except to characters as a text qualifier. I decided to use a text qualifier of 2 chars ~^ following comma as a filed separator.
exec master..xp_cmdshell 'sqlcmd -s -W -Q "set nocount on;select * from MyTable" | findstr /v /c:"-" /b > "E:\MyFile.csv"'
Where -s is col_separator Specifies the column-separator character. The default is a blank space. This option sets the sqlcmd scripting variable SQLCMDCOLSEP. To use characters that have special meaning to the operating system such as the ampersand (&), or semicolon (;), enclose the character in quotation marks ("). The column separator can be any 8-bit character.( https://learn.microsoft.com/en-us/sql/tools/sqlcmd-utility?view=sql-server-ver15 )
Also we have another alternative solution that includes headers, but there you have no control on how you pass your query.
set BCP_EXPORT_SERVER=YourServerName
set BCP_EXPORT_DB=YourDBName
set BCP_EXPORT_TABLE=YourTableName
BCP "DECLARE #colnames VARCHAR(max);SELECT #colnames = COALESCE(#colnames + ',', '') + column_name from %BCP_EXPORT_DB%.INFORMATION_SCHEMA.COLUMNS where TABLE_NAME='%BCP_EXPORT_TABLE%'; select #colnames;" queryout HeadersOnly.csv -c -t, -T -S%BCP_EXPORT_SERVER%
BCP %BCP_EXPORT_DB%.dbo.%BCP_EXPORT_TABLE% out TableDataWithoutHeaders.csv -c -t"|", -T -S%BCP_EXPORT_SERVER%
set BCP_EXPORT_SERVER=
set BCP_EXPORT_DB=
set BCP_EXPORT_TABLE=
copy /b HeadersOnly.csv+TableDataWithoutHeaders.csv TableData.csv
del HeadersOnly.csv
del TableDataWithoutHeaders.csv

Custom column name for bcp queryout SQL Server

I want to rename the column name in the select clause of a bcp queryout command.I've tried the below variations and none of them work.I want to rename the first and third column to email_address and id respectively.
I am calling the bcp command in a batch script.
bcp "select email as 'email_address', first_name, p_id as 'id' from table_name" queryout 15Days.txt -c -Sservername -Uusername -Ppassword -t,
bcp "select email as [email_address], first_name, p_id as [id] from table_name" queryout 15Days.txt -c -Sservername -Uusername -Ppassword -t,
bcp "select email 'email_address', first_name, p_id 'id' from table_name" queryout 15Days.txt -c -Sservername -Uusername -Ppassword -t,
bcp "select email email_address, first_name, p_id id from table_name" queryout 15Days.txt -c -Sservername -Uusername -Ppassword -t,
Can someone point me to towards the right solution?
If you looked at the text files produced by bcp you'll notice column names are not exported. Only data is exported.
The link Alex posted describes a way to add the column names to the output.You are actually adding the fields as first data column to your data using UNION. I recommend against it, because it requires casting all fields as strings making for a complex query.
Step 1 Output the columns
I would output a text file with the desired column names to one text file. Lets call that file "columnnames.csv". You may even create a repository of fixed column name files if need be.
Step 2 Output the data
Use bcp to output the data as you did before. Lets call that output "data.csv"
Step 3 Combine the two files
You can use this simple batch command to combine the data
copy /b columnnames.csv+data.csv combined.csv
or
type columnnames.csv data.csv > combined.csv
Helpful resources
How to query against the schema to get the column names.

Difference in "format" and "queryout" options available in BCP

I am trying to import data into Sybase ASE 15.7 (trial version) using BCP (version 10) on Windows.
To do so, I first try to create format file for the table with following command and get an error: Copy direction must be either 'in' or 'out'. Syntax Error in 'format'.
C:\Sybase\OCS-15_0\bin>bcp TEST_EMP2 format nul -f C:\test_files\TEST_EMP2.fmt -
c -T -t,
Copy direction must be either 'in' or 'out'.
Syntax Error in 'format'.
usage: bcp [[db_name.]owner.]table_name[:slice_num] [partition pname] {in | out}
[filename]
[-m maxerrors] [-f formatfile] [-e errfile] [-d discardfileprefix]
[-F firstrow] [-L lastrow] [-b batchsize]
[-n] [-c] [-t field_terminator] [-r row_terminator]
[-U username] [-P password] [-I interfaces_file] [-S server]
[-a display_charset] [-z language] [-v]
[-i input_file] [-o output_file]
[-A packet size] [-J client character set]
[-T text or image size] [-E] [-g id_start_value] [-N] [-W] [-X]
[-M LabelName LabelValue] [-labeled]
[-K keytab_file] [-R remote_server_principal] [-C]
[-V [security_options]] [-Z security_mechanism] [-Q] [-Y]
[-y sybase directory] [-x trusted.txt_file]
[--clienterr errfile] [--maxconn maximum_connections]
[--show-fi] [--hide-vcc]
[--colpasswd [[[db_name.[owner].]table_name.]column_name [password]]]
[--keypasswd [[db_name.[owner].]key_name [password]]]
[--initstring ASE initialization string] [--quoted-fname]
C:\Sybase\OCS-15_0\bin>
I looked up the BCP utility options on the following site and found that option to create format file or queryout is not listed there.
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.infocenter.dc30191.1550/html/utility/X14951.htm
bcp [[database_name.]owner.]table_name [: [ partition_id | slice_number ] |
partition partition_name] {in | out} datafile
[-f formatfile]
[-e errfile]
[-d discardfileprefix]
[-F firstrow]
[-L lastrow]
[-b batchsize]
[-m maxerrors]
[-n]
[-c]
[-t field_terminator]
[-r row_terminator]
[-U username]
[-P password]
[-I interfaces_file]
[-S server]
[-a display_charset]
[-z language]
[-A packet_size]
[-J client_charset]
[-T text_or_image_size]
[-E]
[-g id_start_value]
[-N]
[-W]
[-X]
[-M LabelName LabelValue]
[-labeled]
[-K keytab_file]
[-R remote_server_principal]
[-C]
[-V [security_options]]
[-Z security_mechanism]
[-Q]
[-Y]
[-y sybase directory]
[-x trusted.txt_file]
[--maxconn maximum_connections
[--show-fi]
[--hide-vcc]
[--colpasswd [[[database_name.[owner].table_name.]column_name
[password]]]
[--keypasswd [[database_name.[owner].]key_name [password]]]
However for SQL server, BCP description shows format and queryout options in the following description of the utility.
http://technet.microsoft.com/en-us/library/ms162802.aspx
_xCodexBlockxPlacexHolderx_Can anyone please tell me if the options to queryout and create format file using "format" option is not supported in BCP on Sybase ASE? Or is there a way to enable these?
I am aware that we can create format files manually, but that is not feasible as I have to create format files for 2000 tables.
I would like to know if anyone has faced similar issue to generate format files using BCP, and if there is a workaround to solve this issue.
Despite the same names and overlap in syntax, the utilities for Sybase ASE and MS SQL Server are quite different.
Sybase ASE bcp does not support queryout.
Also format files for Sybase BCP's are only used to define column datatypes, they are not for reordering/skipping or other actions that are available using SQL Server format files.
For Sybase ASE, format files aren't necessary to import or export data. They are only necessary if you do not specify -c (character mode) or -n (native mode) in your bcp command.

How to spilt a 4GB .sql file into smaller files

I have a 4GB sql data script of my database. I want to execute it but SQL Server Management Studio does allow me to do it because it's a very big file.
So I want to split this file into smaller files so that I can execute it. I have googled it but didn't get any good solutions. I have also used HJSplit to split the files but only first split file is in correct format but others are not in correct format due to which can not be execute in SQL query Interface.
Please help me guys how can I execute this .sql file with data with or without splitting?
use the sqlcmd tool to execute the file..
sqlcmd -S myServer\instanceName -i C:\myScript.sql
or
sqlcmd -S <server> -i C:\<your file here>.sql -o
Just replace with the location of your SQL box and with the name of your script. Don't forget if you're using a SQL instance the syntax is:
sqlcmd -S \instance.
Here is the list of all arguments you can pass sqlcmd:
Sqlcmd [-U login id] [-P password]
[-S server] [-H hostname] [-E trusted connection]
[-d use database name] [-l login timeout] [-t query timeout]
[-h headers] [-s colseparator] [-w screen width]
[-a packetsize] [-e echo input] [-I Enable Quoted Identifiers]
[-c cmdend] [-L[c] list servers[clean output]]
[-q "cmdline query"] [-Q "cmdline query" and exit]
[-m errorlevel] [-V severitylevel] [-W remove trailing spaces]
[-u unicode output] [-r[0|1] msgs to stderr]
[-i inputfile] [-o outputfile] [-z new password]
[-f | i:[,o:]] [-Z new password and exit]
[-k[1|2] remove[replace] control characters]
[-y variable length type display width]
[-Y fixed length type display width]
[-p[1] print statistics[colon format]]
[-R use client regional setting]
[-b On error batch abort]
[-v var = "value"...] [-A dedicated admin connection]
[-X[1] disable commands, startup script, enviroment variables [and exit]]
[-x disable variable substitution]
[-? show syntax summary]

SQL Server BCP: How to put quotes around all fields?

I have this BCP command:
'bcp DBName..vieter out c:\test003.txt -c -T /t"\",\"" -S SERVER'
The output CSV I get does not put quotes around the field names, instead it puts it around the commas! How can I get the /t"\",\"" to put quotes around all fields.
Thanks all
Setting the row terminator in addition to the field terminator should do the trick
'bcp DBName..vieter out c:\test003.txt -c -T -t"\",\"" -r"\"\n\"" -S SERVER'
This will likely work, but miss off the leading " for the first field of the first line, and perhaps the last field of the last line - I'm not sure, just guessing really, no server here!
or try using QUOTENAME to wrap text fields (you could also wrap numbers, but that isn't normally required.)
'bcp "SELECT id, age, QUOTENAME(name,'"') FROM DBName..vieter" queryout c:\test003.txt -c -T -t"," -S SERVER'
You need to use CHAR(34) for the quote. This page has more details: http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=153000
Alternatively, if you are fine for Powershell based script, you can try with below code, which does automatic quoting.
Invoke-sqlcmd -ConnectionString "Server=SERVERNAME, `
3180;Database=DATABASENAME;Trusted_Connection=True;" `
-Query "SET NOCOUNT ON;SELECT * FROM TABLENAME" -MaxCharLength 700 | `
Export-Csv -NoTypeInformation -path C:\temp\FileName.csv -Encoding UTF8
bcp "SELECT char(34) + * +char(34) FROM atable queryout "C:\temp\out.csv" -T -N -c /t"\",\""
This will put quotes before and after each field (including the first and the last).
Here are the list of commands i used .
BCP "DECLARE #colnames VARCHAR(max);SELECT #colnames = COALESCE(#colnames + ',', '') + column_name from databaseName.INFORMATION_SCHEMA.COLUMNS where TABLE_NAME='tableName'; select #colnames;" queryout "C:\HeadersOnly.csv" -r"\n\"" -c -T -Uusername -Ppassword -SserverName
bcp databaseName.schema.tableName out "C:\EmployeeDatawithoutheaders.csv" -T -t"\",\"" -r"\"\n\"" -c -Uusername -Ppassword -SserverName
copy /b C:\HeadersOnly.csv+C:\EmployeeDatawithoutheaders.csv C:\EmployeeData.csv
del C:\HeadersOnly.csv
del C:\EmployeeDatawithoutheaders.csv
I guess your goal was to clearly seperate field values by using an unique identifier so that import procedure doesn't have an issue.
I had same issue and found this workaroud useful: Using an unusual field terminator, for example | or even a string /#/ can be very unique and shouldn't mess with your string content. You also can HEX-Values (limited, see https://learn.microsoft.com/en-us/sql/tools/bcp-utility?view=sql-server-2017)
export
bcp DB.dbo.Table out /tmp/output2.csv -c -t "/#/" -U sa -P secret -S localhost
import
bcp TargetTable in /tmp/output2.csv -t "/#/" -k -U sa -P secret -S localhost -d DBNAME -c -b 50000
The actual workable answer, that removes the leading quote, is to :
A) generate format file with bcp :
bcp db.schema.tabel format nul -c -x -f file.xml -t"\",\"" -r"\"\r\n" -T -k
B) edit that file to manually copy field 1 to field 0 above, as the first field, set Max_Length=1 and remove the separator and one quot the was in field1
<FIELD ID="0" xsi:type="CharTerm" TERMINATOR="\"" MAX_LENGTH="1" COLLATION="SQL_Latin1_General_CP1_CI_AS"/>
The trick works, as you are adding a field (interface to the file) to detect the first seprator, which results in an always null-value, but not add a row (interface for the query output).

Resources