I'm experiencing the following error when using the bcp utility in SQL Server 2017.
I need to export data from a view in SQL Server to a .CSV file using a comma as the column delimiter. My data contains comma in certain columns. So I used " as a column qualifier when created a view using [+'"' column_name + '"'].
I tried 3 different ways:
Option 1:
declare #sql varchar(8000)
select #sql = 'bcp "SELECT * FROM MyDB.dbo.MyTable
WHERE Rn BETWEEN 1 AND 100 ORDER BY Rn"
queryout "E:\MyFolder\MyFileName.txt" -c -t, -T -S' + ##servername
exec master..xp_cmdshell #sql
Option 2:
declare #sql varchar(8000)
select #sql = 'bcp "SELECT * FROM MyDB.dbo.MyTable
WHERE Rn BETWEEN 1 AND 100 ORDER BY Rn"
queryout "E:\MyFolder\MyFileName.txt" -c -t, [MyServer_Name] -T -S'
exec master..xp_cmdshell #sql
Option 3
declare #sql varchar(8000)
select #sql = 'bcp "SELECT * FROM MyDB.dbo.MyTable
WHERE Rn BETWEEN 1 AND 100 ORDER BY Rn"
queryout "E:\MyFolder\MyFileName.txt" -c -t, [MyServer_Name] -T -S ' + '##servername'
exec master..xp_cmdshell #sql
In every case I'm getting the same output but the CSV file never gets created:
usage: bcp {dbtable | query} {in | out | queryout | format} datafile
[-m maxerrors] [-f formatfile] [-e errfile]
[-F firstrow] [-L lastrow] [-b batchsize]
[-n native type] [-c character type] [-w wide character type]
[-N keep non-text native] [-V file format version] [-q quoted identifier]
[-C code page specifier] [-t field terminator] [-r row terminator]
[-i inputfile] [-o outfile] [-a packetsize]
[-S server name] [-U username] [-P password]
[-T trusted connection] [-v version] [-R regional enable]
[-k keep null values] [-E keep identity values]
[-h "load hints"] [-x generate xml format file]
[-d database name] [-K application intent] [-l login timeout]
NULL
PS: I found alternative solution to import data into .CSV from SQL Server. But the problem here is that sqlcmd - unlike bcp - does not accept two characters as text qualifier. I decided to use a text qualifier of 2 chars ~^ following comma as a field separator.
exec master..xp_cmdshell 'sqlcmd -s -W -Q "set nocount on;select * from MyTable" | findstr /v /c:"-" /b > "E:\MyFile.csv"'
where -s is col_separator
Specifies the column-separator character. The default is a blank space. This option sets the sqlcmd scripting variable SQLCMDCOLSEP. To use characters that have special meaning to the operating system such as the ampersand (&), or semicolon (;), enclose the character in quotation marks ("). The column separator can be any 8-bit character.( https://learn.microsoft.com/en-us/sql/tools/sqlcmd-utility?view=sql-server-ver15 )
Command lines supplied to xp_cmdshell cannot span multiple lines of text, they need to be specified completely on a single line. Try building the command as a series of concatenated strings like this:
declare #sql varchar(8000) =
'bcp' +
' "select * from MyDB.dbo.MyTable WHERE Rn BETWEEN 1 AND 100 order by Rn"' +
' queryout' +
' "E:\MyFolder\MyFileName.txt"' +
' -c -t, -T -S' + ##servername
exec master..xp_cmdshell #sql
When concatenating the strings be mindful of where you need to include white space between each segment so that command line parameters don't run into each other.
P.S.
I found alternative solution to Import data into CSV fro SQL Server. But the problem here is that sqlcmd unlike bcp does not except to characters as a text qualifier. I decided to use a text qualifier of 2 chars ~^ following comma as a filed separator.
exec master..xp_cmdshell 'sqlcmd -s -W -Q "set nocount on;select * from MyTable" | findstr /v /c:"-" /b > "E:\MyFile.csv"'
Where -s is col_separator Specifies the column-separator character. The default is a blank space. This option sets the sqlcmd scripting variable SQLCMDCOLSEP. To use characters that have special meaning to the operating system such as the ampersand (&), or semicolon (;), enclose the character in quotation marks ("). The column separator can be any 8-bit character.( https://learn.microsoft.com/en-us/sql/tools/sqlcmd-utility?view=sql-server-ver15 )
Also we have another alternative solution that includes headers, but there you have no control on how you pass your query.
set BCP_EXPORT_SERVER=YourServerName
set BCP_EXPORT_DB=YourDBName
set BCP_EXPORT_TABLE=YourTableName
BCP "DECLARE #colnames VARCHAR(max);SELECT #colnames = COALESCE(#colnames + ',', '') + column_name from %BCP_EXPORT_DB%.INFORMATION_SCHEMA.COLUMNS where TABLE_NAME='%BCP_EXPORT_TABLE%'; select #colnames;" queryout HeadersOnly.csv -c -t, -T -S%BCP_EXPORT_SERVER%
BCP %BCP_EXPORT_DB%.dbo.%BCP_EXPORT_TABLE% out TableDataWithoutHeaders.csv -c -t"|", -T -S%BCP_EXPORT_SERVER%
set BCP_EXPORT_SERVER=
set BCP_EXPORT_DB=
set BCP_EXPORT_TABLE=
copy /b HeadersOnly.csv+TableDataWithoutHeaders.csv TableData.csv
del HeadersOnly.csv
del TableDataWithoutHeaders.csv
I am exporting data from SQL Server table to CSV using SQLCMD:
sqlcmd -S serverdetails -s"^" -d dbname -U username -P password -W -Q "SET NOCOUNT on; Select * from table with (nolock) " > c:\\USERS\\a\\b\\export_file.csv -s"^" -W """;
I am getting data like this:
id^column1^column2^column3^column4
1^abc^cde^www.google.com^8776565
2^abc^cde^www.google.com^8776565
3^abc^cde^www.google.com^8776565
I want output like this:
"id"^"column1"^"column2"^"column3"^"column4"
"1"^"abc"^"cde"^"www.google.com"^"8776565"
"2"^"abc"^"cde"^"www.google.com"^"8776565"
"3"^"abc"^"cde"^"www.google.com"^"8776565"
Please suggest how I can do this with select * from table. I don't want to specify all columns and concatenate " with them.
The double quote character has to be escaped with a backslash,
try to add "\"^\"";
sqlcmd -S serverdetails -s"\"^\"" -d dbname -U username -P password -W -Q "SET NOCOUNT on; Select * from table with (nolock) " > c:\\USERS\\a\\b\\export_file.csv -s"\"^\"" -W """
I have a table that is truncated and loaded with data everyday the problem is truncating the table is taking a while and users are noticing this. What I am wondering is, is there a way to have two of the same tables and truncate one then load the new data and then have the users user that new table and just keep switching between the two table.
If you're clearing out the old table, as well as populating new you could use the OUTPUT clause. Be mindful of the potential for log growth, consider a loop/batch approach if this may be a problem.
DELETE
OldDatabase.dbo.MyTable
OUTPUT
DELETED.col1
, DELETED.col2
, DELETED.col3
INTO
NewDatabase.dbo.MyTable
Or you can Use BCP which is a handy alternative to be aware of. Note this is using SQLCMD syntax.
:setvar SourceServer OldServer
:setvar SourceDatabase OldDatabase
:setvar DestinationServer NewServer
:setvar DestinationDatabase NewDatabase
:setvar BCPFilePath "C:\"
!!bcp "$(SourceDatabase).dbo.MyTable" FORMAT nul -S "$(SourceServer)" -T -n -q -f "$(BCPFilePath)MyTable.fmt"
!!bcp "SELECT * FROM $(SourceDatabase).dbo.MyTable WHERE col1=x AND col2=y" queryout "$(BCPFilePath)MyTable.dat" -S "$(SourceServer)" -T -q -f "$(BCPFilePath)MyTable.fmt" -> "$(BCPFilePath)MyTable.txt"
!!bcp "$(DestinationDatabase).dbo.MyTable" in $(BCPFilePath)MyTable.dat -S $(DestinationServer) -T -E -q -b 2500 -h "TABLOCK" -f $(BCPFilePath)MyTable.fmt
That I'm a beginner and I have to export from SQL SERVER Management studio, the result of a query by generating an Excel file, I use either BCP or SQLCMD like this:
enter code here
exec xp_cmdshell 'sqlcmd -S localhost -d BaseName -E -Q "SELECT * FROM TableName" -o "c:\MyTest.csv" -W -w 1024 -s"|"'
This is supposed to separate the columns in Excel
My problem is that all the results are placed in the first column and within this column actually columns are separated by the character I entered (here "|"). But I want each column to go in a different column.
I'm beginning to think that this might be possible with BCP or with SQLCMD
would anyone answer?
thank you in advance for reading
This needs to be formatted in Excel using "Text to Columns"
Refer Link: Text to Columns in Excel
exec master..xp_cmdshell 'BCP "sp_who" QUERYOUT C:\av\sp_who.txt -S MC0XENTC -T -c '
SQLCMD supports the -s parameter to specify the column separator, but I couldn't figure how how to represent the tab (CHAR(9)) character. I have tried the following but both don't work:
sqlcmd -S ServerName -E -Q"select * from mytable" -s"\t" -o results.txt
sqlcmd -S ServerName -E -Q"select * from mytable" -s'\t' -o results.txt
Any ideas how to do this in SQLCMD?
In a batch file, putting a tab between the double quotes works.
sqlcmd -S ServerName -E -Q"select * from mytable" -s" " -o results.txt
to do the same in a PowerShell file use escaped double quotes wrapped around an escaped tab
sqlcmd -S ServerName -E -Q"select * from mytable" -s `"`t`" -o results.txt
It's difficult to get unformatted results from SQLCMD.
If you want to create a tab-delimited output file, BCP might be a better bet:
bcp "select * from mytable" queryout results.txt -S server -T -c
Found a good answer here: SQLCMD outfile as tab delimited text file
Open Notepad
Paste this: sqlcmd -S (local) -E -s"<TAB>" -Q "select * from sys.dm_exec_query_stats" -o MyOutput.txt -h-1 -W
Highlight <TAB>, then hit the Tab key
Save the file as MyBatch.bat
Run MyBatch.bat
I've tried numerous times to pass the actual TAB character in to SQLCMD, and I simply can't get it to take it. My favorite work-around to-date is to pass SQLCMD the ASCII "Unit Separator", which is hex 0x1F, and can be entered on the command line by typing Ctrl-_ (control underscore, which on a US keyboard becomes ctrl-shift-'-' (the '-' next to the '0' on the top row of the keyboard).
The advantage of using the 'Unit Separator' is that is is HIGHLY unlikely to be present in text of any description, and was designed specifically for this purpose (see https://en.wikipedia.org/wiki/Delimiter)
Having got SQLCMD to do that for me, I then pipe it's output though a Unix-style translate command as:
tr '\037' '\t'
The \037 is octal for the 'Unit Separator', and \t represents the tab character, 'tr' will translate BOTH of these for us, we don't need to rely on any quoting tricks in our scripts or shells.
To get 'tr' on windows, you can install the CoreUtils package from GnuWin32 (see http://gnuwin32.sourceforge.net/packages/coreutils.htm) or go heavy-weight and install a full Unix environment such as Cygwin (http://cygwin.com/).
Putting the two together we get:
sqlcmd ... -h-1 -W -k -r1 -s^_ ... | tr '\037' '\t'
and this will give you your output with tabs.
Look up the other options I've used above, they're essential for trying to get clean output from SQLCMD (in order; no headers, trim white-space, CRLF to spaces, errors to STDERR (not your output file!) and the '^_' is how the Unit Separator will appear on the command line). You'll also need to add "SET NOCOUNT ON;" to your query or sql script, otherwise you'll get the row-count as a trialling message appearing in your output!
A similar answer to one posted above, but it's simpler in a way that I think is significant.
Open your text editor
Press Tab
Highlight the chunk of whitespace (the tab) created
Copy and paste that into the spot in your SQL command
Even though this tab is represented as a wide chunk of whitespace, it is a single character.
The other answer had some unnecessary stuff about pasting the whole command with "<TAB>" in it. I think that throws people off (it certainly threw me off).
To work in the Command Prompt window instead in batch file, this is the only way that I have found to solve it:
sqlcmd -S ServerName -E -d database_Name -Q"select col1, char(9), col2, char(9), col3, char(9), col4, char(9), col5 from mytable" -o results.txt -W -w 1024 -s "" -m 1
tldr: use ALT+009 the ascii tab code for the separator character
In the example, replace {ALTCHAR} with ALT+009 (hold the ALT key and enter the digits 009)
sqlcmd -E -d tempdb -W -s "{ALTCHAR}" -o junk.txt -Q "select 1 c1,2 c2,3 c3"
Edit junk.txt. Tabs will be between columns.
For other command line options:
sqlcmd -?
Note: The shell converts the ALT char to ^I, but if you try the command by typing -s "^I", you won't get the same results.
Use dynamic sql with CHAR(9):
SET #cmd ='SQLCMD -S MyServer -d MyDatabase -E -W -Q "SELECT * FROM MyTable" -s"' + CHAR(9) + '" -o "MyFilePath.txt"'
Try using horizontal scroll bars with cmd.exe or powershell. Right click shortcut and click properties for repeated use, or right click title bar and click properties after opening then click layout tab. In screen buffer size set width and height to 8000 and then unselect wrap text output on resize (important). Click ok. Then restore down by clicking button next to minimize. You should see horizontal and vertical scroll bars. You can maximize window now and scroll in any direction. Now you can see all records in database.
I had this problem while trying to run sqlcmd on terminal. I got it working by entering a tab character (copying from text editor didn't work for me).
Press cntrl + v then tab.
How to enter a tab char on command line?
To achieve this using sqlcmd you need to use the Tab character like so: \t
An example query exporting a single sql database table into a text file using a tab delimiter is as follows:
sqlcmd -S ServerName -d databaseTableName -Q "SELECT * FROM TABLE_NAME" -o C:\backups\tab_delimiter_bakup.txt -s"\t"
If on Linux then this will work as the -s (col_separator)
-s "$(printf "t" | tr 't' '\t')" or
SQLCMDCOLSEP="$(printf "t" | tr 't' '\t')" sqlcmd -S ...