I want to export varbinary with bcp to a flat file (csv).
It seems to work, but in front of the 0x is a strange enconding, I can not get rid of. Can someone explain what I'm doing wrong? Following the bcp-command I execute and the screenshot from notepad++
PS C:\Windows\system32> bcp "SELECT TOP 5 CONVERT(varchar(max), [Bild],1) AS SomeImageFieldAsHex FROM [SWTA_HH].[dbo].[Fenster]" queryout "c:\export.csv" -d "SWTA_HH" -T -n -r \n
Thanks Darius
As per the bcp Utility documentation, the -n switch is instructing BCP to output data in "native" format, which includes binary information to allow a receiving SQL Server to ingest the data again using the correct data types.
If you're wanting to output these varbinary values to a .CSV (comma-separated values) file then you'll probably want to use the -c switch instead, for character data, e.g.:
bcp "SELECT TOP 5 CONVERT(varchar(max), [Bild], 1) AS SomeImageFieldAsHex FROM [SWTA_HH].[dbo].[Fenster]" queryout "c:\export.csv" -d "SWTA_HH" -T -c -r \n
Related
I have a CSV file I'm creating by exporting a table in SQL Server 2016 SP2 using the bulk copy utility (bcp.exe). I'm setting the code page to 65001 (which Microsoft's documentation states is UTF-8). However, when I stage the file in Snowflake and then try to use the COPY command to move it into a table, I get an error that says, "Invalid UTF8 detected in string '0xFF0xFE00x0000x0010x0010x0060x0000x0000x0000x0000x0010x00n0x0040x00M0x00M0x00c0x00A0x00A0x00M0x00'."
If I use the IGNORE_UTF8_ERRORS flag, I get data in my table that is unintelligible. Any suggestions about how to fix the issue would be gratefully received.
Here's my BCP call:
BCP "SELECT Id, Name FROM database_name.owner_name.table_name WHERE Id = '0011602001r4ddgAZA'" queryout C:\temp\test.csv "-t|" -w -T -S. -C 65001
Here's the code in Snowflake:
--Create a file format
create or replace file format SFCI_Account
type = 'CSV'
field_delimiter = '|'
validate_utf8 = True
;
-- Create a Stage object
create or replace stage SFCI_Account_stage
file_format = SFCI_Account;
-- Check my file is there
list #SFCI_Account_stage;
-- Copy the file into the table
copy into Test
from #SFCI_Account_stage
file_format = (format_name = SFCI_Account)
pattern='.*.csv.gz'
on_error = 'skip_file';
Apparently, all I needed to do was change the -w to -c in my BCP call and add the following:
-r "\r\n"
So, my final BCP call looks like this:
BCP "SELECT Id, Name FROM database_name.owner_name.table_name WHERE Id = '0011602001r4ddgAZA'" queryout C:\temp\test.csv "-t|" -c -T -S. -C 65001 -r "\r\n"
Now, that fixed the issue of the the UTF-8 error, but now I have to figure out how to deal with carriage returns in the data.
I'm trying to transfer table data from one SQL Server to another and wanting to use the bcp utility for it. This is purely to transfer data between two identical schemas, but I'm not able to use something like SSDT; I need something that can be scriptable and portable so it can be run by others with just SQL server and SSMS access.
I am generating a native output file and format file like so:
$> bcp database.TableName OUT c:\data\bcp\TableName.bcp -T -N -S SQLINSTANCE
$> bcp database.TableName format nul -f c:\data\bcp\TableName.fmt -T -N
Then in Management Studio I am trying to in turn read the files like this:
SELECT
*
FROM
OPENROWSET (BULK 'c:\data\bcp\TableName.bcp',
FORMATFILE = 'c:\data\bcp\TableName.fmt') AS t1
But am getting this error:
The bulk load failed. The column is too long in the data file for row 6, column 19. Verify that the field terminator and row terminator are specified correctly.
I have followed this process before successfully, and it works for other tables. But I'm running into issue with this table. The column mentioned is of datatype nvarchar(max). I can inspect what I think is the "problem" record in the source data and it's just a very long string but I don't see anything else special about it.
Is there something else I should be doing when generating the format file or what else am I missing?
If you are only exporting for the purpose of importing to another SQL Server, native format is the way to go. And is this case you don't need to use format files. Just do a native export and import.
Note you are specifying a capital -N and that's not native. Native is lower -n.
You should export using something like:
bcp database.Schema.TableName OUT c:\data\bcp\TableName.bcp -T -n -S SQLINSTANCE
Then on the importing side I sugest using BULK IMPORT, which don't need a format file for native at all:
BULK INSERT TargetDB.dbo.TargetTable
FROM 'c:\data\bcp\TableName.bcp'
WITH (DATAFILETYPE = 'native');
If you can't use BULK INSERT and must absolutely go for OPENROWSET, you need a format file. bcp can generate that for you, but again, lower case -n:
bcp database.Schema.TableName format nul -f c:\data\bcp\TableName.fmt -T -n -S SQLINSTANCE
Now your OPENROWSET should work.
I'am trying to import data into sql server table from a file using a format file.
In fact I have 2 databases: a production database and a local database
I want to insert some row of the table shipper of the production database in the local one. The table shipper don't have neither the same columns nor the same order of column in the 2 databases.
That's why I used a file format to do my bcp.
I generate file containing the rows I want to insert in my local database with the following commande
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
It works !!
I generate then the format file with the schema of my local table with the following commande
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
It works !!
Unfortunately when I tried to insert the file data in my local table
with the following commande:
bcp LocalDatabase.dbo.shipper in shipper.txt -T -f shipper-n.fmt
it generates the following error (translated from french)
Can anyone know what is the problem and how can I get arround it.
Thanks in advance
unexpected end of file encountered in the bcp data file
Your format file does not match the data. You are exporting using text using -c
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
But your format file is made for native (binary) data using -n
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
Either export both as native (my recommendation), or both as text. To prevent this error, export the data file and the format file at the same time, simply add -f shipper.fmt to your export
Text version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T -f shipper.fmt
or
Native Version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -n -T -f shipper.fmt
PS. Since you can run into scenarios where your record or row delimiters exist in the data you should pick a character sequence that does not exist in your data as a separator for instance -t"\t|\t" (Tab-Pipe-Tab) for fields and -r"\t|\n" (Tab-Pipe-Newline) for rows. If you combine the format statement with the export the data and the format file will match and you have the freedom to change the separators on a single command line.
Specify separators after the -n or -c on the command line
That I'm a beginner and I have to export from SQL SERVER Management studio, the result of a query by generating an Excel file, I use either BCP or SQLCMD like this:
enter code here
exec xp_cmdshell 'sqlcmd -S localhost -d BaseName -E -Q "SELECT * FROM TableName" -o "c:\MyTest.csv" -W -w 1024 -s"|"'
This is supposed to separate the columns in Excel
My problem is that all the results are placed in the first column and within this column actually columns are separated by the character I entered (here "|"). But I want each column to go in a different column.
I'm beginning to think that this might be possible with BCP or with SQLCMD
would anyone answer?
thank you in advance for reading
This needs to be formatted in Excel using "Text to Columns"
Refer Link: Text to Columns in Excel
exec master..xp_cmdshell 'BCP "sp_who" QUERYOUT C:\av\sp_who.txt -S MC0XENTC -T -c '
I'm trying to IMPORT data from EXPORTED Dat file from SQL Server in this way:
bcp "SELECT FieldName FROM [BaseName].[dbo].[TableName] where xxxxxx=16"
queryout Message_out.dat -n -Uusername -Sservername
When I try to import dat to sql server like this
bcp basename.dbo.tablename in "path\to\datfile.dat" -c -T
I get error:
Error = [Microsoft][SQL Server Native Client 10.0]Unexpected EOF
encountered in BCP data-file
regards, Grigor.
Try to explicitly indicate the field and row terminators for your file, for example, if your file is comma delimited and each row is in a new line:
bcp basename.dbo.tablename in "path\to\datfile.dat" -c -T -r\n -t,
And if if there are any other peculiarities in the format of your file, use the options to help bcp understand your file format using the options. A detailed documentation is available at msdn.
I see this question is old, but maybe it will help someone in the future...