I'am trying to import data into sql server table from a file using a format file.
In fact I have 2 databases: a production database and a local database
I want to insert some row of the table shipper of the production database in the local one. The table shipper don't have neither the same columns nor the same order of column in the 2 databases.
That's why I used a file format to do my bcp.
I generate file containing the rows I want to insert in my local database with the following commande
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
It works !!
I generate then the format file with the schema of my local table with the following commande
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
It works !!
Unfortunately when I tried to insert the file data in my local table
with the following commande:
bcp LocalDatabase.dbo.shipper in shipper.txt -T -f shipper-n.fmt
it generates the following error (translated from french)
Can anyone know what is the problem and how can I get arround it.
Thanks in advance
unexpected end of file encountered in the bcp data file
Your format file does not match the data. You are exporting using text using -c
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
But your format file is made for native (binary) data using -n
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
Either export both as native (my recommendation), or both as text. To prevent this error, export the data file and the format file at the same time, simply add -f shipper.fmt to your export
Text version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T -f shipper.fmt
or
Native Version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -n -T -f shipper.fmt
PS. Since you can run into scenarios where your record or row delimiters exist in the data you should pick a character sequence that does not exist in your data as a separator for instance -t"\t|\t" (Tab-Pipe-Tab) for fields and -r"\t|\n" (Tab-Pipe-Newline) for rows. If you combine the format statement with the export the data and the format file will match and you have the freedom to change the separators on a single command line.
Specify separators after the -n or -c on the command line
Related
I'm trying to copy csv to database table using bcp command. It's successfully copied, but the problem is non-English letters was migrated as some junk characters.
BCP query:
bcp "[table_name]" in "[file_path]" -F 1 -a 32768 -c -C RAW -t"|~|" -r "|\n|" -S "[server]" -U "[username]" -P "[password]" -d [database]
after migration
This is the source
Another Scenario:
When I use -C 65001 instead of -C RAW above issue is fixed. But I've fixed another bug that is opened again. Please help me on the same.
Source: enter image description here
AfterMigration: enter image description here
I've found the solution to my problem.
Previously I've used text and nvarchar data types in my destination tables.
using -C 65001 the non English letters migrated properly for nvarchar columns not for text datatype.
using -C RAW the non English letters migrated properly for text columns not for nvarchar datatype.
I want to export varbinary with bcp to a flat file (csv).
It seems to work, but in front of the 0x is a strange enconding, I can not get rid of. Can someone explain what I'm doing wrong? Following the bcp-command I execute and the screenshot from notepad++
PS C:\Windows\system32> bcp "SELECT TOP 5 CONVERT(varchar(max), [Bild],1) AS SomeImageFieldAsHex FROM [SWTA_HH].[dbo].[Fenster]" queryout "c:\export.csv" -d "SWTA_HH" -T -n -r \n
Thanks Darius
As per the bcp Utility documentation, the -n switch is instructing BCP to output data in "native" format, which includes binary information to allow a receiving SQL Server to ingest the data again using the correct data types.
If you're wanting to output these varbinary values to a .CSV (comma-separated values) file then you'll probably want to use the -c switch instead, for character data, e.g.:
bcp "SELECT TOP 5 CONVERT(varchar(max), [Bild], 1) AS SomeImageFieldAsHex FROM [SWTA_HH].[dbo].[Fenster]" queryout "c:\export.csv" -d "SWTA_HH" -T -c -r \n
I have a CSV file I'm creating by exporting a table in SQL Server 2016 SP2 using the bulk copy utility (bcp.exe). I'm setting the code page to 65001 (which Microsoft's documentation states is UTF-8). However, when I stage the file in Snowflake and then try to use the COPY command to move it into a table, I get an error that says, "Invalid UTF8 detected in string '0xFF0xFE00x0000x0010x0010x0060x0000x0000x0000x0000x0010x00n0x0040x00M0x00M0x00c0x00A0x00A0x00M0x00'."
If I use the IGNORE_UTF8_ERRORS flag, I get data in my table that is unintelligible. Any suggestions about how to fix the issue would be gratefully received.
Here's my BCP call:
BCP "SELECT Id, Name FROM database_name.owner_name.table_name WHERE Id = '0011602001r4ddgAZA'" queryout C:\temp\test.csv "-t|" -w -T -S. -C 65001
Here's the code in Snowflake:
--Create a file format
create or replace file format SFCI_Account
type = 'CSV'
field_delimiter = '|'
validate_utf8 = True
;
-- Create a Stage object
create or replace stage SFCI_Account_stage
file_format = SFCI_Account;
-- Check my file is there
list #SFCI_Account_stage;
-- Copy the file into the table
copy into Test
from #SFCI_Account_stage
file_format = (format_name = SFCI_Account)
pattern='.*.csv.gz'
on_error = 'skip_file';
Apparently, all I needed to do was change the -w to -c in my BCP call and add the following:
-r "\r\n"
So, my final BCP call looks like this:
BCP "SELECT Id, Name FROM database_name.owner_name.table_name WHERE Id = '0011602001r4ddgAZA'" queryout C:\temp\test.csv "-t|" -c -T -S. -C 65001 -r "\r\n"
Now, that fixed the issue of the the UTF-8 error, but now I have to figure out how to deal with carriage returns in the data.
I have source file in which 90% of first field is empty. I want to load this file to SQL server table with BCP utility. When i run BCP command, BCP utility is not able to recognize or distinguish records.
My Source file has data as below.
|100168|27238800000|14750505|1|273
|100168|27238800000|14750505|1|273
|100681|88392930052|37080101|1|252
|101014|6810000088|90421505|12|799
|101595|22023000000|21050510|8|780
I am using
**bcp [DBNAME].[dbo].[TABLE1] in \\filelocation\filename -e \\filelocation\filename_Error.txt -c -t | -S ServerName -T -h TABLOCK -m 1**
I am getting error message in error.txt
as ## Row 1, Column 28: String data, right truncation ## 100168 27238800000 14750505 1 273
100168|27238800000|14750505|1|273. Here BCP is not able to recognize
records. Due to this BCP is trying loaded next record data into last
field which is causing data truncation.
Table schema is
CREATE TABLE [DBO].[TABLE1](
FLD1 VARCHAR(10)
,FLD2 VARCHAR(10)
,FLD3 VARCHAR(22)
,FLD4 VARCHAR(15)
,FLD5 VARCHAR(10)
,FLD6 VARCHAR(12) )
You need to quote the pipe. Pipe (character |) is used for redirecting standard output for command lines
The following simplified line works with your sample
bcp.exe [db].dbo.[table1] in "path\Data.dat" -S".\instance" -T -c -t"|"
I omitted the error limit -m, log -e and table lock hint -h, but those should not affect the import, but if you still have an issue try quoting parameters like the server name and filenames
I used a text file with standard \r\n row terminators as expected by -c
That I'm a beginner and I have to export from SQL SERVER Management studio, the result of a query by generating an Excel file, I use either BCP or SQLCMD like this:
enter code here
exec xp_cmdshell 'sqlcmd -S localhost -d BaseName -E -Q "SELECT * FROM TableName" -o "c:\MyTest.csv" -W -w 1024 -s"|"'
This is supposed to separate the columns in Excel
My problem is that all the results are placed in the first column and within this column actually columns are separated by the character I entered (here "|"). But I want each column to go in a different column.
I'm beginning to think that this might be possible with BCP or with SQLCMD
would anyone answer?
thank you in advance for reading
This needs to be formatted in Excel using "Text to Columns"
Refer Link: Text to Columns in Excel
exec master..xp_cmdshell 'BCP "sp_who" QUERYOUT C:\av\sp_who.txt -S MC0XENTC -T -c '