When I run
bcp "select * from table where a='xyz'" format null -c -t, -f x.fmt -Sserver -T
it returns error "A valid table name is required for in,out or format options". Does it now accept a query with the format option? I tried the "out" option, it works with a query, but when import into a table, it complains on "Invalid character value for cast specification". I seem to need the "format" bcp file, and I really don't want to dump the entire table but only a selection of it. What's the alternative?
After testing the "format nul" option of bcp, I concluded that the option only supports a dump of table with no queries. However, I did solve my problems of "Invalid character value for cast specification" by specify "-r" with a specified terminating character (instead of the default \n). That solved the problem of import
bcp <table> in outfile.bcp -c -t, -r? -S<server> -T
this works fine now.
Related
I'm trying to transfer table data from one SQL Server to another and wanting to use the bcp utility for it. This is purely to transfer data between two identical schemas, but I'm not able to use something like SSDT; I need something that can be scriptable and portable so it can be run by others with just SQL server and SSMS access.
I am generating a native output file and format file like so:
$> bcp database.TableName OUT c:\data\bcp\TableName.bcp -T -N -S SQLINSTANCE
$> bcp database.TableName format nul -f c:\data\bcp\TableName.fmt -T -N
Then in Management Studio I am trying to in turn read the files like this:
SELECT
*
FROM
OPENROWSET (BULK 'c:\data\bcp\TableName.bcp',
FORMATFILE = 'c:\data\bcp\TableName.fmt') AS t1
But am getting this error:
The bulk load failed. The column is too long in the data file for row 6, column 19. Verify that the field terminator and row terminator are specified correctly.
I have followed this process before successfully, and it works for other tables. But I'm running into issue with this table. The column mentioned is of datatype nvarchar(max). I can inspect what I think is the "problem" record in the source data and it's just a very long string but I don't see anything else special about it.
Is there something else I should be doing when generating the format file or what else am I missing?
If you are only exporting for the purpose of importing to another SQL Server, native format is the way to go. And is this case you don't need to use format files. Just do a native export and import.
Note you are specifying a capital -N and that's not native. Native is lower -n.
You should export using something like:
bcp database.Schema.TableName OUT c:\data\bcp\TableName.bcp -T -n -S SQLINSTANCE
Then on the importing side I sugest using BULK IMPORT, which don't need a format file for native at all:
BULK INSERT TargetDB.dbo.TargetTable
FROM 'c:\data\bcp\TableName.bcp'
WITH (DATAFILETYPE = 'native');
If you can't use BULK INSERT and must absolutely go for OPENROWSET, you need a format file. bcp can generate that for you, but again, lower case -n:
bcp database.Schema.TableName format nul -f c:\data\bcp\TableName.fmt -T -n -S SQLINSTANCE
Now your OPENROWSET should work.
I'am trying to import data into sql server table from a file using a format file.
In fact I have 2 databases: a production database and a local database
I want to insert some row of the table shipper of the production database in the local one. The table shipper don't have neither the same columns nor the same order of column in the 2 databases.
That's why I used a file format to do my bcp.
I generate file containing the rows I want to insert in my local database with the following commande
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
It works !!
I generate then the format file with the schema of my local table with the following commande
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
It works !!
Unfortunately when I tried to insert the file data in my local table
with the following commande:
bcp LocalDatabase.dbo.shipper in shipper.txt -T -f shipper-n.fmt
it generates the following error (translated from french)
Can anyone know what is the problem and how can I get arround it.
Thanks in advance
unexpected end of file encountered in the bcp data file
Your format file does not match the data. You are exporting using text using -c
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
But your format file is made for native (binary) data using -n
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
Either export both as native (my recommendation), or both as text. To prevent this error, export the data file and the format file at the same time, simply add -f shipper.fmt to your export
Text version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T -f shipper.fmt
or
Native Version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -n -T -f shipper.fmt
PS. Since you can run into scenarios where your record or row delimiters exist in the data you should pick a character sequence that does not exist in your data as a separator for instance -t"\t|\t" (Tab-Pipe-Tab) for fields and -r"\t|\n" (Tab-Pipe-Newline) for rows. If you combine the format statement with the export the data and the format file will match and you have the freedom to change the separators on a single command line.
Specify separators after the -n or -c on the command line
I've got a problem while copying data from SQL Server 2012 to Azure DB.
Here I'm listing the steps I have made.
Created .dat and .xml format files as follows:
bcp.exe dbo.user_sayti out "c:\\dbo.user_sayti.dat" -w -k -Slocalhost -dsource_db -Uuser -Ppwd
bcp.exe dbo.user_sayti format nul -f "c:\\dbo.user_sayti.xml" -w -x -Slocalhost -dsource_db -Uuser -Ppwd
dbo.user_sayti.dat
dbo.user_sayti.xml
Made an attempt to copy them to Azure DB with keeping identity values:
bcp.exe dbo.user_sayti in "c:\\dbo.user_sayti.dat" -E -f "c:\\dbo.user_sayti.xml" -Sserver.database.windows.net -dtarget_db -Uuser -Ppwd
And have got "Invalid character value for cast specification" error.
I don't understand why, because the .dat file contains value 127 for the identity column id (PK, int, not NULL) and the number of values matches the number of rows.
Then I've tried the same command without -E and the process finished successfully (there is the only row in this particular table and it appeared in the target_db with identity column value = 1).
Replacing -f parameter (schema definition) with -w solved that stupid problem :)
I am trying to execute BCP within SSIS to export the results of a query into a CSV file for several different tables. However, for some reason I keep getting parsing errors when I try to put the following into the arguments for the bcp exec:
"SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T
When I execute BCP with these arguments (changing the variable to an existing table name), everything works fine. I tried to remove several parts of the argument and I still get the following error
Attempt to parse the expression ""SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T" failed. The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a required element such as a parenthesis.
What is the issue with my argument?
Your expression is incorrect.
"SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T
The SSIS expression language isn't like PowerShell where it will see the variable inside the string. Instead, we are back in the stone ages with string concatenation. You also get to deal with escaping slashes and double quotes so your final expression would look something like
"""SELECT * from myDB.dbo."
+ #[User::Table]
+ """ queryout C:\\users\\MSSQLSERVER\\Downloads\\"
+ #[User::Table]
+ ".csv -c -t, -T"
I find I have the most success when I build complex strings in Variables and then only ever reference the built Variable in an Expression on a Task versus trying to build it in there. That way I can put a breakpoint in a package or raise an Information event back with my Variable's value. Makes debugging a fiddly process a much less painful experience.
bcp reference
What i want to do is copy a table into a file, truncate the table and copy the data back into the table.
For this, i am using the following two commands:
Out: bcp TABLE out file.csv -S SERVER -U user -P password -r '\n' -t '^|' -c
In: bcp TABLE in file.csv -S SERVER -U user-P password-r '\n' -t '^|' -c -J iso_1 -b 5000
This is the error i get:
CSLIB Message: - L0/O0/S0/N36/1/0:
cs_convert: cslib user api layer: common library error: The result is truncated because the conversion/operation resulted in overflow.
The interesting part ( for me, at least ) is that i get the error only for rows with the first column being an ODD number. From the first 3 million rows, it cuts half of them, all having the first column ( the PK ) an odd number.
I tried with different options, but none seem to work: no problem with the charset as far as i can tell, there are no huge columns such that they are truncated and it is not the carriage return missing.
Any help would be greatly appreciated.
UPDATE: After creating a format-file there are no more errors, but it only copies half of the data back into the table.
UPDATE: I managed to create a format file which works and loads all data, but i cannot use it on another server (it works in testing environment, it needs to run in production environment), since it says Attempt to read an unknown version of bcp format-file.? I know what this means, but is there any way of finding the correct values of the version?
SOLVED: After digging back in the database, it seems that the problem was indeed data inconsistency due to the fact that the VIEW used in production to copy the table only copied 25 columns, but the table has 26 columns ( somebody altered the table and i didn't know and hadn't noticed that it happened ). Fixed the View and now it works.
Since you are going out of/into the same server, I recommend you use bcp with the native flag.
bcp DBNAME..TABLE out file.bcp -SSERVER -Uuser -Ppassword -n
bcp DBNAME..TABLE in file.bcp -SSERVER -Uuser -Ppassword -n -b5000
Character mode can get wierd, and I only use it when it is required.