I am trying to execute BCP within SSIS to export the results of a query into a CSV file for several different tables. However, for some reason I keep getting parsing errors when I try to put the following into the arguments for the bcp exec:
"SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T
When I execute BCP with these arguments (changing the variable to an existing table name), everything works fine. I tried to remove several parts of the argument and I still get the following error
Attempt to parse the expression ""SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T" failed. The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a required element such as a parenthesis.
What is the issue with my argument?
Your expression is incorrect.
"SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T
The SSIS expression language isn't like PowerShell where it will see the variable inside the string. Instead, we are back in the stone ages with string concatenation. You also get to deal with escaping slashes and double quotes so your final expression would look something like
"""SELECT * from myDB.dbo."
+ #[User::Table]
+ """ queryout C:\\users\\MSSQLSERVER\\Downloads\\"
+ #[User::Table]
+ ".csv -c -t, -T"
I find I have the most success when I build complex strings in Variables and then only ever reference the built Variable in an Expression on a Task versus trying to build it in there. That way I can put a breakpoint in a package or raise an Information event back with my Variable's value. Makes debugging a fiddly process a much less painful experience.
bcp reference
Related
Trying to export some data running PowerShell script as agent job. all works but need data surrounded by double quotes. here is the end of script, I get my headers hidden and comma delimited by cant get the data surrounded by double quotes (if anyone has a list of parameters for that would be great
-sqlcmd ORDER BY s.name " -s "," -o $instfile -h -1
I'm getting the following error message with execution T-SQL script with this construction "$(", in sqlcmd utility:
C:\Users\Admin>sqlcmd -S DESKTOP-5AA9JIA\SQLEXPRESS -q "select '$(' " -R
Sqlcmd: Error: Syntax error at line 1 near command '''.
1>
if run this script via SSMS then everything works smoothly.
Getting the same error, when using INSERT INTO statement or any other statement.
Any suggestion on how to resolve it?
You should use -x when launch sqlcmd to ignore scripting variables.
-x
Causes sqlcmd to ignore scripting variables. This is useful when a
script contains many INSERT statements that may contain strings that
have the same format as regular variables, such as $(variable_name).
sqlcmd Utility
When sqlcmd parser sees $(it expects scripting variable that is not provided so it throws the error.
Here is my test:
I'am trying to import data into sql server table from a file using a format file.
In fact I have 2 databases: a production database and a local database
I want to insert some row of the table shipper of the production database in the local one. The table shipper don't have neither the same columns nor the same order of column in the 2 databases.
That's why I used a file format to do my bcp.
I generate file containing the rows I want to insert in my local database with the following commande
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
It works !!
I generate then the format file with the schema of my local table with the following commande
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
It works !!
Unfortunately when I tried to insert the file data in my local table
with the following commande:
bcp LocalDatabase.dbo.shipper in shipper.txt -T -f shipper-n.fmt
it generates the following error (translated from french)
Can anyone know what is the problem and how can I get arround it.
Thanks in advance
unexpected end of file encountered in the bcp data file
Your format file does not match the data. You are exporting using text using -c
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
But your format file is made for native (binary) data using -n
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
Either export both as native (my recommendation), or both as text. To prevent this error, export the data file and the format file at the same time, simply add -f shipper.fmt to your export
Text version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T -f shipper.fmt
or
Native Version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -n -T -f shipper.fmt
PS. Since you can run into scenarios where your record or row delimiters exist in the data you should pick a character sequence that does not exist in your data as a separator for instance -t"\t|\t" (Tab-Pipe-Tab) for fields and -r"\t|\n" (Tab-Pipe-Newline) for rows. If you combine the format statement with the export the data and the format file will match and you have the freedom to change the separators on a single command line.
Specify separators after the -n or -c on the command line
When I run
bcp "select * from table where a='xyz'" format null -c -t, -f x.fmt -Sserver -T
it returns error "A valid table name is required for in,out or format options". Does it now accept a query with the format option? I tried the "out" option, it works with a query, but when import into a table, it complains on "Invalid character value for cast specification". I seem to need the "format" bcp file, and I really don't want to dump the entire table but only a selection of it. What's the alternative?
After testing the "format nul" option of bcp, I concluded that the option only supports a dump of table with no queries. However, I did solve my problems of "Invalid character value for cast specification" by specify "-r" with a specified terminating character (instead of the default \n). That solved the problem of import
bcp <table> in outfile.bcp -c -t, -r? -S<server> -T
this works fine now.
I have a stored procedure that uses bcp queryout and populates the file with the results of
SELECT * FROM ##TempTable
Occasionally, the file created is empty. I know it cannot be permission based on the output location as the file is created and saved, so the SELECT must be returning zero rows. This is a production environment and I am not permitted to put any debugging etc to see what the count of the SELECT statement returns prior to the bcp line - but I know the table is populated as it is referenced later on in the sp and that section of the code never fails
Has anyone seen bcp act this way?
The switches I am using against bcp are
-t -T -c -S
Thanks
Have you check to ensure that none of your fields contain a NULL. Use ISNULL() to replace NULL with ''.