I'm importing a set of data files in a folder into a SQL Server table. I'm able to run a recursive bcp if I enter it manually on the command line, but it doesn't work if I put it in a batch file.
The command is:
for /r %i in (*) do bcp databasename.dbo.tablename in %i -c -t -S servername -U username -P password -t "|"
The error returned is -c was unexpected at this time. If I remove the -c I get -t was unexpected at this time, and so on.
The problem was that the filenames ended up with space and dash characters when loaded to %i and these were interpreted as meaningful parts of the bcp command, which of course screwed it up. The simple but annoying solution is to make sure the directory names and filenames don't have spaces or dashes.
Related
I'm working on a batch file that will import data into the PostgreSQL database I use for testing. The batch file drops all of the databases, then recreates/reloads them from a previous dump file made from our production database. However, I sometimes run into a problem if I've accidentally left a connection open to that server/database. The "drop" portion fails because there are still users connected (me).
I've been trying to "tweak" my batch file with a command to disconnect all users from the database(s) prior to issuing the command to drop them, but I can't get that part (disconnection) to work. I've taken the disconnect code from another SO question How to drop a PostgreSQL database if there are active connections to it?, and I've been looking at other questions like How to execute postgres' sql queries from batch file? for help with the syntax.
I've also seen the "alternate" syntax for a not equal operator on the 9.2. Comparison Functions and Operators page of the official PostgreSQL documentation, but that seems to also be using "special" characters that would require escaping, so I'm not sure how to proceed.
At this point, the batch file looks like this:
#Echo OFF
SET PGPASSWORD=PASSWORD
cd /D "C:\PostgreSQL\bin"
psql.exe -h localhost -p 5432 -d postgres -U username -c 'SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = ''betadb'' AND pid \<\> pg_backend_pid();'
dropdb.exe -h localhost -p 5432 -U username betadb
psql.exe -h localhost -p 5432 -d postgres -U username < "C:\PostgresSQL\prodserverdump.sql"
Everything else works except for the pg_terminate_backend query. Every time I run that, I get strange errors indicating a problem with a path, or a file, or something else like that. I believe I've narrowed the problem down to the "not equal" operator (<>) in the query, but I can't seem to find the correct way to escape this so it doesn't try to pipe in data from a file that's not being defined.
I've tried using single backslashes (\) and double backslashes (\\), in front of one or both of the characters in the operator, but that doesn't appear to work. Is there a special way to escape the "greater than" and "less than" characters for the -c command line option in psql?
Using a combination of suggestions and "trial & error", I believe I found the correct syntax for executing this particular SQL command through a batch file.
Trying the "alternative" not equal operator (!=), I was still getting errors. They were different errors (it was giving me some nonsense about too many parameters), but it still wouldn't execute.
Using #Compo's suggestion from the comments, I then tried to enclose the entire SELECT statement in double quotes instead of single quotes. Still not quite there.
Finally, I removed the "extra" single quotes I was using around the database names from before. The query appears to have executed properly.
The final result looks like this:
#Echo OFF
SET PGPASSWORD=PASSWORD
cd /D "C:\PostgreSQL\bin"
psql.exe -h localhost -p 5432 -d postgres -U username -c "SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'betadb' AND pid != pg_backend_pid();"
dropdb.exe -h localhost -p 5432 -U username betadb
psql.exe -h localhost -p 5432 -d postgres -U username < "C:\PostgresSQL\prodserverdump.sql"
I suppose I had assumed that, because all of the examples I had found were using single quotes to surround the SQL statement, that's what I had to use. Apparently, that assumption was incorrect.
Regardless, it all seems to be working correctly now. Hope this helps someone else who's looking to accomplish something similar.
I have a folder with 55 .sql files, SQL scripts.
Each file consists of 10000 insert statements. I'm using this command to run each file:
C:\sqlcmd -S . -d Minuf_Customers_July -i C:\ACL\MyScript3.sql
And I need to do it for MyScript1 , MyScript2.. to MyScript54.
Is there a way to loop through all the commands in the command prompt?
But it is important! that the each row will run separately because otherwise it will throw an "out of memory" error.
The DOS command for is your friend.
for %f in (*.sql) do sqlcmd -S . -d Minuf_Customers_July -i %f
The command help for gives details of various options.
I'am trying to import data into sql server table from a file using a format file.
In fact I have 2 databases: a production database and a local database
I want to insert some row of the table shipper of the production database in the local one. The table shipper don't have neither the same columns nor the same order of column in the 2 databases.
That's why I used a file format to do my bcp.
I generate file containing the rows I want to insert in my local database with the following commande
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
It works !!
I generate then the format file with the schema of my local table with the following commande
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
It works !!
Unfortunately when I tried to insert the file data in my local table
with the following commande:
bcp LocalDatabase.dbo.shipper in shipper.txt -T -f shipper-n.fmt
it generates the following error (translated from french)
Can anyone know what is the problem and how can I get arround it.
Thanks in advance
unexpected end of file encountered in the bcp data file
Your format file does not match the data. You are exporting using text using -c
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T
But your format file is made for native (binary) data using -n
bcp LocalDatabase.dbo.shipper nul -T -n -f shipper-n.fmt
Either export both as native (my recommendation), or both as text. To prevent this error, export the data file and the format file at the same time, simply add -f shipper.fmt to your export
Text version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -c -T -f shipper.fmt
or
Native Version:
bcp "SELECT shipper_id,Shipper_name FROM ProductionDatabase.dbo.shipper where shipper_id >5" queryout shipper.txt -n -T -f shipper.fmt
PS. Since you can run into scenarios where your record or row delimiters exist in the data you should pick a character sequence that does not exist in your data as a separator for instance -t"\t|\t" (Tab-Pipe-Tab) for fields and -r"\t|\n" (Tab-Pipe-Newline) for rows. If you combine the format statement with the export the data and the format file will match and you have the freedom to change the separators on a single command line.
Specify separators after the -n or -c on the command line
i have looked all over the internet and cant seem to find a solution to this problem.
i am trying to output query results as a CSV through using a combination of sqlcmd and windows batch. here is what i have so far:
sqlcmd.exe -S %DBSERVER% -U %DBUSER% -P %DBPASS% -d %USERPREFIX% -Q "SELECT Username, UserDOB, UserGender FROM TABLE" -o %USERDATA%\%USERPREFIX%\FACT_BP.CSV -h-1 -s","
is there something i'm missing here? some setting that only looks at the first column of the query results?
any advice at all would be a huge help - i'm lost.
Here is the reference page from MSDN on SQLCMD.
http://technet.microsoft.com/en-us/library/ms162773.aspx
I placed this command in a batch file in C:\temp as go.bat.
sqlcmd -S(local) -E -dmaster
-Q"select cast(name as varchar(16)), str(database_id,1,0), create_date from sys.databases"
-oc:\temp\sys.databases.csv -h-1 -s,
Notice I hard coded the file name and removed the "" around the field delimiter.
I get the expected output below.
Either the command does not like the system variables or something else is wrong. Please try my code as a base line test. It works for SQL 2012.
Also, the number of lines is always dumped to file. You must clear this out of the file. That is why I do not use SQLCMD for ETL.
Why not use BCP instead?
I have writing several articles on my website.
http://craftydba.com/?p=1584
If I call sqlcmd with the -i command line switch, I'd like to be able to get the name of the file. So, I call
sqlcmd -S <servername> -E -i filename.sql
I'd like to be able to somehow have the contents of the script be able to print the filename without having to hard code it in the file. Looking at the variables and commands that are documented in BOL, I don't see anything like this, but just wanted to make sure. Thanks in advance.
Among the list of sqlcmd Scripting Variables, I don't see anything that has the name for the input file.
But you can send the file name as a parameter when you call sqlcmd.
Input file (filename.sql)
PRINT '$(p1)'
Sqlcmd:
sqlcmd -S .\Server -i filename.sql -v p1="filename.sql"
May be you should explore powershell for this