Trying to export some data running PowerShell script as agent job. all works but need data surrounded by double quotes. here is the end of script, I get my headers hidden and comma delimited by cant get the data surrounded by double quotes (if anyone has a list of parameters for that would be great
-sqlcmd ORDER BY s.name " -s "," -o $instfile -h -1
Related
I'm working on a batch file that will import data into the PostgreSQL database I use for testing. The batch file drops all of the databases, then recreates/reloads them from a previous dump file made from our production database. However, I sometimes run into a problem if I've accidentally left a connection open to that server/database. The "drop" portion fails because there are still users connected (me).
I've been trying to "tweak" my batch file with a command to disconnect all users from the database(s) prior to issuing the command to drop them, but I can't get that part (disconnection) to work. I've taken the disconnect code from another SO question How to drop a PostgreSQL database if there are active connections to it?, and I've been looking at other questions like How to execute postgres' sql queries from batch file? for help with the syntax.
I've also seen the "alternate" syntax for a not equal operator on the 9.2. Comparison Functions and Operators page of the official PostgreSQL documentation, but that seems to also be using "special" characters that would require escaping, so I'm not sure how to proceed.
At this point, the batch file looks like this:
#Echo OFF
SET PGPASSWORD=PASSWORD
cd /D "C:\PostgreSQL\bin"
psql.exe -h localhost -p 5432 -d postgres -U username -c 'SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = ''betadb'' AND pid \<\> pg_backend_pid();'
dropdb.exe -h localhost -p 5432 -U username betadb
psql.exe -h localhost -p 5432 -d postgres -U username < "C:\PostgresSQL\prodserverdump.sql"
Everything else works except for the pg_terminate_backend query. Every time I run that, I get strange errors indicating a problem with a path, or a file, or something else like that. I believe I've narrowed the problem down to the "not equal" operator (<>) in the query, but I can't seem to find the correct way to escape this so it doesn't try to pipe in data from a file that's not being defined.
I've tried using single backslashes (\) and double backslashes (\\), in front of one or both of the characters in the operator, but that doesn't appear to work. Is there a special way to escape the "greater than" and "less than" characters for the -c command line option in psql?
Using a combination of suggestions and "trial & error", I believe I found the correct syntax for executing this particular SQL command through a batch file.
Trying the "alternative" not equal operator (!=), I was still getting errors. They were different errors (it was giving me some nonsense about too many parameters), but it still wouldn't execute.
Using #Compo's suggestion from the comments, I then tried to enclose the entire SELECT statement in double quotes instead of single quotes. Still not quite there.
Finally, I removed the "extra" single quotes I was using around the database names from before. The query appears to have executed properly.
The final result looks like this:
#Echo OFF
SET PGPASSWORD=PASSWORD
cd /D "C:\PostgreSQL\bin"
psql.exe -h localhost -p 5432 -d postgres -U username -c "SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'betadb' AND pid != pg_backend_pid();"
dropdb.exe -h localhost -p 5432 -U username betadb
psql.exe -h localhost -p 5432 -d postgres -U username < "C:\PostgresSQL\prodserverdump.sql"
I suppose I had assumed that, because all of the examples I had found were using single quotes to surround the SQL statement, that's what I had to use. Apparently, that assumption was incorrect.
Regardless, it all seems to be working correctly now. Hope this helps someone else who's looking to accomplish something similar.
I am using below command to write data to csv file in isql
$ISQL -S DSA1_PROD -U emer_r_gh5432 -X
Query -
Select * from SecDb..LoginOwnerTb where SvrId= 45566 and OwnerRitsId = '1001167635';
OUTPUT TO '/tmp/sometable.csv' FORMAT ASCII DELIMITED BY ';' QUOTE '';
go
it says
Server 'ABC', Line 1:
Incorrect syntax near ';'.
Please help
NOTE: I'm assuming you're working with Sybase ASE and the isql command line tool. There may be other ways to accomplish what you're trying to do when going against the SQLAnywhere, IQ and/or Advantage database products ... *shrug* ...
The OUTPUT TO clause is used with the dbisql GUI tool.
To perform a somewhat-similar OUTPUT operation with the isql command line tool:
-- once logged in via isql ...
-- to write to new file; to overwrite existing file:
select ....
go > /path/to/local/file/accessible/by/user/running/isql
-- to append to existing file:
select ...
go >> /path/to/local/file/accessible/by/user/running/isql
To set the column delimiter you can use the -s flag when invoking isql from the command line, eg:
# set the column delimiter to a semi-colon:
$ isql ... -s ';' ...
# set the column delimiter to a pipe:
$ isql ... -s '|' ...
Keep in mind that the output will still be generated using fixed-width columns, with each column's width determined by either a) the column's datatype 'width' or b) the column's title/label width, whichever is wider.
I'm not aware of any way to perform the following with the isql command line tool:
designate a column delimiter on-the-fly while inside a isql session
designate a quote character
remove extra spaces (ie, ouput data in true delimited format as opposed to fixed-width format)
To generate true delimited files you have a few options:
see if the dbisql GUI tool serves your purpose [I don't use dbisql so I'm *assuming* the OUTPUT TO clause works as expected]
use the bcp (command line) utility to place the data into a delimited file [bcp options, and how to handle subsets of tables, is a much larger discussion, ie, too much to address in this response]
see if you can find another (3rd party) tool that can extract the desired data set to a delimited file
I have created a BCP utility and I have wrapped it in a bat file. I have then created a daily task using Task Scheduler in Windows Server 2012.
The function of the BCP utility is to rename a file called 'myfile.csv' (located in C:) by adding a date stamp to it and updating the file with the result of a SQL query.
The codes currently stand as follows:
cd:\Program Files\ Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn
set vardate=%DATE:~4,10%
set varDateWithoutSlashes=%vardate:/=-%
ren C:\myfile.csv myfile_%varDateWithoutSlashes%.csv
bcp "SELECT TOP 100 ReservationStayID,NameTitle,FirstName,LastName,ArrivalDate,DepartureDate FROM MyDatabase.dbo.GuestNameInfo" queryout C:\myfile.csv -t, -c -S [ipaddress] -U sa -P 1234
My problem is that when the task runs, it renames the file correctly with a the date stamp but it seems that the SELECT query does not run as the file is empty (except the headers, which have been pre-loaded by the way).
What is wrong with my codes?
I should also add the following:
Are the double quotes in the select statement above correct? Or should they be single quotes?
Should the ipaddress in my codes above be in square brackets or should I remove them?
I have left the "Location" filed 'as is' in the Task Scheduler (please see screenshot below). Should that be filled? If yes, by what?
Thanks for helping out!
I was wondering if anyone can help.
I have a number of queries in SQL (all in separate *.sql files). I wanted to know if there is a way to run these queries automatically or mass run them to be saved to either a csv or txt file?
Also, I have come variables within these queries which will need to be amended on a weekly bases before the queries are run.
Thanks.
KJ
Could you please provide some additional help in relation to the variables? Previously I would declare and set variables as:
DECLARE #TW_FROM DATETIME
DECLARE #TW_TO DATETIME
SET #TW_FROM = '2015-11-16 00:00:00';
SET #TW_TO = '2015-11-22 23:00:00';
How do I do this using sqlcmd?
Yes, you can use sqlcmd to do this.
First of all - variables. You can refer to your variables in the .sql files using $(variablename) wherever you want to substitue the variable. For example,
use $(dbname);
select $(columnname) from table1 where column= '$(var1)'
You then call sqlcmd with the following command (note the argument -v variables)
sqlcmd -S servername -d database -i "yoursqlfile.sql" -v dbname="database" columnname="column" var1="Fred"
In order to output this to a file, you tag > filename.txt on the end
sqlcmd -S servername -d database -i "yoursqlfile.sql" -v dbname="database" columnname="column" var1="Fred" > filename.txt
If you want to output to a csv, you can also specify the delimiter using the argument -s (note the idfference with the capital S for server). So now we have
sqlcmd -S servername -d database -s "," -i "yoursqlfile.sql" -v dbname="database" columnname="column" var1="Fred" > filename.csv
If you want to output several commands to the same csv or txt file, use >> instead of > as it add to teh bottom of the file, rather than replacing it.
sqlcmd -S servername -d database -s "," -i "yoursqlfile.sql" -v dbname="database" columnname="column" var1="Fred" >> filename.csv
To run this for several scripts, you can put the statements in a batch file, and then change the variables every week.
You could write a batch file that uses sqlcmd:
MSDN sqlcmd
That will allow you to call script files in a loop and output the results to a file.
Convert your current scrips to a Stored Procedure.
You can then pass your variables to that and run the query.
If you have SQL Server agent available (SQL standard or better) you can use this to automate the running of the stored procedures.
Otherwise the same can be achieved with Task Scheduler in windows.
As for exporting to CSV this will be useful.
It depends on where your SQL Server is acutally running. It might be quite tricky to write anything to the location you want.
You could read about BCP.
My suggestion is:
Create an UDF (best is inline-UDF!) from all of your queries within your database. Than call them from EXCEL or any other fitting product. You might want to set up an Excel where all your queries are filled one on each Sheet automatically
I am trying to execute BCP within SSIS to export the results of a query into a CSV file for several different tables. However, for some reason I keep getting parsing errors when I try to put the following into the arguments for the bcp exec:
"SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T
When I execute BCP with these arguments (changing the variable to an existing table name), everything works fine. I tried to remove several parts of the argument and I still get the following error
Attempt to parse the expression ""SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T" failed. The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a required element such as a parenthesis.
What is the issue with my argument?
Your expression is incorrect.
"SELECT * from myDB.dbo.#[User::Table] " queryout C:\users\MSSQLSERVER\Downloads\#[User::Table].csv -c -t, -T
The SSIS expression language isn't like PowerShell where it will see the variable inside the string. Instead, we are back in the stone ages with string concatenation. You also get to deal with escaping slashes and double quotes so your final expression would look something like
"""SELECT * from myDB.dbo."
+ #[User::Table]
+ """ queryout C:\\users\\MSSQLSERVER\\Downloads\\"
+ #[User::Table]
+ ".csv -c -t, -T"
I find I have the most success when I build complex strings in Variables and then only ever reference the built Variable in an Expression on a Task versus trying to build it in there. That way I can put a breakpoint in a package or raise an Information event back with my Variable's value. Makes debugging a fiddly process a much less painful experience.
bcp reference