I have the following batch file and SQL script that runs several .sql files. Is there a way to log any errors and continue with script?
myautosql.sql
PRINT 'Starting...'
--:On Error exit
:r SQLQuery10.sql
:r SQLQuery20.sql
:r SQLQuery30.sql
PRINT 'Completed normally on ' + (CONVERT( VARCHAR(24), GETDATE(), 101))
GO
myautosql.bat
SQLCMD -E -d rstestdb1 -i myautosql.sql
PAUSE
When I intentionally raiseerror in the SQLQuery20.sql file the batch program stops. Is there a way for it to log the error and continue with script?
When you raiserror, the 2nd parameter severity dictates whether the query will continue to run or not. A severity of 0-10 are informational (no error is raised), 11-19 are non fatal errors, 20-25 will raise the error and then immediately terminate your connection to the database server. You must be a sysadmin to use a severity from 19-25.
I think this simulates what you are trying to do.
auto.sql
PRINT 'Starting...'
:r 1.sql
:r 2.sql
PRINT 'Completed normally on ' + (CONVERT( VARCHAR(24), GETDATE(), 101))
1.sql
select 1 as value
raiserror ('This will not stop execution', 11, 1)
select 2 as value
2.sql
select 3 as value
raiserror ('This is an information message, not an error', 10, 1)
select 4 as value
Then you run the following command to capture the query output to output.txt and informational/error messages to error.txt:
sqlcmd -E -d tempdb -i auto.sql -r1 >output.txt 2>error.txt
-r1 tells sqlcmd to redirect informational/error messages to STDERR.
>output.txt redirects STDOUT from the queries (including affected row counts) to a file called output.txt.
2>error.txt redirects STDERR to a file called error.txt.
Here are the two files from the above script:
output.txt
value
-----------
1
(1 rows affected)
value
-----------
2
(1 rows affected)
value
-----------
3
(1 rows affected)
value
-----------
4
(1 rows affected)
error.txt
Starting...
Msg 50000, Level 11, State 1, Server NAME, Line 4
This will not stop execution
This is an information message, not an error
Completed normally on 02/27/2020
HTH
Related
I am running a moderately large script (about 900MB) with SQLCMD.exe and I am running into an odd situation.
The script consists of several thousand insert statements. When I run it, it will run for some time, executing fine, then at some point it will exit with the error:
Sqlcmd: Error: Syntax error at line _SOME_LINE_NUMBER_ in file
'd:\myscript.sql'.
Now, the problem is, there IS no error at that line number (or anywhere else in the script). If I go to the given line and grab the statement block in which it is contained, it will run no problem in SSMS. I usually go to the file and truncate it such that it starts at the statement statement that SQLCMD didn't like (actually, I start a couple statements before it) - i.e. I remove the stuff that has already run. I run the new sql file with SQLCMD and the whole process repeats itself with the same error referencing a different line (the original line that it complained about causes no problems this time).
If I re-run the script without editing it (cutting out the already processed parts), then it seems to fail on the same line number every time I re-run it. Again, there is actually no syntax error.
This is an example of the several thousand statement blocks in the file:
DECLARE #tblOutput table (ID int);
BEGIN TRANSACTION
IF NOT EXISTS (SELECT * FROM MyTable WHERE ID=123)
BEGIN
INSERT INTO MyTable(Col1, Col2, Col3, Col4, Col5, Col6, Col7, Col8, Col9)
OUTPUT INSERTED.ID INTO #tblOutput(ID)
VALUES(1, '2', '3', '4', 5, 6, (SELECT TOP 1 ID FROM SomeOtherTable WHERE Name='Something'), (SELECT TOP 1 ID FROM YetAnotherTable WHERE Name='321'), 9 )
INSERT INTO MyTable2(Col1, Col2, FKID)
VALUES(1, 0, (SELECT TOP 1 ID FROM #tblOutput) )
END
IF NOT EXISTS (SELECT * FROM RelatedTable WHERE stuff='something')
BEGIN
INSERT INTO RelatedTable(Col1, Col2)
VALUES ('test', (SELECT TOP 1 ID FROM #tblOutput) )
END
COMMIT TRANSACTION
DELETE FROM #tblOutput
GO
Running SQLCMD.exe as follows:
SQLCMD.EXE -S localhost -d mydatabase -V 17 -i dataimport.sql -o dataimport.log
Can anybody provide a guess as to why this might be occurring? I thought it might be memory-related (I'm running this on a local instance that only had 512MB allocated to SQL Server) so I bumped available memory up to 2GB but with no change in outcome.
FYI, this is SQL Server 2014 and I run the script with -V 17 (which has no effect I guess because a syntax error is sufficiently high error level?).
Thanks in advance.
To summarize from my comments:
Verify the command line options you're using for sqlcmd.exe. Note that the options are case sensitive. You should be able to use -b -e to tell sqlcmd to stop executing when it encounters an error and to also echo the command that it's trying to run.
You should also consider the -x (disable variable substitution) and -X (disable script commands) parameters. If data you're inserting potentially contains data that looks like an sqlcmd substitution variable, $(variable), you will need to specify -x to disable that feature.
In Windows 10, it seems that all RAISERROR WITH NOWAIT messages called from sqlcmd are now hidden in the command prompt until the batch is complete:
> sqlcmd -Q "raiserror('Test message', 10, 1) with nowait; waitfor delay '00:00:05'" -b -r1
(no output to console until waitfor is finished)
Compare this to the same command run from Windows 2008 R2:
> sqlcmd -Q "raiserror('Test message', 10, 1) with nowait; waitfor delay '00:00:05'" -b -r1
Test message
(immediate output)
In the above example, you can see the message even though the batch hasn't completed.
Is there a configuration setting in Windows 10 that will bring back this functionality? I have legacy batch files that depend on WITH NOWAIT to show progression.
This occurs because sqlcmd11 (not sure about higher) does not honor RAISERROR WITH NOWAIT. It does not seem to have anything to do with the underlying OS.
Here's a link to a stackoverflow post with some possible work-arounds.
Using SQLCMD, I am running a script to output to STDOUT then gziping the output. When I look at the output file, I see this warning message:
Database name 'tempdb' ignored, referencing object in tempdb.
In my script, I have a check at the start of the script to drop the temp table if it exists:
IF OBJECT_ID('tempdb..#TheTable') IS NOT NULL
BEGIN
DROP TABLE tempdb..#TheTable
END
However - I have also SET NOCOUNT ON, but file still captures the warning message.
SQLCMD Script:
sqlcmd -i TheScript.sql -h-1 -k1 -s"," -W -u | gzip > "C:\TheOutput.gz"
Is there a way to suppress a message like that?
Change your if condition to the following pattern:
IF 0 < OBJECT_ID('tempdb..#TheTable')
DROP TABLE #TheTable
This should not result in any error messages.
Simple clean version that works on SQL Server 2016 and above without any messages:
drop table if exists #TheTable
I am calling an SQL Server script on a batch file, but I need get the error(when the script fail) in the batch file, what can I do?
This is the batch file:
sqlcmd -S HOST -U User -P password -i test.sql
echo %errorlevel%
and this is the Script File(Test.sql):
USE TrainingSitecore_Web
go
SELECT * FROM items
go
RAISERROR ('This is a test Error.',-- Message text.
16,-- Severity.
1 -- State.
);
Make sense?
It can be done using -r parameter:
Example:
sqlcmd -Q "select 1 as a; select 1/0 as b" -E -r1 1> NUL
-r[ 0 | 1] msgs to stderr
Redirects the error message output to the screen (stderr). If you do
not specify a parameter or if you specify 0, only error messages that
have a severity level of 11 or higher are redirected. If you specify
1, all error message output including PRINT is redirected. Has no
effect if you use -o. By default, messages are sent to stdout.
MSDN for more
How do I stop psql (PostgreSQL client) from outputting notices? e.g.
psql:schema/auth.sql:20: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "users_pkey" for table "users"
In my opinion a program should be silent unless it has an error, or some other reason to output stuff.
SET client_min_messages TO WARNING;
That could be set only for the session or made persistent with ALTER ROLE or ALTER DATABASE.
Or you could put that in your ".psqlrc".
Probably the most comprehensive explanation is on Peter Eisentrauts blog entry here (Archive)
I would strongly encourage that the original blog be studied and digested but the final recommendation is something like :
PGOPTIONS='--client-min-messages=warning' psql -X -q -a -1 -v ON_ERROR_STOP=1 --pset pager=off -d mydb -f script.sql
Use --quiet when you start psql.
A notice is not useless, but that's my point of view.
It can be set in the global postgresql.conf file as well with modifiying the client_min_messages parameter.
Example:
client_min_messages = warning
I tried the various solutions suggested (and permutations thereof) suggested in this thread, but I was unable to completely suppress PSQL output / notifications.
I am executing a claws2postgres.sh BASH script that does some preliminary processing then calls/executes a PSQL .sql script, to insert 1000's of entries into PostgreSQL.
...
PGOPTIONS="-c client_min_messages=error"
psql -d claws_db -f claws2postgres.sql
Output
[victoria#victoria bash]$ ./claws2postgres.sh
pg_terminate_backend
----------------------
DROP DATABASE
CREATE DATABASE
You are now connected to database "claws_db" as user "victoria".
CREATE TABLE
SELECT 1
INSERT 0 1
UPDATE 1
UPDATE 1
UPDATE 1
Dropping tmp_table
DROP TABLE
You are now connected to database "claws_db" as user "victoria".
psql:/mnt/Vancouver/projects/ie/claws/src/sql/claws2postgres.sql:33: NOTICE: 42P07: relation "claws_table" already exists, skipping
LOCATION: transformCreateStmt, parse_utilcmd.c:206
CREATE TABLE
SELECT 1
INSERT 0 1
UPDATE 2
UPDATE 2
UPDATE 2
Dropping tmp_table
DROP TABLE
[ ... snip ... ]
SOLUTION
Note this modified PSQL line, where I redirect the psql output:
psql -d claws_db -f $SRC_DIR/sql/claws2postgres.sql &>> /tmp/pg_output.txt
The &>> /tmp/pg_output.txt redirect appends all output to an output file, that can also serve as a log file.
BASH terminal output
[victoria#victoria bash]$ time ./claws2postgres.sh
pg_terminate_backend
----------------------
DROP DATABASE
CREATE DATABASE
2:40:54 ## 2 h 41 min
[victoria#victoria bash]$
Monitor progress:
In another terminal, execute
PID=$(pgrep -l -f claws2postgres.sh | grep claws | awk '{ print $1 }'); while kill -0 $PID >/dev/null 2>&1; do NOW=$(date); progress=$(cat /tmp/pg_output.txt | wc -l); printf "\t%s: %i lines\n" "$NOW" $progress; sleep 60; done; for i in seq{1..5}; do aplay 2>/dev/null /mnt/Vancouver/programming/scripts/phaser.wav && sleep 0.5; done
...
Sun 28 Apr 2019 08:18:43 PM PDT: 99263 lines
Sun 28 Apr 2019 08:19:43 PM PDT: 99391 lines
Sun 28 Apr 2019 08:20:43 PM PDT: 99537 lines
[victoria#victoria output]$
pgrep -l -f claws2postgres.sh | grep claws | awk '{ print $1 }' gets the script PID, assigned to $PID
while kill -0 $PID >/dev/null 2>&1; do ... : while that script is running, do ...
cat /tmp/pg_output.txt | wc -l : use the output file line count as a progress indicator
when done, notify by playing phaser.wav 5 times
phaser.wav: https://persagen.com/files/misc/phaser.wav
Output file:
[victoria#victoria ~]$ head -n22 /tmp/pg_output.txt
You are now connected to database "claws_db" as user "victoria".
CREATE TABLE
SELECT 1
INSERT 0 1
UPDATE 1
UPDATE 1
UPDATE 1
Dropping tmp_table
DROP TABLE
You are now connected to database "claws_db" as user "victoria".
psql:/mnt/Vancouver/projects/ie/claws/src/sql/claws2postgres.sql:33: NOTICE: 42P07: relation "claws_table" already exists, skipping
LOCATION: transformCreateStmt, parse_utilcmd.c:206
CREATE TABLE
SELECT 1
INSERT 0 1
UPDATE 2
UPDATE 2
UPDATE 2
Dropping tmp_table
DROP TABLE
References
[re: solution, above] PSQL: How can I prevent any output on the command line?
[re: this SO thread] disable NOTICES in psql output
[related SO thread] Postgresql - is there a way to disable the display of INSERT statements when reading in from a file?
[relevant to solution] https://askubuntu.com/questions/350208/what-does-2-dev-null-mean
The > operator redirects the output usually to a file but it can be to a device. You can also use >> to append.
If you don't specify a number then the standard output stream is assumed but you can also redirect errors
> file redirects stdout to file
1> file redirects stdout to file
2> file redirects stderr to file
&> file redirects stdout and stderr to file
/dev/null is the null device it takes any input you want and throws it away. It can be used to suppress any output.
Offering a suggestion that is useful for a specific scenario I had:
Windows command shell calls psql.exe call to execute one essential SQL command
Only want to see warnings or errors, and suppress NOTICES
Example:
psql.exe -c "SET client_min_messages TO WARNING; DROP TABLE IF EXISTS mytab CASCADE"
(I was unable to make things work with PGOPTIONS as a Windows environment variable--couldn't work out the right syntax. Tried multiple approaches from different posts.)