Unix Script exiting when select query returns no rows - database

Below is the part of my script.
The "set -e" will make the script exit whenever a negative return code comes from any of the commands.
But, when the below select statement returns no rows from the table, the script exits there itself (echo "Get Eg names end" is not executed). Which means the below command is giving negative return code.
db2 -x "SELECT EG_NAME FROM MS.CFG_CACHE_REFRESH WHERE
EG_RELOAD_UPD_BK2_TS < M_TABLE_UPD_TS AND CURRENT_TIME >= EG_RELOAD_START_TIME AND
CURRENT_TIME <= EG_RELOAD_END_TIME"
> /home/DummyUser/gm4/logs/MP_CACHE_REFRESH_TEMP_BK2.txt
If the select statement returns some rows, it works fine. The script doesn't exit and runs till the end.
My requirement is to exit if a genuine error occurs, like unable to connect database, invalid syntax etc.
If no rows are returned from the table, it should not be considered as an error.
Why am I getting a -ve return code for the select query not returning rows and how can I handle this?
Below is the part of the script:
#!/bin/ksh
set -e
brokername=$1
if [ "$#" -ne "1" ]
then
echo "Invalid arugments supplied"
echo "call the script using command:"
echo "MP_CACHE_REFRESH_BRK2.ksh <BrokerName>"
exit -1
fi
touch /home/DummyUser/gm4/logs/MP_CACHE_REFRESH_TEMP_BK2.txt
chmod 777 /home/DummyUser/gm4/logs/MP_CACHE_REFRESH_TEMP_BK2.txt
db2 CONNECT TO MSAPP USER DummyUser using paasss
db2 -x "SELECT EG_NAME FROM MS.CFG_CACHE_REFRESH WHERE EG_RELOAD_UPD_BK2_TS < M_TABLE_UPD_TS AND CURRENT_TIME >= EG_RELOAD_START_TIME AND CURRENT_TIME <= EG_RELOAD_END_TIME" > /home/DummyUser/gm4/logs/MP_CACHE_REFRESH_TEMP_BK2.txt
echo "Get Eg names end"

The problem is not negative return codes, it is any return code that is != 0. DB2 exits with:
- 0, success
- 1, no row found
- 2, warning (for example using existing index instead of creating a new one)
- 4, error (for example object not found)
- 8, system error (os related)
Unless you wrap db2 and return 0 for $? -lt 4 I don't see how you are going to success. Example on how to deal with db2 exit codes (-e removed)
db2 -x "SELECT EG_NAME FROM MS.CFG_CACHE_REF ..."
rc=$?
if [ $rc -ge 4 ]; then
...
EDIT: Alternative with sql stmts in separate file
Keeping all the sql in a separate file - say myfile.sql - you can do something like from your sh:
db2 -l myfile.log +c -s -tf myfile.sql
rc=$?
if [ $rc -ge 4 ]; then
echo "Error"
db2 rollback
exit 1
elif [ $rc -ge 1 ]; then
echo "Warning"
fi
db2 commit
exit 0
-s terminates execution on error ( -ge 4 ). You can find out what caused the problem by tailing the log file, tail -10 myfile.log. Beware that certain operations such as reorg will commit the ongoing transaction.

Related

psql -v ON_ERROR_STOP=1 not working, cannot terminate script execution

I have a bash script that calls another script containing a psql command, i want to stop executing when the psql command fails. The below psql call is to execute a postgres function.
In below script the echo $exitcode always returns 0 even if script calling postgres function returns a error. I want psql command to return right error code, so i can handle it properly in below script.
FirstScript:
list=`sh /opt/dbb_promo_list.sh -60 -30`
ID_LIST=`echo $list |sed 's/[a-z _-]*//g'|sed 's/(.*//g'`
exitcode=$?
echo $exitcode //always returns 0 both in successful and error case, i want it to return corresponding exit code for error, so i can handle it and stop the script.
if [ $exitcode -ne 0 ] ; then
echo "There seems to have errors while running script "
echo "Exit code is : "$exitcode
exit 1
fi
dbb_promo_list.sh : This script contains the following command
psql -U $DB_USERNAME -h $DB_HOST -p $DB_PORT -d $DATABASE -c "SELECT schema.${PROCEDURE}($day_begin,$day_end)"
Error thrown below when calling the above command. Since this error is thrown the psql command shouldnt return exit code 0 but its not the case, it returns 0 even if the postgres function works fine or returns a error like below. I am not sure if there a way to tweak in bash script to recognize a error is thrown
ERROR: function string_agg(numeric, unknown) does not exist
LINE 1: SELECT string_agg(pmotn_id, ',') FROM ppcapr.p...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
When the sql function fails. It then reports the error on the console, but continues to proceed with executing remainder of the script. But i want the script execution to terminate once the error occurs.
Any suggestions?

While loop in snakemake; checkpoints

One of the tools I use in snakemake can on the same data in one of the N cases return me a weird signal (non zero status) and does not record output. According to this, I want to restart this rule every time I get such a signal and get empty output. I see it as a kind of "while-cycle". I know that this kind of logic to work without an explicit DAG contradicts the idea of snakemake, but with the appearance of checkpoints, I believe that a solution can be found.
Thanks!
You cannot "restart the rule", but you can run the same command multiple times.
Here is the recipe how to mask the error:
shell:
"""
set +e
somecommand ...
exitcode=$?
if [ $exitcode -eq 1 ]
then
exit 1
else
exit 0
fi
"""
The same approach should work with the while loop in bash:
shell:
"""
set +e
somecommand ...
exitcode=$?
while [ $exitcode -ne 0 ]
do
somecommand ...
exitcode=$?
done
"""

Check database connectivity

I'm writing a unix script to check for database connectivity in a server. When my database connection gets errored out or when there is delay observed in connecting to the database, I want the output as "Not connected". In case it gets connected, my output should be "Connected". It is a Oracle databse.
When there is delay in database connectivity, my code is not working and my script gets hung. What changes should I make in my code so that it is able to handle both the conditions(when I get an error connecting to the database and when there is delay observed in connecting to the database)??
if sqlplus $DB_USER/$DB_PASS#$DB_INSTANCE< /dev/null | grep 'Connected to'; then
echo "Connectivity is OK"
else
echo "No Connectivity"
fi
The first thing to add to your code is a timeout. Checking database connectivity is not easy and there can be all kinds of problems in the various layers that your connection passes. A timeout gives you the option to break out of a hanging session and continue the task with reporting that the connection failed.
googleFu gave me a few nice examples:
Timeout a command in bash without unnecessary delay
If you are using Linux, you can use the timeout command to do what you want. So the following will have three outcomes, setting the variable RC as follows:
"Connected to" successful: RC set to 0
"Connected to" not found: RC set to 1
sqlplus command timed out after 5 minutes: RC set to 124
WAIT_MINUTES=5
SP_OUTPUT=$(timeout ${WAIT_MINUTES}m sqlplus $DB_USER/$DB_PASS#$DB_INSTANCE < /dev/null )
CMD_RC=$?
if [ $CMD_RC -eq 124 ]
then
ERR_MSG="Connection attempt timed out after $WAIT_MINUES minutes"
RC=$CMD_RC
else
echo $SP_OUTPUT | grep -q 'Connected to'
GREP_RC=$?
if [ $GREP_RC -eq 0 ]
then
echo "Connectivity is OK"
RC=0
else
ERR_MSG="Connectivity or user information is bad"
RC=1
fi
fi
if [ $RC -gt 0 ]
then
# Add code to send email with subject of $ERR_MSG and body of $SP_OUTPUT
echo Need to email someone about $ERR_MSG
fi
exit $RC
I'm sure there are several improvements to this, but this will get you started.
Briefly, we use the timeout command to wait the specified time for the sqlplus command to run. I separated out the grep as a separate command to allow the use of timeout and to allow more flexibility in checking additional text messages.
There are several examples on StackOverflow on sending email from a Linux script.

repeat pipe command until first command succeeds and second command fails

I am trying to figure out how to get my bash script to work. I have a the following command:
curl http://192.168.1.2/api/queue | grep -q test
I need it to repeat until the first command in the pipline succeeds (meaning the server responds) and the second command fails (meaning the pattern is not found or the queue is empty). I've tried a number of combinations but just can't seem to get it. I looked at using $PIPESTATUS but can't get it to function in a loop how I want. I've tried all kind of variations but can't get it to work. This is what I am currently trying:
while [[ "${PIPESTATUS[0]}" -eq 0 && "${PIPESTATUS[1]}" -eq 1 ]]
do curl http://192.168.1.2 | grep -q regular
echo "The exit status of first command ${PIPESTATUS[0]}, and the second command ${PIPESTATUS[1]}"
sleep 5
done
Although it's not really clear what kind of output is returned by the curl call, maybe you are looking for something like this:
curl --silent http://192.168.1.2 |while read line; do
echo $line |grep -q regular || { err="true"; break }
done
if [ -z "$err" ]; then
echo "..All lines OK"
else
echo "..Abend on line: '$line'" >&2
fi
Figured it out. Just had to re-conceptualize it. I couldn't figure it out strictly with a while or until loop but creating an infinite loop and breaking out of it when the condition is met worked.
while true
do curl http://192.168.1.2/api/queue | grep -q test
case ${PIPESTATUS[*]} in
"0 1")
echo "Item is no longer in the queue."
break;;
"0 0")
echo "Item is still in the queue. Trying again in 5 minutes."
sleep 5m;;
"7 1")
echo "Server is unreachable. Trying again in 5 minutes."
sleep 5m;;
esac
done

Check database connectivity using Shell script

I am trying to write a shell script to check database connectivity. Within my script I am using the command
sqlplus uid/pwd#database-schemaname
to connect to my Oracle database.
Now I want to save the output generated by this command (before it drops to SQL prompt) in a temp file and then grep / find the string "Connected to" from that file to see if the connectivity is fine or not.
Can anyone please help me to catch the output and get out of that prompt and test whether connectivity is fine?
Use a script like this:
#!/bin/sh
echo "exit" | sqlplus -L uid/pwd#dbname | grep Connected > /dev/null
if [ $? -eq 0 ]
then
echo "OK"
else
echo "NOT OK"
fi
echo "exit" assures that your program exits immediately (this gets piped to sqlplus).
-L assures that sqlplus won't ask for password if credentials are not ok (which would make it get stuck as well).
(> /dev/null just hides output from grep, which we don't need because the results are accessed via $? in this case)
You can avoid the SQL prompt by doing:
sqlplus uid/pwd#database-schemaname < /dev/null
SqlPlus exits immediately.
Now just grep the output of the above as:
if sqlplus uid/pwd#database-schemaname < /dev/null | grep 'Connected to'; then
# have connectivity to Oracle
else
# No connectivity
fi
#! /bin/sh
if echo "exit;" | sqlplus UID/PWD#database-schemaname 2>&1 | grep -q "Connected to"
then echo connected OK
else echo connection FAIL
fi
Not knowing whether the "Connected to" message is put to standard output or standard error, this checks both. "qrep -q" instead of "grep... >/dev/null" assumes Linux.
#!/bin/bash
output=`sqlplus -s "user/pass#POLIGON.TEST " <<EOF
set heading off feedback off verify off
select distinct machine from v\\$session;
exit
EOF
`
echo $output
if [[ $output =~ ERROR ]]; then
echo "ERROR"
else
echo "OK"
fi
Here's a good option which does not expose the password on the command line
#!/bin/bash
CONNECT_STRING=<USERNAME>/<PASS>#<SID>
sqlplus -s -L /NOLOG <<EOF
whenever sqlerror exit 1
whenever oserror exit 1
CONNECT $CONNECT_STRING
exit
EOF
SQLPLUS_RC=$?
echo "RC=$SQLPLUS_RC"
[ $SQLPLUS_RC -eq 0 ] && echo "Connected successfully"
[ $SQLPLUS_RC -ne 0 ] && echo "Failed to connect"
exit SQLPLUS_RC
none of the proposed solutions works for me, as my script is executed in machines running several countries, with different locales, I can't simply check for one String simply because this string in the other machine is translated to a different language. As a solution I'm using SQLcl
https://www.oracle.com/database/technologies/appdev/sqlcl.html
which is compatible with all sql*plus scripts and allow you to test the database connectivity like this:
echo "disconnect" | sql -L $DB_CONNECTION_STRING > /dev/null || fail "cannot check connectivity with the database, check your settings"
#!/bin/sh
echo "exit" | sqlplus -S -L uid/pwd#dbname
if [ $? -eq 0 ]
then
echo "OK"
else
echo "NOT OK"
fi
For connection validation -S would be sufficient.
The "silent" mode doesn't prevent terminal output. All it does is:
-S Sets silent mode which suppresses the display of
the SQL*Plus banner, prompts, and echoing of
commands.
If you want to suppress all terminal output, then you'll need to do something like:
sqlplus ... > /dev/null 2>&1
This was my one-liner for docker container to wait until DB is ready:
until sqlplus -s sys/Oracle18#oracledbxe/XE as sysdba <<< "SELECT 13376411 FROM DUAL; exit;" | grep "13376411"; do echo "Could not connect to oracle... sleep for a while"; sleep 3; done
And the same in multiple lines:
until sqlplus -s sys/Oracle18#oracledbxe/XE as sysdba <<< "SELECT 13376411 FROM DUAL; exit;" | grep "13376411";
do
echo "Could not connect to oracle... sleep for a while";
sleep 3;
done
So it basically does select with magic number and checks that correct number was actually returned.

Resources