I want to execute a command which return all user's home directory that exist and store the result into bash variable
res=$(ssh -q -o LogLevel=error user#server << 'EOF'
t=$(awk -F':' '{ if ( $3 >= 500 ) print $1 }' /etc/passwd)
readarray -t res_array <<< "${t}"
UHOME="/home"
for u in "$res_array"
do
_dir="${UHOME}"/"${u}"
if [[ -d "$_dir" ]]
then
echo "$u"
fi
done
EOF
)
locally the code works but not on remote server
I used echo $u for store the resultat home user exist to the variable :(
do this:
res=$(ssh -q -o LogLevel=error user#server << 'EOF'
t=$(awk -F':' '{ if ( $3 >= 500 ) print $1 }' /etc/passwd)
readarray -t res_array <<< "${t}"
UHOME="/home"
for u in "${res_array[#]}" # change this line
do
_dir="${UHOME}"/"${u}"
if [[ -d "$_dir" ]]
then
echo "$u"
fi
done
EOF
)
Related
I have written a very simple port scanning script in bash. For my ports argument, i accept a single port (eg: 443), a comma-separated list (eg: 80,443), or a range (eg: 1-1000).
When I run my script with a single port or comma-separated list of ports, everything runs as it should:
~/projects/bash-port-scan# ./bash-port-scan.sh -i xx.xx.xxx.xxx -p 1,80,443 -v
Beginning scan of xx.xx.xxx.xxx
Port 1 closed
Port 80 open
Port 443 open
Scan complete.
~/projects/bash-port-scan# ./bash-port-scan.sh -i xx.xx.xxx.xxx -p 80 -v
Beginning scan of xx.xx.xxx.xxx
Port 80 open
Scan complete.
However, when I run with a range, I get:
~/projects/bash-port-scan# ./bash-port-scan.sh -i xx.xx.xxx.xxx -p 1-10 -v
Beginning scan of xx.xx.xxx.xxx
Port 1
2
3
4
5
6
7
8
9
10 closed
Scan complete.
Relevant code where I assign the array:
portarray=()
if [[ "$ports" == *","* ]]; then
IFS=','
read -r -a portarray <<< $ports
IFS=' '
elif [[ "$ports" == *"-"* ]]; then
IFS='-'
read -r -a range <<< $ports
IFS=' '
first="${range[0]}"
last="${range[1]}"
portarray=($(seq $first 1 $last))
else
portarray=($ports)
fi
and the loop itself:
empty=""
for p in "${portarray[#]}"; do
result=$(nc -zvw5 $ip $p 2>&1 | grep open)
if [ "$result" = "$empty" ]; then
if [ $verbose -eq 1 ]; then
str="Port "
closed=" closed"
echo "$str$p$closed"
fi
else
str="Port "
closed=" open"
echo "$str$p$closed"
fi
done
I'm not sure if this is because of how I'm assigning my port array, or if it is because of something I have wrong in my loop. I'm relatively new to bash scripting, and I'm having a terrible time figuring out what I have wrong.
I've read here on SO about some commands run in loops eating the output of other portions of the script, but I don't believe that to be the case here, as the script does actually print to screen, just not as expected.
EDIT:
Here is the full script:
#!/bin/bash
verbose=0
while [ "$1" != "" ]; do
case "$1" in
-h | --help )
echo "bash-port-scan.sh v0.1\r\nUsage: ./bash-port-scan.sh -i 127.0.0.1 -p 80,443\r\n./bash-port-scan.sh -i 127.0.0.1 -p 1-1000"; shift;;
-v | --verbose )
verbose=1; shift;;
-i | --ip )
ip="$2"; shift;;
-p | --ports )
ports="$2"; shift;;
esac
shift
done
if [[ $ip = "" ]]; then
echo "Please enter an IP address with -i"
exit
fi
if [[ $ports = "" ]]; then
echo "Please enter the port(s) with -p"
exit
fi
portarray=()
if [[ "$ports" == *","* ]]; then
IFS=','
read -r -a portarray <<< $ports
IFS=' '
elif [[ "$ports" == *"-"* ]]; then
IFS='-'
read -r -a range <<< $ports
IFS=' '
first="${range[0]}"
last="${range[1]}"
portarray=($(seq $first $last))
else
portarray=($ports)
fi
if [ $verbose -eq 1 ]; then
echo "Beginning scan of $ip"
fi
shuf -e "${portarray[#]}"
empty=""
for p in "${portarray[#]}"; do
result=$(nc -zvw5 $ip $p 2>&1 | grep open)
if [ "$result" = "$empty" ]; then
if [ $verbose -eq 1 ]; then
str="Port "
closed=" closed"
echo "$str$p$closed"
fi
else
str="Port "
closed=" open"
echo "$str$p$closed"
fi
done
echo "Scan complete."
Addressing just the portarray=(...) assignment (when ports=1-10)
Consider:
$ first=1
$ last=10
$ portarray=($(seq $first 1 $last))
$ typeset -p portarray
declare -a portarray=([0]=$'1\n2\n3\n4\n5\n6\n7\n8\n9\n10')
NOTE: the output from the $(seq ...) call is processed as a single string with embedded linefeeds.
A couple ideas:
### define \n as field separator; apply custom IFS in same line to limit IFS change to just the follow-on array assignment:
$ IFS=$'\n' portarray=($(seq $first 1 $last))
$ typeset -p portarray
declare -a portarray=([0]="1" [1]="2" [2]="3" [3]="4" [4]="5" [5]="6" [6]="7" [7]="8" [8]="9" [9]="10")
### use mapfile to read each *line* into a separate array entry:
$ mapfile -t portarray < <(seq $first 1 $last)
$ typeset -p portarray
declare -a portarray=([0]="1" [1]="2" [2]="3" [3]="4" [4]="5" [5]="6" [6]="7" [7]="8" [8]="9" [9]="10")
Struggling with this...
for i in `cat services.txt`
do
if ! grep -q $i; then
echo " $i Is NOT Running"
else
echo " Checking $i on `hostname`..."
ps aux | grep -i $i | awk '{print $1, $11}'| cut -d' ' -f1-2| sort
echo -e " "
sleep 4
fi
done
The block just hangs - Ive not been able to capture the success/failure of grep
If a string in services.txt is NOT found ... the script hangs... Id like for grep to skip it if not found
services.txt contain just single words
thanks!
The reason your script hangs is beacuse the command grep -q $i is waiting for an input. You can try running that command separately in a shell and verify that it prompts for an input.
Changing your command to ps aux | grep -i $i in the if statement should fix your issue.
NOTE: ps aux | grep -i $i lists grep also as one of the process. Make sure you exclude that process by piping it to another grep ps aux | grep -i $i | grep -v 'grep'
here is the working code
checkservices() {
cat >$HOME/services.txt <<EOF
ganglia
hbase
hdfs
hive
hue
livy
mapred
test-missing-service
mysql
oozie
presto
spark
yarn
zeppelin
EOF
for i in `cat $HOME/services.txt`
do
if `ps -ef | grep ^$i | grep -v grep >/dev/null`
then
i=$(echo "$i" | awk '{print toupper($0)}')
echo "$i -- is running on `hostname`"
echo ""
sleep 2
else
i=$(echo "$i" | awk '{print tolower($0)}')
echo "$i -- IS NOT running on `hostname` error"
echo ""
fi
done
}
This script should alert me in case of an error response.
Issue: Even when it executes successfully I am getting the email.
Bash script
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
IFS=\" read __ KEY __ MESSAGE __ <<< "$DATA"
if [[ $KEY == Success ]]; then
echo something
else
send email on failure
fi
Response on
Failure: {"ErrorCode":"11","ErrorMessage":"random message as per error code"}
Sucess: {"ErrorCode":"000","ErrorMessage":"Success"}
This worked finally -
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
MESSAGE=$(jq '.ErrorMessage' <<< "$DATA")
if [[ "$MESSAGE" == '"Success"' ]] ; then
echo something
else
send email
fi
A proper tool to address your issue would be jq :
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
KEY=$(jq -r '.ErrorCode' <<< "$DATA")
MESSAGE=$(jq -r '.ErrorMessage' <<< "$DATA")
if [[ "$KEY" = "000" ]]
then
echo success
else
echo fail
fi
Note : the -r flag for jq removes the double quotes
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
KEY=$(echo "$DATA" | grep -oP '"ErrorCode":"\K(\d+)"')
MESSAGE=$(echo "$DATA" | grep -oP '"ErrorMessage":"\K(.+?)(?=")')
((KEY == 0)) && echo success || echo "$MESSAGE"
I wrote the bash like so:
#!/bin/bash
GAP=1
Out=$1
ResultFile=$2
len=`wc -l $Out | awk '{print $1}'`
eval "(COMMAND) &"
pid=$!
i=0
while kill -0 $pid; do
if [ -N $Out ]; then
newlen=`wc -l $Out | awk '{print $1}'`
newlines=`expr $newlen - $len`
tail -$newlines $Out > temp
IP=( $(sed -n '<SomeThing>' temp) )
host=${IP[$i]}
echo "exit" | nc $host 23
if [ "$?" -eq "0" ]; then
(
<DoingSomeThing>
) | nc $host 23 1>>$ResultFile 2>&1
fi
len=$newlen
let i++
fi
sleep $GAP
done
When the command IP=( $(sed -n '<SomeThing>' temp) ) is running in my bash maybe the result of sed command is nothing and maybe the output is ip. I want only when output of sed command get ip write it into array and when the output of sed is empty does not write it to array.
Thank you
You're not doing your script right in many ways but about your question, the quick way is to store the output first on a variable:
SED_OUT=$(sed -n '<SomeThing>' temp)
[[ -n $SED_OUT ]] && IP=($SED_OUT) ## Would only alter IP if $SED_OUT has a value.
I'm having problems getting a variable "${Error[*]}", which is a regular indexed array, to stay set from the time it's declared until it's checked. It seems to me that a sub-shell must be launched so the declaration doesn't stick. I didn't think sub-shells were opened when using braces { stuff...; }. I want to know how to get my variable, Error to stick in the case I'm trying to write up. Here's a sample of my script:
TestFunction () {
unset Error
local archive="$1" extlist="$2" && local ext="${archive##*.}"
shopt -s nocasematch
local -i run=0
while [[ "$run" == 0 || -n "${Error[run]}" ]]; do
(( run++ ))
local IFS=$'\n\r\t '
if [[ ! "${Error[*]}" =~ 'cpio' && "$ext" =~ ^(pax|cpio|cpgz|igz|ipa|cab)$ && -n "$(which 'cpio')" ]]; then
## Try to cpio the archive. Since cpio cannot handle '.cab' archive, I want to declare an Error ##
{ cpio -ti --quiet <"$archive" 2>'/dev/null' || local -a Error[run]='cpio'; } | grep -Ei '$extlist'
elif [[ ! "${Error[*]}" =~ 'zipinfo' && "$ext" =~ ^(zip|[jw]ar|ipa|cab)$ && -n "$(which 'unzip')" ]]; then
## If cpio fails, then try zipinfo, or unzip on the next run through the loop... ##
if which 'zipinfo' &>'/dev/null'; then
{ zipinfo -1 "$archive" 2>'/dev/null' || local -a Error[run]='zipinfo'; } | grep -Ei "$scanlist"
elif which 'unzip' &>'/dev/null'; then
{ unzip -lqq "$archive" 2>'/dev/null' || local -a Error[run]='unzip'; } | gsed -re '/^ +[0-9]+/!d;s|^ +[0-9]+ +[0-9-]+ [0-9:]+ +||' | grep -Ei "$exlist"
fi
## many more elifs... ##
fi
done
shopt -u nocasematch
return 0
}
Archives='\.(gnutar|7-zip|lharc|toast|7zip|boz|bzi?p2?|cpgz|cpio|gtar|g?z(ip)?|lzma(86)?|t[bg]z2?|ar[cgjk]|bz[2a]?|cb[7rz]|cdr|deb|[dt]lz|dmg|exe|fbz|fgz|gz[aip]|igz|img|iso|lh[az]|lz[hmswx]?|mgz|mpv|mpz|pax|piz|pka|[jrtwx]ar|rpm|s?7-?z|sitx?|m?pkg|sfx|nz|xz)$'
IFS=$'\n'
declare -a List=($(TestFunction '/Users/aesthir/Programming/│My Projects│/Swipe Master/Test Folder/SDKSetup.cab' "$Archives"))
IFS=$' \t\n'
xtrace output:
〔xtrace〕 unset Error
〔xtrace〕 local 'archive=/Users/aesthir/Programming/│My Projects│/Swipe Master/Test Folder/SDKSetup.cab' 'extlist=\.(gnutar|7-zip|lharc|toast|7zip|boz|bzi?p2?|cpgz|cpio|gtar|g?z(ip)?|lzma(86)?|t[bg]z2?|ar[cgjk]|bz[2a]?|cb[7rz]|cdr|deb|[dt]lz|dmg|exe|fbz|fgz|gz[aip]|igz|img|iso|lh[az]|lz[hmswx]?|mgz|mpv|mpz|pax|piz|pka|[jrtwx]ar|rpm|s?7-?z|sitx?|m?pkg|sfx|nz|xz)$'
〔xtrace〕 local ext=cab
〔xtrace〕 shopt -s nocasematch
〔xtrace〕 local -i run=0
〔xtrace〕 [[ 0 == 0 ]]
〔xtrace〕 (( run++ ))
〔xtrace〕 local 'IFS=
'
〔xtrace〕 [[ ! '' =~ cpio ]]
〔xtrace〕 [[ cab =~ ^(pax|cpio|cpgz|igz|ipa|cab)$ ]]
〔xtrace〕 which cpio
〔xtrace〕 [[ -n /usr/bin/cpio ]]
〔xtrace〕 grep -Ei '$extlist'
〔xtrace〕 cpio -ti --quiet
〔xtrace〕 local -a 'Error[run]=cpio'
〔xtrace〕 [[ 1 == 0 ]]
〔xtrace〕 [[ -n '' ]] ## <—— Problem is here... when checking "${Error[run]}", it's unset ##
〔xtrace〕 shopt -u nocasematch
〔xtrace〕 return 0
Now obviously I know cpio, zipinfo, and unzip cannot handle cab files... I put 'cab' in the extension list on purpose to cause an error.
I want to stay in TestFunction and keep looping with different archivers until a success (file list is dumped, which cabextract would gladly do in this case) without repeating an already failed archiver.
Finally, since this works fine...
TestFunction () {
unset Error
local archive="$1" extlist="$2" && local ext="${archive##*.}"
local -i run=0
while [[ "$run" == 0 || -n "${Error[run]}" ]]; do
(( run++ ))
local IFS=$'\n\r\t '
if [[ ! "${Error[*]}" =~ 'cpio' && "$ext" =~ ^(pax|cpio|cpgz|igz|ipa|cab)$ && -n "$(which 'cpio')" ]]; then
cpio -ti --quiet <"$archive" 2>'/dev/null' || local -a Error[run]='cpio'
fi
done
shopt -u nocasematch
return 0
}
... I have to assume the problem is the braces because I want the results grep'd right away. However, I need those braces there because I don't want Error[run] to be set if grep turns up no results, only if cpio fails. I dont want to grep outside TestFunction for other reasons (I would have to do a complete re-write).
Any quick solution to this without massive rewriting? Maybe echo 'cpio' to some fd and read -u6ing it somehow?
I'd much prefer not to have to set an array to the file list and then for loop | grep through every file as it would really slow things down.
The problem is not the braces, but the pipe. Because you're using a pipe, the assignment to Error[run] is happening in a subshell, so that assignment disappears when the subshell exits.
Change:
{ cpio -ti --quiet <"$archive" 2>'/dev/null' || local -a Error[run]='cpio'; } | grep -Ei '$extlist'
to:
cpio -ti --quiet <"$archive" 2>'/dev/null' | grep -Ei "$extlist"
[[ ${PIPESTATUS[0]} -ne 0 ]] && Error[run]='cpio'
(btw, need double quotes in the grep part)