I'm having problems getting a variable "${Error[*]}", which is a regular indexed array, to stay set from the time it's declared until it's checked. It seems to me that a sub-shell must be launched so the declaration doesn't stick. I didn't think sub-shells were opened when using braces { stuff...; }. I want to know how to get my variable, Error to stick in the case I'm trying to write up. Here's a sample of my script:
TestFunction () {
unset Error
local archive="$1" extlist="$2" && local ext="${archive##*.}"
shopt -s nocasematch
local -i run=0
while [[ "$run" == 0 || -n "${Error[run]}" ]]; do
(( run++ ))
local IFS=$'\n\r\t '
if [[ ! "${Error[*]}" =~ 'cpio' && "$ext" =~ ^(pax|cpio|cpgz|igz|ipa|cab)$ && -n "$(which 'cpio')" ]]; then
## Try to cpio the archive. Since cpio cannot handle '.cab' archive, I want to declare an Error ##
{ cpio -ti --quiet <"$archive" 2>'/dev/null' || local -a Error[run]='cpio'; } | grep -Ei '$extlist'
elif [[ ! "${Error[*]}" =~ 'zipinfo' && "$ext" =~ ^(zip|[jw]ar|ipa|cab)$ && -n "$(which 'unzip')" ]]; then
## If cpio fails, then try zipinfo, or unzip on the next run through the loop... ##
if which 'zipinfo' &>'/dev/null'; then
{ zipinfo -1 "$archive" 2>'/dev/null' || local -a Error[run]='zipinfo'; } | grep -Ei "$scanlist"
elif which 'unzip' &>'/dev/null'; then
{ unzip -lqq "$archive" 2>'/dev/null' || local -a Error[run]='unzip'; } | gsed -re '/^ +[0-9]+/!d;s|^ +[0-9]+ +[0-9-]+ [0-9:]+ +||' | grep -Ei "$exlist"
fi
## many more elifs... ##
fi
done
shopt -u nocasematch
return 0
}
Archives='\.(gnutar|7-zip|lharc|toast|7zip|boz|bzi?p2?|cpgz|cpio|gtar|g?z(ip)?|lzma(86)?|t[bg]z2?|ar[cgjk]|bz[2a]?|cb[7rz]|cdr|deb|[dt]lz|dmg|exe|fbz|fgz|gz[aip]|igz|img|iso|lh[az]|lz[hmswx]?|mgz|mpv|mpz|pax|piz|pka|[jrtwx]ar|rpm|s?7-?z|sitx?|m?pkg|sfx|nz|xz)$'
IFS=$'\n'
declare -a List=($(TestFunction '/Users/aesthir/Programming/│My Projects│/Swipe Master/Test Folder/SDKSetup.cab' "$Archives"))
IFS=$' \t\n'
xtrace output:
〔xtrace〕 unset Error
〔xtrace〕 local 'archive=/Users/aesthir/Programming/│My Projects│/Swipe Master/Test Folder/SDKSetup.cab' 'extlist=\.(gnutar|7-zip|lharc|toast|7zip|boz|bzi?p2?|cpgz|cpio|gtar|g?z(ip)?|lzma(86)?|t[bg]z2?|ar[cgjk]|bz[2a]?|cb[7rz]|cdr|deb|[dt]lz|dmg|exe|fbz|fgz|gz[aip]|igz|img|iso|lh[az]|lz[hmswx]?|mgz|mpv|mpz|pax|piz|pka|[jrtwx]ar|rpm|s?7-?z|sitx?|m?pkg|sfx|nz|xz)$'
〔xtrace〕 local ext=cab
〔xtrace〕 shopt -s nocasematch
〔xtrace〕 local -i run=0
〔xtrace〕 [[ 0 == 0 ]]
〔xtrace〕 (( run++ ))
〔xtrace〕 local 'IFS=
'
〔xtrace〕 [[ ! '' =~ cpio ]]
〔xtrace〕 [[ cab =~ ^(pax|cpio|cpgz|igz|ipa|cab)$ ]]
〔xtrace〕 which cpio
〔xtrace〕 [[ -n /usr/bin/cpio ]]
〔xtrace〕 grep -Ei '$extlist'
〔xtrace〕 cpio -ti --quiet
〔xtrace〕 local -a 'Error[run]=cpio'
〔xtrace〕 [[ 1 == 0 ]]
〔xtrace〕 [[ -n '' ]] ## <—— Problem is here... when checking "${Error[run]}", it's unset ##
〔xtrace〕 shopt -u nocasematch
〔xtrace〕 return 0
Now obviously I know cpio, zipinfo, and unzip cannot handle cab files... I put 'cab' in the extension list on purpose to cause an error.
I want to stay in TestFunction and keep looping with different archivers until a success (file list is dumped, which cabextract would gladly do in this case) without repeating an already failed archiver.
Finally, since this works fine...
TestFunction () {
unset Error
local archive="$1" extlist="$2" && local ext="${archive##*.}"
local -i run=0
while [[ "$run" == 0 || -n "${Error[run]}" ]]; do
(( run++ ))
local IFS=$'\n\r\t '
if [[ ! "${Error[*]}" =~ 'cpio' && "$ext" =~ ^(pax|cpio|cpgz|igz|ipa|cab)$ && -n "$(which 'cpio')" ]]; then
cpio -ti --quiet <"$archive" 2>'/dev/null' || local -a Error[run]='cpio'
fi
done
shopt -u nocasematch
return 0
}
... I have to assume the problem is the braces because I want the results grep'd right away. However, I need those braces there because I don't want Error[run] to be set if grep turns up no results, only if cpio fails. I dont want to grep outside TestFunction for other reasons (I would have to do a complete re-write).
Any quick solution to this without massive rewriting? Maybe echo 'cpio' to some fd and read -u6ing it somehow?
I'd much prefer not to have to set an array to the file list and then for loop | grep through every file as it would really slow things down.
The problem is not the braces, but the pipe. Because you're using a pipe, the assignment to Error[run] is happening in a subshell, so that assignment disappears when the subshell exits.
Change:
{ cpio -ti --quiet <"$archive" 2>'/dev/null' || local -a Error[run]='cpio'; } | grep -Ei '$extlist'
to:
cpio -ti --quiet <"$archive" 2>'/dev/null' | grep -Ei "$extlist"
[[ ${PIPESTATUS[0]} -ne 0 ]] && Error[run]='cpio'
(btw, need double quotes in the grep part)
Related
I have written a very simple port scanning script in bash. For my ports argument, i accept a single port (eg: 443), a comma-separated list (eg: 80,443), or a range (eg: 1-1000).
When I run my script with a single port or comma-separated list of ports, everything runs as it should:
~/projects/bash-port-scan# ./bash-port-scan.sh -i xx.xx.xxx.xxx -p 1,80,443 -v
Beginning scan of xx.xx.xxx.xxx
Port 1 closed
Port 80 open
Port 443 open
Scan complete.
~/projects/bash-port-scan# ./bash-port-scan.sh -i xx.xx.xxx.xxx -p 80 -v
Beginning scan of xx.xx.xxx.xxx
Port 80 open
Scan complete.
However, when I run with a range, I get:
~/projects/bash-port-scan# ./bash-port-scan.sh -i xx.xx.xxx.xxx -p 1-10 -v
Beginning scan of xx.xx.xxx.xxx
Port 1
2
3
4
5
6
7
8
9
10 closed
Scan complete.
Relevant code where I assign the array:
portarray=()
if [[ "$ports" == *","* ]]; then
IFS=','
read -r -a portarray <<< $ports
IFS=' '
elif [[ "$ports" == *"-"* ]]; then
IFS='-'
read -r -a range <<< $ports
IFS=' '
first="${range[0]}"
last="${range[1]}"
portarray=($(seq $first 1 $last))
else
portarray=($ports)
fi
and the loop itself:
empty=""
for p in "${portarray[#]}"; do
result=$(nc -zvw5 $ip $p 2>&1 | grep open)
if [ "$result" = "$empty" ]; then
if [ $verbose -eq 1 ]; then
str="Port "
closed=" closed"
echo "$str$p$closed"
fi
else
str="Port "
closed=" open"
echo "$str$p$closed"
fi
done
I'm not sure if this is because of how I'm assigning my port array, or if it is because of something I have wrong in my loop. I'm relatively new to bash scripting, and I'm having a terrible time figuring out what I have wrong.
I've read here on SO about some commands run in loops eating the output of other portions of the script, but I don't believe that to be the case here, as the script does actually print to screen, just not as expected.
EDIT:
Here is the full script:
#!/bin/bash
verbose=0
while [ "$1" != "" ]; do
case "$1" in
-h | --help )
echo "bash-port-scan.sh v0.1\r\nUsage: ./bash-port-scan.sh -i 127.0.0.1 -p 80,443\r\n./bash-port-scan.sh -i 127.0.0.1 -p 1-1000"; shift;;
-v | --verbose )
verbose=1; shift;;
-i | --ip )
ip="$2"; shift;;
-p | --ports )
ports="$2"; shift;;
esac
shift
done
if [[ $ip = "" ]]; then
echo "Please enter an IP address with -i"
exit
fi
if [[ $ports = "" ]]; then
echo "Please enter the port(s) with -p"
exit
fi
portarray=()
if [[ "$ports" == *","* ]]; then
IFS=','
read -r -a portarray <<< $ports
IFS=' '
elif [[ "$ports" == *"-"* ]]; then
IFS='-'
read -r -a range <<< $ports
IFS=' '
first="${range[0]}"
last="${range[1]}"
portarray=($(seq $first $last))
else
portarray=($ports)
fi
if [ $verbose -eq 1 ]; then
echo "Beginning scan of $ip"
fi
shuf -e "${portarray[#]}"
empty=""
for p in "${portarray[#]}"; do
result=$(nc -zvw5 $ip $p 2>&1 | grep open)
if [ "$result" = "$empty" ]; then
if [ $verbose -eq 1 ]; then
str="Port "
closed=" closed"
echo "$str$p$closed"
fi
else
str="Port "
closed=" open"
echo "$str$p$closed"
fi
done
echo "Scan complete."
Addressing just the portarray=(...) assignment (when ports=1-10)
Consider:
$ first=1
$ last=10
$ portarray=($(seq $first 1 $last))
$ typeset -p portarray
declare -a portarray=([0]=$'1\n2\n3\n4\n5\n6\n7\n8\n9\n10')
NOTE: the output from the $(seq ...) call is processed as a single string with embedded linefeeds.
A couple ideas:
### define \n as field separator; apply custom IFS in same line to limit IFS change to just the follow-on array assignment:
$ IFS=$'\n' portarray=($(seq $first 1 $last))
$ typeset -p portarray
declare -a portarray=([0]="1" [1]="2" [2]="3" [3]="4" [4]="5" [5]="6" [6]="7" [7]="8" [8]="9" [9]="10")
### use mapfile to read each *line* into a separate array entry:
$ mapfile -t portarray < <(seq $first 1 $last)
$ typeset -p portarray
declare -a portarray=([0]="1" [1]="2" [2]="3" [3]="4" [4]="5" [5]="6" [6]="7" [7]="8" [8]="9" [9]="10")
I am trying to convert the following script which I use to create archives into one which extracts them.
[[ $# -lt 2 ]] && exit 1
name=$1; shift
files=("$#")
#exclude all files/directories that are not readable
for index in "${!files[#]}"; do
[[ -r ${files[index]} ]] || unset "files[index]"
done
[[ ${#files[#]} -eq 0 ]] && exit 1
if tar -czvf "${name:-def_$$}.tar.gz" "${files[#]}"; then
echo "Ok"
else
echo "Error"
exit 1
fi
So far I have this:
[[ $# -lt 1 ]] && exit 1
files=("$#")
#remove files and directories which are not readable
for index in "${!files[#]}"; do
[[ -r ${files[index]} ]] || unset "files[index]"
done
[[ ${#files[#]} -eq 0 ]] && exit 1
if tar -xzvf "${files[#]}".tar.gz; then
echo "OK"
else
echo "Error"
exit 1
fi
I dont know whether I needed to keep the shift as for this script I do not need to discard any arguments. I want to be able to take them all and unzip each one. Also I see there is a -C switch which allows the user to choose where the unzipped files go. How would I go about also adding this as an option for the user because they may or may not want to change the directory where the files get unzipped to.
You unfortunately can't just do tar -xzvf one.tar.gz two.tar.gz. Straightforward approach is to use a good old for loop:
for file in "${files[#]}"; do
tar -xzvf "$file"
done
Or you can use this:
cat "${files[#]}" | tar -xzvf - -i
You can have the first argument to be the specified directory for the -C option:
[[ $# -lt 2 ]] && exit 1
target=$1; shift
files=("$#")
#remove files and directories which are not readable
for index in "${!files[#]}"; do
[[ -r ${files[index]} ]] || unset "files[index]"
done
[[ ${#files[#]} -eq 0 ]] && exit 1
mkdir -p -- "$target" || exit 1
for file in "${files[#]}"; do
tar -xzvf "$file" -C "$target"
done
./script /some/path one.tar.gz two.tar.gz
List of files for tar can be also constructed like this:
target=$1; shift
for file; do
[[ -r $file ]] && files+=("$file")
done
This script should alert me in case of an error response.
Issue: Even when it executes successfully I am getting the email.
Bash script
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
IFS=\" read __ KEY __ MESSAGE __ <<< "$DATA"
if [[ $KEY == Success ]]; then
echo something
else
send email on failure
fi
Response on
Failure: {"ErrorCode":"11","ErrorMessage":"random message as per error code"}
Sucess: {"ErrorCode":"000","ErrorMessage":"Success"}
This worked finally -
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
MESSAGE=$(jq '.ErrorMessage' <<< "$DATA")
if [[ "$MESSAGE" == '"Success"' ]] ; then
echo something
else
send email
fi
A proper tool to address your issue would be jq :
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
KEY=$(jq -r '.ErrorCode' <<< "$DATA")
MESSAGE=$(jq -r '.ErrorMessage' <<< "$DATA")
if [[ "$KEY" = "000" ]]
then
echo success
else
echo fail
fi
Note : the -r flag for jq removes the double quotes
#!/bin/bash
DATA=$(wget --timeout 5 -O - -q -t 1 http://this.url/?parm=1\&par=2)
KEY=$(echo "$DATA" | grep -oP '"ErrorCode":"\K(\d+)"')
MESSAGE=$(echo "$DATA" | grep -oP '"ErrorMessage":"\K(.+?)(?=")')
((KEY == 0)) && echo success || echo "$MESSAGE"
In a bash script, I have a locale in a variable like so
locale=fr_ma
I also have an associative array like this
declare -A new_loc_map
new_loc[fr_ma]=en_ma
new_loc[el_gr]=en_gr
new_loc[sl_si]=en_si
I want to check if new_loc element with the key ${locale} exists
I thought this should work but it doesn't:
if [[ -v "${new_loc[${locale}]}" ]]
then
echo -e "${locale} has a new_loc"
fi
fi
any ideas on how otherwise I can test for this?
For older verions of bash (looks like [[ -v array[index] ]] was introduced in version 4.3), you can use the ${var-word} form to test is a variable has been set:
$ zz="$RANDOM$RANDOM$RANDOM"
$ echo $zz
270502100415054
$ declare -a name
$ locale=foo
$ [[ ${name[$locale]-$zz} = "$zz" ]] && echo var is unset || echo var has a value
var is unset
$ name[$locale]=""
$ [[ ${name[$locale]-$zz} = "$zz" ]] && echo var is unset || echo var has a value
var has a value
$ [[ ${name[$locale]:-$zz} = "$zz" ]] && echo var is unset or empty || echo var has a value
var is unset or empty
The tricky part is devising a $zz string that won't appear as actual data in your array.
Much better suggestion from #chepner:
if [[ -z "${name[$locale]+unset}" ]]; then
echo "no name for $locale"
else
echo "name for $locale is ${name[$locale]}"
fi
-v takes an (indexed) name as its argument, since you are trying to determine if the expansion makes sense in the first place.
if [[ -v new_loc[$locale] ]]; then
echo "Locale ${locale} now maps to ${new_loc[$locale]}"
fi
Word of warning
While the BASH manual page describes -v for [[ and test, reliable results are returned from [[ only.
Consider this (Bash 4.4):
> [ -v "$a[1]" ] && echo true
> a[1]=''
> [ -v "$a[1]" ] && echo true
> declare -p a
declare -a a=([1]="")
> [ -v $a[1] ] && echo true
> [[ -v $a[1] ]] && echo true
> [[ -v a[1] ]] && echo true
true
> [[ -v a[0] ]] && echo true
>
I managed to solve the problem by checking if the variable is not an empty string instead.
Example:
locale=fr_ma
declare -A new_loc
new_loc[fr_ma]=en_ma
new_loc[el_gr]=en_gr
if [[ ! -z ${new_loc[$locale]} ]]; then
echo "Locale ${locale} now maps to ${new_loc[$locale]}"
fi
Output:
Locale fr_ma now maps to en_ma
In practicing bash, I tried writing a script that searches the home directory for duplicate files in the home directory and deletes them. Here's what my script looks like now.
#!/bin/bash
# create-list: create a list of regular files in a directory
declare -A arr1 sumray origray
if [[ -d "$HOME/$1" && -n "$1" ]]; then
echo "$1 is a directory"
else
echo "Usage: create-list Directory | options" >&2
exit 1
fi
for i in $HOME/$1/*; do
[[ -f $i ]] || continue
arr1[$i]="$i"
done
for i in "${arr1[#]}"; do
Name=$(sed 's/[][?*]/\\&/g' <<< "$i")
dupe=$(find ~ -name "${Name##*/}" ! -wholename "$Name")
if [[ $(find ~ -name "${Name##*/}" ! -wholename "$Name") ]]; then
mapfile -t sumray["$i"] < <(find ~ -name "${Name##*/}" ! -wholename "$Name")
origray[$i]=$(md5sum "$i" | cut -c 1-32)
fi
done
for i in "${!sumray[#]}"; do
poten=$(md5sum "$i" | cut -c 1-32)
for i in "${!origray[#]}"; do
if [[ "$poten" = "${origray[$i]}" ]]; then
echo "${sumray[$i]} is a duplicate of $i"
fi
done
done
Originally, where mapfile -t sumray["$i"] < <(find ~ -name "${Name##*/}" ! -wholename "$Name") is now, my line was the following:
sumray["$i"]=$(find ~ -name "${Name##*/}" ! -wholename "$Name")
This saved the output of find to the array. But I had an issue. If a single file had multiple duplicates, then all locations found by find would be saved to a single value. I figured I could use the mapfile command to fix this, but now it's not saving anything to my array at all. Does it have to do with the fact that I'm using an associative array? Or did I just mess up elsewhere?
I'm not sure if I'm allowed to answer my own question, but I figured that I should post how I solved my problem.
As it turns out, the mapfile command does not work on associative arrays at all. So my fix was to save the output of find to a text file and then store that information in an indexed array. I tested this a few times and I haven't seemed to encounter any errors yet.
Here's my finished script.
#!/bin/bash
# create-list: create a list of regular files in a directory
declare -A arr1 origray
declare indexray
#Verify that Parameter is a directory.
if [[ -d "$HOME/$1/" && -n "$1" ]]; then
echo "Searching for duplicates of files in $1"
else
echo "Usage: create-list Directory | options" >&2
exit 1
fi
#create list of files in specified directory
for i in $HOME/${1%/}/*; do
[[ -f $i ]] || continue
arr1[$i]="$i"
done
#search for all duplicate files in the home directory
#by name
#find checksum of files in specified directory
for i in "${arr1[#]}"; do
Name=$(sed 's/[][?*]/\\&/g' <<< "$i")
if [[ $(find ~ -name "${Name##*/}" ! -wholename "$Name") ]]; then
find ~ -name "${Name##*/}" ! -wholename "$Name" >> temp.txt
origray[$i]=$(md5sum "$i" | cut -c 1-32)
fi
done
#create list of duplicate file locations.
if [[ -f temp.txt ]]; then
mapfile -t indexray < temp.txt
else
echo "No duplicates were found."
exit 0
fi
#compare similarly named files by checksum and delete duplicates
count=0
for i in "${!indexray[#]}"; do
poten=$(md5sum "${indexray[$i]}" | cut -c 1-32)
for i in "${!origray[#]}"; do
if [[ "$poten" = "${origray[$i]}" ]]; then
echo "${indexray[$count]} is a duplicate of a file in $1."
fi
done
count=$((count+1))
done
rm temp.txt
This is kind of sloppy but it does what it's supposed to do. md5sum may not be the optimal way to check for file duplicates but it works. All I have to do is replace echo "${indexray[$count]} is a duplicate of a file in $1." with rm -i ${indexray[$count]} and it's good to go.
So my next question would have to be...why doesn't mapfile work with associative arrays?