Using:
set -o nounset
Having an indexed array like:
myArray=( "red" "black" "blue" )
What is the shortest way to check if element 1 is set?
I sometimes use the following:
test "${#myArray[#]}" -gt "1" && echo "1 exists" || echo "1 doesn't exist"
I would like to know if there's a preferred one.
How to deal with non-consecutive indexes?
myArray=()
myArray[12]="red"
myArray[51]="black"
myArray[129]="blue"
How to quick check that 51 is already set for example?
How to deal with associative arrays?
declare -A myArray
myArray["key1"]="red"
myArray["key2"]="black"
myArray["key3"]="blue"
How to quick check that key2 is already used for example?
To check if the element is set (applies to both indexed and associative array)
[ "${array[key]+abc}" ] && echo "exists"
Basically what ${array[key]+abc} does is
if array[key] is set, return abc
if array[key] is not set, return nothing
References:
See Parameter Expansion in Bash manual and the little note
if the colon is omitted, the operator tests only for existence [of parameter]
This answer is actually adapted from the answers for this SO question: How to tell if a string is not defined in a bash shell script?
A wrapper function:
exists(){
if [ "$2" != in ]; then
echo "Incorrect usage."
echo "Correct usage: exists {key} in {array}"
return
fi
eval '[ ${'$3'[$1]+muahaha} ]'
}
For example
if ! exists key in array; then echo "No such array element"; fi
From man bash, conditional expressions:
-v varname
True if the shell variable varname is set (has been assigned a value).
example:
declare -A foo
foo[bar]="this is bar"
foo[baz]=""
if [[ -v "foo[bar]" ]] ; then
echo "foo[bar] is set"
fi
if [[ -v "foo[baz]" ]] ; then
echo "foo[baz] is set"
fi
if [[ -v "foo[quux]" ]] ; then
echo "foo[quux] is set"
fi
This will show that both foo[bar] and foo[baz] are set (even though the latter is set to an empty value) and foo[quux] is not.
New answer
From version 4.2 of bash (and newer), there is a new -v option to built-in test command.
From version 4.3, this test could address element of arrays.
array=([12]="red" [51]="black" [129]="blue")
for i in 10 12 30 {50..52} {128..131};do
if [ -v 'array[i]' ];then
echo "Variable 'array[$i]' is defined"
else
echo "Variable 'array[$i]' not exist"
fi
done
Variable 'array[10]' not exist
Variable 'array[12]' is defined
Variable 'array[30]' not exist
Variable 'array[50]' not exist
Variable 'array[51]' is defined
Variable 'array[52]' not exist
Variable 'array[128]' not exist
Variable 'array[129]' is defined
Variable 'array[130]' not exist
Variable 'array[131]' not exist
Note: regarding ssc's comment, I've single quoted 'array[i]' in -v test, in order to satisfy shellcheck's error SC2208. This seem not really required here, because there is no glob character in array[i], anyway...
This work with associative arrays in same way:
declare -A aArray=([foo]="bar" [bar]="baz" [baz]=$'Hello world\041')
for i in alpha bar baz dummy foo test;do
if [ -v 'aArray[$i]' ];then
echo "Variable 'aArray[$i]' is defined"
else
echo "Variable 'aArray[$i]' not exist"
fi
done
Variable 'aArray[alpha]' not exist
Variable 'aArray[bar]' is defined
Variable 'aArray[baz]' is defined
Variable 'aArray[dummy]' not exist
Variable 'aArray[foo]' is defined
Variable 'aArray[test]' not exist
With a little difference:In regular arrays, variable between brackets ([i]) is integer, so dollar symbol ($) is not required, but for associative array, as key is a word, $ is required ([$i])!
Old answer for bash prior to V4.2
Unfortunately, bash give no way to make difference betwen empty and undefined variable.
But there is some ways:
$ array=()
$ array[12]="red"
$ array[51]="black"
$ array[129]="blue"
$ echo ${array[#]}
red black blue
$ echo ${!array[#]}
12 51 129
$ echo "${#array[#]}"
3
$ printf "%s\n" ${!array[#]}|grep -q ^51$ && echo 51 exist
51 exist
$ printf "%s\n" ${!array[#]}|grep -q ^52$ && echo 52 exist
(give no answer)
And for associative array, you could use the same:
$ unset array
$ declare -A array
$ array["key1"]="red"
$ array["key2"]="black"
$ array["key3"]="blue"
$ echo ${array[#]}
blue black red
$ echo ${!array[#]}
key3 key2 key1
$ echo ${#array[#]}
3
$ set | grep ^array=
array=([key3]="blue" [key2]="black" [key1]="red" )
$ printf "%s\n" ${!array[#]}|grep -q ^key2$ && echo key2 exist || echo key2 not exist
key2 exist
$ printf "%s\n" ${!array[#]}|grep -q ^key5$ && echo key5 exist || echo key5 not exist
key5 not exist
You could do the job without the need of externals tools (no printf|grep as pure bash), and why not, build checkIfExist() as a new bash function:
$ checkIfExist() {
eval 'local keys=${!'$1'[#]}';
eval "case '$2' in
${keys// /|}) return 0 ;;
* ) return 1 ;;
esac";
}
$ checkIfExist array key2 && echo exist || echo don\'t
exist
$ checkIfExist array key5 && echo exist || echo don\'t
don't
or even create a new getIfExist bash function that return the desired value and exit with false result-code if desired value not exist:
$ getIfExist() {
eval 'local keys=${!'$1'[#]}';
eval "case '$2' in
${keys// /|}) echo \${$1[$2]};return 0 ;;
* ) return 1 ;;
esac";
}
$ getIfExist array key1
red
$ echo $?
0
$ # now with an empty defined value
$ array["key4"]=""
$ getIfExist array key4
$ echo $?
0
$ getIfExist array key5
$ echo $?
1
What about a -n test and the :- operator?
For example, this script:
#!/usr/bin/env bash
set -e
set -u
declare -A sample
sample["ABC"]=2
sample["DEF"]=3
if [[ -n "${sample['ABC']:-}" ]]; then
echo "ABC is set"
fi
if [[ -n "${sample['DEF']:-}" ]]; then
echo "DEF is set"
fi
if [[ -n "${sample['GHI']:-}" ]]; then
echo "GHI is set"
fi
Prints:
ABC is set
DEF is set
tested in bash 4.3.39(1)-release
declare -A fmap
fmap['foo']="boo"
key='foo'
# should echo foo is set to 'boo'
if [[ -z "${fmap[${key}]}" ]]; then echo "$key is unset in fmap"; else echo "${key} is set to '${fmap[${key}]}'"; fi
key='blah'
# should echo blah is unset in fmap
if [[ -z "${fmap[${key}]}" ]]; then echo "$key is unset in fmap"; else echo "${key} is set to '${fmap[${key}]}'"; fi
Reiterating this from Thamme:
[[ ${array[key]+Y} ]] && echo Y || echo N
This tests if the variable/array element exists, including if it is set to a null value. This works with a wider range of bash versions than -v and doesn't appear sensitive to things like set -u. If you see a "bad array subscript" using this method please post an example.
This is the easiest way I found for scripts.
<search> is the string you want to find, ASSOC_ARRAY the name of the variable holding your associative array.
Dependign on what you want to achieve:
key exists:
if grep -qe "<search>" <(echo "${!ASSOC_ARRAY[#]}"); then echo key is present; fi
key exists not:
if ! grep -qe "<search>" <(echo "${!ASSOC_ARRAY[#]}"); then echo key not present; fi
value exists:
if grep -qe "<search>" <(echo "${ASSOC_ARRAY[#]}"); then echo value is present; fi
value exists not:
if ! grep -qe "<search>" <(echo "${ASSOC_ARRAY[#]}"); then echo value not present; fi
I wrote a function to check if a key exists in an array in Bash:
# Check if array key exists
# Usage: array_key_exists $array_name $key
# Returns: 0 = key exists, 1 = key does NOT exist
function array_key_exists() {
local _array_name="$1"
local _key="$2"
local _cmd='echo ${!'$_array_name'[#]}'
local _array_keys=($(eval $_cmd))
local _key_exists=$(echo " ${_array_keys[#]} " | grep " $_key " &>/dev/null; echo $?)
[[ "$_key_exists" = "0" ]] && return 0 || return 1
}
Example
declare -A my_array
my_array['foo']="bar"
if [[ "$(array_key_exists 'my_array' 'foo'; echo $?)" = "0" ]]; then
echo "OK"
else
echo "ERROR"
fi
Tested with GNU bash, version 4.1.5(1)-release (i486-pc-linux-gnu)
For all time people, once and for all.
There's a "clean code" long way, and there is a shorter, more concise, bash centered way.
$1 = The index or key you are looking for.
$2 = The array / map passed in by reference.
function hasKey ()
{
local -r needle="${1:?}"
local -nr haystack=${2:?}
for key in "${!haystack[#]}"; do
if [[ $key == $needle ]] ;
return 0
fi
done
return 1
}
A linear search can be replaced by a binary search, which would perform better with larger data sets. Simply count and sort the keys first, then do a classic binary halving of of the haystack as you get closer and closer to the answer.
Now, for the purist out there that is like "No, I want the more performant version because I may have to deal with large arrays in bash," lets look at a more bash centered solution, but one that maintains clean code and the flexibility to deal with arrays or maps.
function hasKey ()
{
local -r needle="${1:?}"
local -nr haystack=${2:?}
[ -n ${haystack["$needle"]+found} ]
}
The line [ -n ${haystack["$needle"]+found} ]uses the ${parameter+word} form of bash variable expansion, not the ${parameter:+word} form, which attempts to test the value of a key, too, which is not the matter at hand.
Usage
local -A person=(firstname Anthony lastname Rutledge)
if hasMapKey "firstname" person; then
# Do something
fi
When not performing substring expansion, using the form described
below (e.g., ‘:-’), Bash tests for a parameter that is unset or null.
Omitting the colon results in a test only for a parameter that is
unset. Put another way, if the colon is included, the operator tests
for both parameter’s existence and that its value is not null; if the
colon is omitted, the operator tests only for existence.
${parameter:-word}
If parameter is unset or null, the expansion of word is substituted. Otherwise, the value of parameter is substituted.
${parameter:=word}
If parameter is unset or null, the expansion of word is assigned to parameter. The value of parameter is then substituted. Positional
parameters and special parameters may not be assigned to in this way.
${parameter:?word}
If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard
error and the shell, if it is not interactive, exits. Otherwise, the
value of parameter is substituted. ${parameter:+word}
If parameter is null or unset, nothing is substituted, otherwise the expansion of word is substituted.
https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Shell-Parameter-Expansion
If $needle does not exist expand to nothing, otherwise expand to the non-zero length string, "found". This will make the -n test succeed if the $needle in fact does exist (as I say "found"), and fail otherwise.
Both in the case of arrays and hash maps I find the easiest and more straightforward solution is to use the matching operator =~.
For arrays:
myArray=("red" "black" "blue")
if [[ " ${myArray[#]} " =~ " blue " ]]; then
echo "blue exists in myArray"
else
echo "blue does not exist in myArray"
fi
NOTE: The spaces around the array guarantee the first and last element can match. The spaces around the value guarantee an exact match.
For hash maps, it's actually the same solution since printing a hash map as a string gives you a list of its values.
declare -A myMap
myMap=(
["key1"]="red"
["key2"]="black"
["key3"]="blue"
)
if [[ " ${myMap[#]} " =~ " blue " ]]; then
echo "blue exists in myMap"
else
echo "blue does not exist in myMap"
fi
But what if you would like to check whether a key exists in a hash map? In the case you can use the ! operator which gives you the list of keys in a hash map.
if [[ " ${!myMap[#]} " =~ " key3 " ]]; then
echo "key3 exists in myMap"
else
echo "key3 does not exist in myMap"
fi
I get bad array subscript error when the key I'm checking is not set. So, I wrote a function that loops over the keys:
#!/usr/bin/env bash
declare -A helpList
function get_help(){
target="$1"
for key in "${!helpList[#]}";do
if [[ "$key" == "$target" ]];then
echo "${helpList["$target"]}"
return;
fi
done
}
targetValue="$(get_help command_name)"
if [[ -z "$targetvalue" ]];then
echo "command_name is not set"
fi
It echos the value when it is found & echos nothing when not found. All the other solutions I tried gave me that error.
Goal:
All I want to do is to check if a USER_PARAMETERS_KEYs[j] exists in RPM_PARAMETERS_HASH
associative array. I have one array and one associative arrays as the following:
Predefined:
declare -a USER_PARAMETERS_KEYS='([0]="userField" [1]="fUserField" [2]="srcIPField" [3]="srcPortField" [4]="dstIPField" [5]="dstPortField" [6]="dateField" [7]="timeField")'
declare -A RPM_PARAMETERS_HASH='([timeField]="Time" [userField]="User-Name" [dstIPField]="" [srcIPField]="Source-IP-Address" [dstPortField]="Target-UDP-Port" [fUserField]="Full-Name" [dateField]="Date" [srcPortField]="Source-UDP-Port" )'
I implemented the following:
if [[ ${RPM_PARAMETERS_HASH[${USER_PARAMETERS_KEYS[j]}]} ]] ; then
Problem
My problem is when ${USER_PARAMETERS_KEYS[j]} becomes equal to dstIPField since it has an empty string value in the associative array, so the if condition is not satisfied although the key is there.
With bash you can just use -v option to [[:
[[ -v RPM_PARAMETERS_HASH["${USER_PARAMETERS_KEYS[j]}"] ]]
#sailnfool - kudos to You for finding this solution. The "-v" solution doesn't work for me.
Regarding "+_" - in fact the "operator" is the "+" sign, as states in a "Shell Parameter Expansion" in the Bash Reference Manual:
"${parameter:+word}
If parameter is null or unset, nothing is substituted, otherwise the expansion of word is substituted."
And in the introduction: "Omitting the colon results in a test only for a parameter that is unset."
$ declare -A arr; arr["a"]="x"
$ echo ${arr[a]+_}
_
$ echo ${arr[a]+1}
1
$ echo "|${arr[c]+1}|"
||
$ arr["c"]=''
$ [[ ${arr[c]+1} ]] && echo "ok" || echo "nok"
ok
$ [[ ${arr[c]:+1} ]] && echo "ok" || echo "nok"
nok
Posting an answer here to add a bit of clarification to answers from #acan and #sailnfool. I found that I needed to run through some example scenarios to full grasp the nuances of the method they both describe (thanks to #sailnfool for the example script you created in your answer). At any rate, my comments here would not have been very legible if I simply replied to #sailnfool's post with a comment, so here goes FBO.
Clarifying the logic/outcomes of #sailnfool's script/answer:
If an array is defined, but a specific element (key) within the array has never been defined: both queries return a result of, "array element does not exist"
If an array element's value previously existed and has been unset: both queries return "array element does not exist"
If an array element's (key's) value is not NULL: both queries return "array element exists"
If an array element's value is NULL: 1st query returns "array element does exist"
If an array element's value is NULL: 2nd query returns "array element does not exist"
I couldn't quite tell from the original question whether the poster was attempting to derive whether or not a particular array element existed (i.e. whether the specific array element had been declared or not) versus whether or not a particular array element had a value (not null).
If the intent is to determine whether or not a specific array element has ever been set (including setting it equal to NULL), then use this method:
[[ ${arr[c]+1} ]] && echo "array key exists" || echo "array key does not exist"
For example:
declare -A arr
arr["c"]=""
[[ ${arr[c]+1} ]] && echo "array key exists" || echo "array key does not exist"
or
declare -A arr
arr["c"]=''
[[ ${arr[c]+1} ]] && echo "array key exists" || echo "array key does not exist"
If a user's intent is to detect when an array element existed and was subsequently unset, or when a specific array element has never been created, either of the two branching methods are going to return the same result. Namely, that the element does not exist. So, re-capping... the only difference between them is the ability of the first method (examples shown above) to branch when an array element exists but is set equal to NULL. While the 2nd branching method will only return a true path (array element exists) when the array element has been defined, and contains a non-null value.
Bottom Line
Your need scenario 1: tell when an array element is defined and = any value, including NULL
Use: 1st branch method
[[ ${arr[c]+1} ]] && echo "array key exists" || echo "array key does not exist"
Your need scenario 2: tell me when an array element is defined and contains a non-NULL value
Use: either logic branching method
[[ ${arr[c]+1} ]] && echo "array key exists" || echo "array key does not exist"
or
[[ ${arr[c]:+1} ]] && echo "array key exists" || echo "array key does not exist"
Your need scenario 3: tell me when an array element is not defined at all (i.e. never defined or was defined and subsequently unset)
Use: either logic branching method
[[ ${arr[c]+1} ]] && echo "array key exists" || echo "array key does not exist"
or
[[ ${arr[c]:+1} ]] && echo "array key exists" || echo "array key does not exist"
In the linuxhint article: "Associative Arrays Bash Examples"1 Under Example #5 it shows:
$ if [ ${ArrayName[searchKEY]+_} ]; then echo "Exists"; else echo "Not available"; fi
Note that the "searchKEY" could either be a literal value or a variable that expands to a literal value. Kudos to the author of the article that uncovered this "+_" operator on shell variables as a way to test for the key existence. I was unable to tease this out of the GNU BASH Reference Manual2. I am sure it is related to parameter expansion but I never realized that it would show associative (hash)
array member existence.
It looks like you can make it work by splitting up the steps (first evaluate the key, then use the -v test):
declare -a USER_PARAMETERS_KEYS='([0]="userField" [1]="fUserField" [2]="srcIPField" [3]="srcPortField" [4]="dstIPField" [5]="dstPortField" [6]="dateField" [7]="timeField")'
declare -A RPM_PARAMETERS_HASH='([timeField]="Time" [userField]="User-Name" [dstIPField]="" [srcIPField]="Source-IP-Address" [dstPortField]="Target-UDP-Port" [fUserField]="Full-Name" [dateField]="Date" [srcPortField]="Source-UDP-Port" )'
j=4
key=${USER_PARAMETERS_KEYS[j]}
[[ -v RPM_PARAMETERS_HASH[$key] ]] && echo yes # output: yes
I want to create a bash function that iterates over an array and returns 0 if an element passed as argument does not exist in the array, or 1 otherwise.
The following code however does not print anything on stdout.
function checkparsed {
tocheck="$1"
shift
for item in $#
do
if [ "$item" = "$tocheck" ]; then
return 0
fi
done
return 1
}
mdfiles=('foo')
echo "$(checkparsed foo ${mdfiles[#]})"
You are capturing the output of the function (there is none).
To print 0 or 1, either echo them in the function directly (don't forget to return), or use echo $? after running the function.
To handle spaces and glob characters in elements in ${mdfiles[#]}, you should use double quotes:
for item in "$#"
# and
checkparsed foo "${mdfiles[#]}"
This line is the issue:
echo "$(checkparsed foo ${mdfiles[#]})"
since your function is not echoing anything but you are returning values 0 or 1.
You actually need to check for $? for the return value from your function:
checkparsed foo ${mdfiles[#]}
echo $?
0
Or else use return value in condition evaluation:
checkparsed foo ${mdfiles[#]} && echo "found" || echo "not found"
found
checkparsed food ${mdfiles[#]} && echo "found" || echo "not found"
not found
I have a script that I'll be using in a system where the first three positional parameters are reserved, but I want to pass other parameters to the script to be used as variables. I can set default values for the variables, but if a parameter is included, I want that to take precedence. Here's a very basic script to illustrate:
#!/bin/bash
param1="$1"
param2="$2"
param3="$3"
param4="default4"
if [[ "$param4" == "" ]] && [[ "$4" == "" ]]; then
echo "A value was not specified for parameter 4. Script cannot execute. Exiting..."
exit 1
elif [[ "$4" != "" ]]; then
param4="$4"
else
echo "The built-in value for parameter 4 will be used in the script."
fi
echo "$param1"
echo "$param2"
echo "$param3"
echo "$param4"
If I run script a b c the output I get is:
The built-in value for parameter 4 will be used in the script.
a
b
c
default4
If I run script a b c d then the output is:
a
b
c
d
This is all well and good, but if I have 5-6 different parameters, I don't want to have to repeat the if..fi block that many times for each parameter.
So I've been trying to use a for loop to iterate through the parameters, but I haven't had much luck. The best attempt I came up with was:
#!/bin/bash
param1="$1"
param2="$2"
param3="$3"
param4="default4"
param5="default5"
param6="default6"
param7="default7"
defaultArray=( param4 param5 param6 param7 )
passedArray=( '$4' '$5' '$6' '$7' )
for (( i=0; i<${#defaultArray[#]}; i++ ));
do
if [[ "${defaultArray[$i]}" == "" ]] && [[ "${passedArray[$i]}" == "" ]]; then
echo "A value was not specified for parameter $((i+4)). Script cannot execute. Exiting..."
exit 1
elif [[ "${passedArray[$i]}" != "" ]]; then
eval "${defaultArray[$i]}"='"${passedArray[$i]}"'
else
echo "The built-in value for parameter $((i+4)) will be used in the script."
fi
done
echo "$param1"
echo "$param2"
echo "$param3"
echo "$param4"
echo "$param5"
echo "$param6"
echo "$param7"
But when I run this script a b c, or script a b c d, and whether or not I delete the default values in the script, I always get:
a
b
c
$4
$5
$6
$7
I think this because I needed to single-quote the positional parameter names in passedArray or else they just blank out, but then that hard-codes their names rather than the passed value.
Is there a way of doing this, or does trying to assign a positional parameter to a variable in an array and/or a for loop just not work? The thing I'm going to try next is a function, but I'm not sure that'd work any better...
Change
param4="default4"
if [[ "$param4" == "" ]] && [[ "$4" == "" ]]; then
echo "A value was not specified for parameter 4. Script cannot execute. Exiting..."
exit 1
elif [[ "$4" != "" ]]; then
param4="$4"
else
echo "The built-in value for parameter 4 will be used in the script."
fi
To:
if [[ ! -z $4 ]]; then
param4="$4"
else
param4="default4"
echo "The built-in value for parameter 4 will be used in the script."
fi
In your version of code, the first part of the if condition would never execute because param4 is already set just before if.
A more elegant and consiste way of doing this, given that you are using a default value for param4 anyway is:
param4=${4:-default4} # use default as "default4" unless $4 is set
For the second problem, your suspicion is right - single quotes won't expand the positional variables in the array assignment. So, change it to:
passedArray=( "$4" "$5" "$6" "$7" )
Your loops seems OK, except for this:
elif [[ "${passedArray[$i]}" != "" ]]; then
eval "${defaultArray[$i]}"='"${passedArray[$i]}"'
Why use eval here? You can say:
defaultArray[$i]=${passedArray[$i]}
I have no idea why the compound array initialization does not work for me.
minimal example:
#!/bin/bash
#
MINRADIUS=( 'foo' 'bar' 'foobar' )
for i in {0..2..1}; do echo ${MINRADIUS[$i]}; done
output is
$ sh test.sh
(foo bar foobar)
with 2 additional blank lines.
Fieldwise initialization works:
#!/bin/bash
#
MINRADIUS[0]="foo"
MINRADIUS[1]="bar"
MINRADIUS[2]="foobar"
for i in {0..2..1}; do echo ${MINRADIUS[$i]}; done
$ sh test.sh
foo
bar
foobar
I have tried every possible combination of braces, quotes and "declare -a".
Could it be related to my bash version? I'm running version 4.1.2(1).
The problem is, you are not using bash. Shebang doesn't matter if you run your script throught sh. Try bash instead.
I tried the below code and its working fine for me. Using bash 3.2.39(1)-release
#!/bin/bash
#
MINRADIUS=( 'foo' 'bar' 'foobar' )
for i in {0,1,2}; do echo ${MINRADIUS[$i]}; done
Output for this was
foo
bar
foobar
For me with your code it was giving an error
line 4: {0..1..2}: syntax error: operand expected (error token is "{0..1..2}")
I would suspect there is some quoting going on in your first example using compound assignments. Using this modified test script:
#!/bin/bash
echo "SHELL=${SHELL}"
echo 'Single-quoted v:'
v='(a b c)'; for i in {0..2}; do echo "v[$i]=${v[i]}"; done
echo 'Double-quoted v:'
v="(a b c)"; for i in {0..2}; do echo "v[$i]=${v[i]}"; done
echo 'Unquoted v:'
v=(a b c); for i in {0..2}; do echo "v[$i]=${v[i]}"; done
I get the following output:
$ sh test.sh
SHELL=/bin/bash
Single-quoted v:
v[0]=(a b c)
v[1]=
v[2]=
Double-quoted v:
v[0]=(a b c)
v[1]=
v[2]=
Unquoted v:
v[0]=a
v[1]=b
v[2]=c
If you quote the assignment, it becomes a simple variable assignment; a simple mistake to make that is easily overlooked.