i'm trying to write a script which lets me send commands to multiple services based on the passed params.
currently i'm declaring an associative array like
declare -A services0=(
[name]='service-a'
[port]='1234'
[type]='typa-x'
[...]='...'
)
declare -A services1=(
[name]='service-b'
[port]='1234'
[type]='typa-y'
[...]='...'
)
declare -A services?=(...
declare -n services
for a single param that works fine because then i do something like
for services in ${!services#}; do
if [[ "$PARAM_TYPE" == ${services[type]} ]]; then
do something...
fi
done
but now i want to add a bunch of new params.
my idea was to filter the associative array per param like
#step1 - filter associative array to objects where attribute-a is param-a if it is set
for services in ${!services#}; do
if [[ "$PARAM_TYPE" != ${services[type]} ]]; then
echo "Removing ${services[name]}"
unset -v 'services[$services]'
fi
#step2 - filter whats left after the previous step where attribute-b is param-b if it is set
#step3 - repeat for all possible params...
#step_last - iterate over the filtered array and execute commands
but i'm struggling removing the objects from the associative array i want to filter as they are not getting removed. when i execute
echo "services: ${!services#}"
output: 'services: services0 services1 services2 ...'
i get the same is if i've havn't filtered at all.
also when i iterate over the array again like
for services in ${!services#}; do
echo "executing commands for ${services[name]}";
done
i get this warning twice
warning: services : circular name reference
once for the echo line and then for the for line.
am i on the right track here? what am i missing to make this work or is this a completely wrong approach?
thanks, bernd
It seems you want to unset an entire array services3.
But your unset line says to unset an entire array services.
The fact that it says to unset some subkey of this variable is irrelevant, since it is not an array. So you may as well be writing
unset 'services'
which will unset the for-loop variable services. Which will then be set again on the next iteration of the for-loop.
What you want is
unset "$services"
which probably would have been clearer if the variable had been named properly. services is never multiple services, only one at a time. Which would make this a simple issue of uninterpreted string 'service' vs bash interpreted "$service" being fed to the unset builtin.
Related
I am creating an array from the output of a command and then i am looping through the array and running commands that use each item in the array.
Before i loop through the array i want to create a variable that uses one of the values in my array. I will use this value when one of the items in the array contains a specific string.
I am not sure how to pick the value i need to set the variable from my array before i then loop through the array. I currently have this which is not working for me. I have also tried looping through to get my value but the value does not follow to the next loop, i dont think its being set and i cant keep the loop open as i am looping inside my loop.
readarray -t ARRAY < <( command that gets array of 5 hostnames )
if [[ $ARRAY[#] == *"FT-01"* ]]; then
FTP="$ARRAY"
fi
for server in "${ARRAY[#]}"; do
echo "Server: ${srv}"
echo "-------------------"
if [[ $server == *"ER-01"* ]]; then
echo " FTP server is ${FTP} and this is ${server}"
fi
done
I'm pretty sure the first if statement would never work but i am at a loss to how to pick out the the value i need from the array.
Sometimes difficulty expressing an idea is a sign that you're thinking like a C programmer rather than a shell scripter. Arrays and for loops aren't the most natural idioms in shell scripts. Consider streaming and pipes instead.
Let's say the command that gets hostnames is called list-of-hostnames. If it prints one host name per line you can filter the results with grep.
FTP=$(list-of-hostnames | grep FT-01)
If you really do want to work with an array you could use printf '%s\n' to turn it into a grep-able stream.
FTP=$(printf '%s\n' "${ARRAY[#]}" | grep FT-01)
I want to check if the following records exist with two arrays. I'm not sure if this is the best way of going about it, but from the logic it looks like it may be possible from the code below:
Domain_checking () {
array=(
grafana
kibana
prometheus
alertmanager
)
array2=(
Name
NXDOMIAN
)
for index in ${!array[*]}; do
echo "checking that ${array[$index]} exists in the domain domain.co.uk"
DOMAIN_CHECK=$(nslookup ${array[$index]}.domain.co.uk | grep {array2[$index]})
if [[ $DOMAIN_CHECK == *'Name'* ]]; then
echo "The A record for ${array[$index]}.domain.co.uk exists"
elif [[ $DOMAIN_CHECK == *'NXDOMIAN'* ]]; then
echo "The A record for ${array[$index]}.domain.co.uk dose not exist"
fi
done
}
Domain_checking
When the code above is run, the loop does start and for the echo statement, I see the values in both arrays when I add {array2[$index]} to the echo statement.
But the array values are not present in DOMAIN_CHECK, which I'm not sure as to why this is as the for loop does iterate.
So I would expect that DOMAIN_CHECK should have some sort of value and hit the if statement but for some reason, this doesn't seem to be the case. Why is that?
It appears you're only using nslookup to see if the domain exists or not, rather than looking for specific information from the command. You can simplify by just checking the exit code instead of using grep:
Domain_checking () {
array=(
grafana
kibana
prometheus
alertmanager
)
for domain in ${array[#]}
do
if nslookup "${domain}.domain.co.uk" >/dev/null 2>&1 ; then
echo "$domain exists"
else
echo "$domain does not exist"
fi
done
}
Domain_checking
If the domain record exists, nslookup will return 0 and the if condition will be satisfied. Anything else indicates a failure and the control will fall through to the else.
You use $index as an index to both arrays, but there's a matching entry in $array2 only for the first entry. That's why the other entries aren't showing up, and also why grep is missing it's required argument.
Thinking through your logic, I don't see any reason not to remove the second array completely and hard code in Name for the grep.
Come think of it, the first array isn't helping much either. You could simplify the code by iterating across the names themselves rather than their array indices.
domain=some.thing
names="kibana prometheus graphite"
for name in $names; do
nslookup $name.$domain ....
done
I'm writing a bash script to analyse some files. In a first iteration I create associative arrays with word counts for each category that is analysed. These categories are not known in advance so the names of these associative arrays are variable but all with the same prefix count_$category. The associative arrays have a word as key and its occurrence count in that category as value.
After all files are analysed, I have to summarise the results for each category. I can iterate over the variable names using ${count_*} but how can I access the associative arrays behind those variable names? For each associative array (each count_* variable) I should iterate over the words and their counts.
I have already tried with indirect access like this but it doesn't work:
for categorycount in ${count_*} # categorycount now holds the name of the associative array variable for each category
do
array=${!categorycount}
for word in ${!array[#]}
do
echo "$word occurred ${array[$word]} times"
done
done
The modern (bash 4.3+) approach uses "namevars", a facility borrowed from ksh:
for _count_var in "${!count_#}"; do
declare -n count=$_count_var # make count an alias for $_count_var
for key in "${!count[#]}"; do # iterate over keys, via same
echo "$key occurred ${count[$key]} times" # extract value, likewise
done
unset -n count # clear that alias
done
declare -n count=$count_var allows "${count[foo]}" to be used to look up item foo in the associative array named count_var; similarly, count[foo]=bar will assign to that same item. unset -n count then removes this mapping.
Prior to bash 4.3:
for _count_var in "${!count_#}"; do
printf -v cmd '_count_keys=( "${!%q[#]}" )' "$_count_var" && eval "$cmd"
for key in "${_count_keys[#]}"; do
var="$_count_var[$key]"
echo "$key occurred ${!var} times"
done
done
Note the use of %q, rather than substituting a variable name directly into a string, to generate the command to eval. Even though in this case we're probably safe (because the set of possible variable names is restricted), following this practice reduces the amount of context that needs to be considered to determine whether an indirect expansion is secure.
In both cases, note that internal variables (_count_var, _count_keys, etc) use names that don't match the count_* pattern.
I'm trying build an sort of property set in ksh.
Thought the easiest way to do so was using arrays but the syntax is killing me.
What I want is to
Build an arbitrary sized array in a config file with a name and a property.
Iterate for each item in that list and get that property.
I theory what I wish I could do is something like
MONITORINGSYS={
SYS1={NAME="GENERATOR" MONITORFUNC="getGeneratorStatus"}
SYS2={NAME="COOLER" MONITORFUNC="getCoolerStatus"}
}
Then later on, be able to do something like:
for CURSYS in $MONITORINGSYS
do
CSYSNAME=$CURSYS.NAME
CSYSFUNC=$CURSYS.MONITORFUNC
REPORT="$REPORT\n$CSYSNAME"
CSYSSTATUS=CSYSFUNC $(date)
REPORT="$REPORT\t$CSYSSTATUS"
done
echo $REPORT
Well, that's not real programming, but I guess you got the point..
How do I do that?
[EDIT]
I do not mean I want to use associative arrays. I only put this way to make my question more clear... I.e. It would not be a problem if the loop was something like:
for CURSYS in $MONITORINGSYS
do
CSYSNAME=${CURSYS[0]}
CSYSFUNC=${CURSYS[1]}
REPORT="$REPORT\n$CSYSNAME"
CSYSSTATUS=CSYSFUNC $(date)
REPORT="$REPORT\t$CSYSSTATUS"
done
echo $REPORT
Same applies to the config file.. I'm just looking for a syntax that makes it minimally readable.
cheers
Not exactly sure what you want... Kornshell can handle both associative and indexed arrays.
However, Kornshell arrays are one dimensional. It might be possible to use indirection to emulate a two dimensional array via the use of $() and eval. I did this a couple of times in the older Perl 4.x and Perl 3.x, but it's a pain. If you want multidimensional arrays, use Python or Perl.
The only thing is that you must declare arrays via the typedef command:
$ typeset -A foohash #foohash is an associative array
$ typeset -a foolist #foolist is an integer indexed array.
Maybe your script can look something like this
typeset -a sysname
typeset -a sysfunct
sysname[1] = "GENERATOR"
sysname[2] = "COOLER"
sysfunc[1] = "getGeneratorStatus"
sysfunc[2] = "getCoolerStatus"
for CURSYS in {1..2}
do
CSYSNAME="${sysname[$CURSYS]}"
CSYSFUNC="${sysfunc[$CURSYS]}"
REPORT="$REPORT\n$CSYSNAME"
CSYSSTATUS=$(eval "CSYSFUNC $(date)")
REPORT="$REPORT\t$CSYSSTATUS"
done
echo $REPORT
ksh93 now has compound variables which can contain a mixture of indexed and associative arrays. No need to declare it as ksh will work it out itself.
#!/bin/ksh
MONITORINGSYS=(
[SYS1]=(NAME="GENERATOR" MONITORFUNC="getGeneratorStatus")
[SYS2]=(NAME="COOLER" MONITORFUNC="getCoolerStatus")
)
echo MONITORING REPORT
echo "-----------------"
for sys in ${!MONITORINGSYS[*]}; do
echo "System: $sys"
echo "Name: ${MONITORINGSYS[$sys].NAME}"
echo "Generator: ${MONITORINGSYS[$sys].MONITORFUNC}"
echo
done
Output:
MONITORING REPORT
-----------------
System: SYS1
Name: GENERATOR
Generator: getGeneratorStatus
System: SYS2
Name: COOLER
Generator: getCoolerStatus
I'm trying to write a bash script that will let me download multiple web pages using curl. For each webpage, I want to be able to pass curl the page and the referer link. I want to be able to supply multiple webpages at once.
In other words, I want to be able to loop through the webpages I supply the script, and for each page, pass the associated webpage and referer link to curl.
I thought I'd use an array to store the webpage and referer link in a single variable, thinking that I could then extract the individual elements of the array when running curl.
My problem is that I can't figure out how to get multiple arrays to work properly in a for loop. Here is an idea of what I want to do. This code does not work, since "$i" (in the for loop) doesn't become an array.
#every array has the information for a separate webpage
array=( "webpage" "referer" )
array2=( "another webpage" "another referer" )
for i in "${array[#]}" "${array2[#]}" #line up multiple web pages
do
#use curl to download the page, giving the referer ("-e")
curl -O -e "${i[1]}" "${i[0]}"
done
If I was only working with one array, I could easily do it like this:
array=( "webpage" "referer" )
REFERER="${array[1]}"
PAGE="${array[0]}"
#use curl to download the page, giving the referer ("-e")
curl -O -e "$REFERER" "$LINK"
It's once I have more than one webpage that I want to process at once that I can't figure out how to do it correctly.
If there is another way to handle multiple webpages, without having to use arrays and a for loop, please let me know.
If there is another way to handle multiple webpages, without having to use arrays and a for loop, please let me know.
Using arrays is fine, at least it's much better than using space-separated lists or similar hacks. Simply loop over the indices:
array=('webpage' 'another webpage')
array2=('referrer' 'another referrer')
# note the different layout!
for i in "${!array[#]}"
do
webpage="${array[$i]}"
referrer="${array2[$i]}"
done
You need a trick here. Note that spaces are not allowed in URLs, so you can say:
webpages=("url referrer" "url2 ref2" ...)
for i in "${webpages[#]}" ; do
set -- "$i"
url="$1"
ref="$2"
curl -O -e "${url}" "${ref}"
done
[EDIT] Maybe a better solution will be to put all the URLs into a file and then use this code:
while read url ref ; do
curl -O -e "${url}" "${ref}"
done < file
or if you prefer here documents:
while read url ref ; do
echo "url=$url ref=$ref"
done <<EOF
url1 ref1
url2 ref2
... xxx
EOF
Just as a general aside: Inside a function at least just declare the IFS variable to limit its scope to that function only. No need to save & restore IFS via OLD_IFS!
help declare
IFS=$' \t\n'
printf "%q\n" "$IFS"
function ifs_test () {
declare IFS
IFS=$'\n'
printf "%q\n" "$IFS"
return 0
}
ifs_test
printf "%q\n" "$IFS"
Thanks to everyone for their responses. Both ideas had merit, but I found some code in the Advanced Bash Guide that does exactly what I want to do.
I can't say I fully understand it, but by using an indirect reference to the array, I can use multiple arrays in the for loop. I'm not sure what the local command does, but it is the key (I think it runs a sort of eval and assigns the string to the variable).
The advantage of this is that I can group each webpage and referer into their own array. I can then easily add a new website, by creating a new array and adding it to the for loop. Also, should I need to add more variables to the curl command (such as a cookie), I can easily expand the array.
function get_page () {
OLD_IFS="$IFS"
IFS=$'\n' # If the element has spaces, when using
# local to assign variables
local ${!1}
# Print variable
echo First Variable: "\"$a\""
echo Second Variable: "\"$b\""
echo ---------------
echo curl -O -e "\"$a\"" "\"$b\""
echo
IFS="$OLD_IFS"
}
#notice the addition of "a=" and "b="
#this is not an associative array, that would be [a]= and [b]=
array=( a="webpage" b="referer" )
array2=( a="another webpage" b="another referer" )
#This is just a regular string in the for loop, it doesn't mean anything
#until the indirect referencing later
for i in "array[*]" "array2[*]" #line up multiple web pages
do
#must use a function so that the local command works
#but I'm sure there's a way to do the same thing without using local
get_page "$i"
done
This results in:
First Variable: "webpage"
Second Variable: "referer"
---------------
curl -O -e "webpage" "referer"
First Variable: "another webpage"
Second Variable: "another referer"
---------------
curl -O -e "another webpage" "another referer"