Bash : Use a variable as an associative array name - arrays

I'm writing a Bash script to simplify file copies from our main site to multiple agencies.
In this script, I'm trying to use a variable as an associative array name but I optain an error, here is the code :
#!/bin/bash
declare -A GROUP1
declare -A GROUP2
declare -A GROUP3
declare -A ARRAY
GROUP1["SITE1"]="x.x.x.x"
GROUP1["SITE2"]="y.y.y.y"
GROUP1["SITE3"]="z.z.z.z"
GROUP2["SITE1"]="1.1.1.1"
GROUP2["SITE2"]="2.2.2.2"
GROUP2["SITE3"]="3.3.3.3"
GROUP2["SITE1"]="a.a.a.a"
GROUP2["SITE2"]="b.b.b.b"
GROUP2["SITE3"]="c.c.c.c"
read -p "Choose a group of sites : " group
case $group in
1 ) DEST="GROUP1" ;;
2 ) DEST="GROUP2" ;;
3 ) DEST="GROUP3" ;;
esac
eval "ARRAY=(\${$DEST[#]})"
for elem in "${!ARRAY[#]}"
do
echo $elem
echo ${ARRAY[$elem]}
done
Here is the error :
./test: line28: TAB : 3.3.3.3 : must use subscript when assigning associative array
./test: line28: TAB : 2.2.2.2 : must use subscript when assigning associative array
./test: line28: TAB : 1.1.1.1 : must use subscript when assigning associative array
Is what I am trying to do possible ?
Thanks in advance.

It's possible but it's not easy unless you have bash v4.3. With 4.3, you can use a "nameref":
declare -A GROUP1
declare -A GROUP2
declare -A GROUP3
GROUP1["SITE1"]="x.x.x.x"
#...
# Yuk. Use command-line options, not stdin
read -p "Choose a group of sites : " group
for g in GROUP1 GROUP2 GROUP3; do if [[ $group == $g ]]; then
# Here's the nameref: After this executes, ARRAY is an alias for
# the indicated variable.
declare -n ARRAY=$group
break
fi
### This is documented but might not work depending on bash version
### Instead you could test for existence of a known key:
### if [[ ! -v ${ARRAY[SITE1]} ]]; then
if [[ ! -R ARRAY ]]; then
echo "No such group" >> /dev/stderr; exit 1
fi
OK, you probably don't yet have bash 4.3 but in the future the above will be useful to someone. So without it, you could follow a strategy like the one which you propose, which is to copy the indicated associative array. That's not too bad if the arrays aren't big. To do that, use the above but replace the nameref line (declare -n ARRAY=$group) with the following:
defn=$(declare -p $group)
eval "${defn/$group/ARRAY}"
unset defn
In this case, you'll have the use the alternative test for success.
Note: It's really not a good idea to use all caps for bash variable names. The convention is that system-generated environment variables and shell builtin variables use all caps. User variables should be lower-cased in order to not collide with these predefined variables.

Related

How to replace an array variable of one shell script from another shell script's array variable?

I have two shell scripts, fruits_original.sh and appending_fruits.sh. In the fruits_original.sh I have one array variable: fruits=('Apple' 'Mango' 'Guava').
What I want to do is I have to write a shell script appending_fruits.sh that will take an argument some new fruits name is Orange and will append that new fruit name to the fruits_original.sh fruits array variable.
After script run fruits array should be remain an array only and its value should be fruits=('Apple' 'Mango' 'Guava' 'Orange').
The file fruits_original.sh has this. Below is the appending_fruits.sh script by this my variable is changing into this fruits= ('Apple' 'Mango' 'Guava' 'Orange'). But when I am trying to do echo "${fruits[#]}" I am getting this error:
line 1: syntax error near unexpected token `('
Any luck ?
fruits= ('Apple' 'Mango' 'Guava')
echo "${fruits[#]}"
declare -a var=$(awk -F'=' '/^fruits=/ {print $2}' fruits_original.sh)
echo "${var[#]}"
var[${#var[#]}]='Orange'
joined=$(printf " '%s'" "${var[#]}")
echo ${joined:1}
echo "${joined[#]}"
sed -i "s/fruits=.*/fruits= ($( echo ${joined:1})) /" fruits_original.sh
Do not modify the script file. Instead, create another file and source the dynamic data from it. I have chosen the location of configuration to be in /tmp directory.
# fruits_original.sh
fruits=()
if [[ -e /tmp/fruits_original.rc ]]; then
. /tmp/fruits_original.rc
fi
some stuff
Then generate the config file. Use declare -p to safely output properly quoted variables.
# appending_fruits.sh
fruits=()
if [[ -e /tmp/fruits_original.rc ]]; then
. /tmp/fruits_original.rc
fi
fruits+=("new fruit")
decalre -p fruits > /tmp/fruits_original.rc
Put a uuid inside fruits_original.sh to recognize where is your snippet that you want to work with.
# fruits_original.sh
# snip 419d0df3-5f08-4511-ad5a-ad24db45aa6c
fruits=()
# snip 419d0df3-5f08-4511-ad5a-ad24db45aa6c
some stuff
Then extract the relevant parts with sed or other tool, declare "$part" it into a variable, append normally and then capture output from declare -p and replace the content between the marks again.
If not going with any of the above and this is only a very toy example to test some stuff, you could:
# read the line from another script
declare "$(sed '/fruits=/!d' fruits_original.sh)"
# append element
fruits+=(Orange)
# create source-able output
new="$(declare -p fruits)"
# remove declare -- in front
new="fruits=${new%*fruits=}"
# Replace the line with declare -p output.
sed -i "s/fruits=.*/fruits=$new/" fruits_original.sh
Notes:
var[${#var[#]}]='Orange' - just var+=(Orange). No need for ${#.
$( echo ${joined:1}) is a useless use of echo (unless you want word splitting and filename expansion).
check your scripts with https://shellcheck.net
fruits= ( is not an assignment and will run a subshell and could cause syntax error. There is no space in assignment around =.
declare -a var=$( - var is not an array (or, it's an array with one element).

Convert JSON dictionary into Bash one?

I am trying to receive output from an aws-cli command into Bash, and use it as an input to the second.
I successfully saved the output into the variable with this helpful answer:
iid=$(aws ec2 run-instances ...)
and receive output like that:
{ "ImageId": "ami-0abcdef1234567890", "InstanceId": "i-1231231230abcdef0", "InstanceType": "t2.micro", ... }
I know that Bash since v4 supports associative arrays but I'm struggling converting one into one.
I tried to parse the dictionary in Bash but received error:
must use subscript when assigning associative array
This is because the right key syntax for Bash dicts is =, not :.
Finally I accessed the members by using this marvelous answer with sed:
echo $iid|sed 's/{//g;s/}//g;s/"//g'|cut -d ':' -f2
My question: is there any standard way of creating a Bash dictionary from JSON or text besides regex? Best-practice?
Considering the snippet from the answer I used, the sed-approach can be very verbose and the verbosity increases exponentially with the number of keys/members:
for items in `echo $db|sed 's/{//;s/}//'`
do
echo one${count} = `echo $items|sed 's/^.*\[//;s/\].*$//'|cut -d ',' -f1`
echo two${count} = `echo $items|sed 's/^.*\[//;s/\].*$//'|cut -d ',' -f2`
echo three${count} = `echo $items|sed 's/^.*\[//;s/\].*$//'|cut -d ',' -f3`
echo four${count} = `echo $items|sed 's/^.*\[//;s/\].*$//'|cut -d ',' -f4`
...
done
For simple dicts it is OK, but for the complex dictionaries with hundreds of keys and a big nesting level it is almost inapplicable.
Is there any unified approach for arbitrary dictionary?
P.S. I found answers about solving the opposite (receiving Bash dict in Python), solving the task through jq, creating dict from Bash to Bash and from non-common input but nothing about specifically JSON. I prefer not to use jq and python and stick to the standard Bash toolset, most of the answers of this collective answer use 3rd-party tools. Is it possible at all?
One way to turn your JSON object members into a Bash4+'s associative array:
#!/usr/bin/env bash
# shellcheck disable=SC2155 # Associative array declaration from JSON
declare -A assoc=$(
jq -r '"(",(to_entries | .[] | "["+(.key|#sh)+"]="+(.value|#sh)),")"' \
input.json
)
# Debug dump the Associative array declaration
typeset -p assoc
Sample output:
declare -A assoc=([InstanceId]="i-1231231230abcdef0" [InstanceType]="t2.micro" [ImageId]="ami-0abcdef1234567890" )

How to pass an array and an associative array between bash scripts?

I want to pass an array and an associative array between bash script.
I tried to send the argument as in the example bellow, but I get the erro message:
./b.sh: line 3: ${1[#]}: bad substitution
How can I do this?
Example:
First script a.sh that call other script b.sh
a.sh
#!/usr/bin/bash -x
declare -a array=("a" "b")
declare -A associative_array
associative_array[10]="Hello world"
./b.sh "${array[#]}" $associative_array
b.sh
#!/usr/bin/bash
declare -a array="${1[#]}"
declare -A associative_array="$2"
echo "${array[#]}"
echo "${associative_array[10]}"
This is going to be flame-bait since it uses the evil eval, but you may find it useful until a better solution comes along. Don't use it without reading Why should eval be avoided in Bash, and what should I use instead? first.
a.sh
declare -a array=("a" "b")
declare -A associative_array
associative_array[10]="Hello world"
./b.sh "$(declare -p array)" "$(declare -p associative_array)"
b.sh
eval "$1"
eval "$2"
echo "${array[#]}"
echo "${associative_array[10]}"

Bash: Is there a way to add UUIDs and their Mount Points into an array?

I'm trying to write a backup script that takes a specific list of disk's UUIDs, mounts them to specified points, rsyncs the data to a specified end point, and then also does a bunch of other conditional checks when that's done. But since there will be a lot of checking after rsync, it would be nice to set each disk's UUID and associated/desired mount point as strings to save hardcoding them throughout the script, and also if say the UUIDs change in future (if a drive is updated or swapped out), this will be easier to maintain the script...
I've been looking at arrays in bash, to make the list of wanted disks, but have questions about how to make this possible, as my experience with arrays is non-existent!
The list of disks---in order of priority wanted to backup---are:
# sdc1 2.7T UUID 8C1CC0C19D012E29 /media/user/Documents/
# sdb1 1.8T UUID 39CD106C6FDA5907 /media/user/Photos/
# sdd1 3.7T UUID 5104D5B708E102C0 /media/user/Video/
... and notice I want to go sdc1, sdb1, sdd1 and so on, (i.e. custom order).
Is it possible to create this list, in order of priority, so it's something like this?
DisksToBackup
└─1
└─UUID => '8C1CC0C19D012E29'
└─MountPoint => '/media/user/Documents/'
└─2
└─UUID => '39CD106C6FDA5907'
└─MountPoint => '/media/user/Photos/'
└─3
└─UUID => '5104D5B708E102C0'
└─MountPoint => '/media/user/Video/'
OR some obviously better idea than this...
And then how to actually use this?
Let's say for example, how to go through our list and mount each disk (I know this is incorrect syntax, but again, I know nothing about arrays:
mount --uuid $DisksToBackup[*][UUID] $DisksToBackup[*][MountPoint]?
Update: Using Linux Mint 19.3
Output of bash --version gives: GNU bash, version 4.4.20(1)
Bash starting with version 4 provides associative arrays but only using a single dimension. Multiple dimensions you would have to simulate using keys such as 'sdc1-uuid' as shown in the following interactive bash examples (remove leading $ and > and bash output when putting into a script).
$ declare -A disks
$ disks=([0-uuid]=8C1CC0C19D012E29 [0-mount]=/media/user/Documents/
> [1-uuid]=39CD106C6FDA5907 [1-mount]=/media/user/Photos/)
$ echo ${disks[sdc1-uuid]}
8C1CC0C19D012E29
$ echo ${disks[*]}
/media/user/Documents/ 39CD106C6FDA5907 8C1CC0C19D012E29 /media/user/Photos/
$ echo ${!disks[*]}
0-mount 0-uuid 1-uuid 1-mount
However, there is no ordering for the keys (the order of the keys differs from the order in which we defined them). You may want to use a second array as in the following example which allows you to break down the multiple dimensions as well:
$ disks_order=(0 1)
$ for i in ${disks_order[*]}; do
> echo "${disks[$i-uuid]} ${disks[$i-mount]}"
> done
8C1CC0C19D012E29 /media/user/Documents/
39CD106C6FDA5907 /media/user/Photos/
In case you use bash version 3, you need to simulate the associative array using other means. See the question on associative arrays in bash 3 or simply represent your structure in a simple array such as which makes everything more readable anyway:
$ disks=(8C1CC0C19D012E29=/media/user/Documents/
> 39CD106C6FDA5907=/media/user/Photos/)
$ for disk in "${disks[#]}"; do
> uuid="${disk%=*}"
> path="${disk##*=}"
> echo "$uuid $path"
> done
8C1CC0C19D012E29 /media/user/Documents/
39CD106C6FDA5907 /media/user/Photos/
The %=* is a fancy way of saying remove everything after (and including) the = sign. And ##*= to remove everything before (and including) the = sign.
As an example of how one could read this data into a series of arrays:
#!/usr/bin/env bash
i=0
declare -g -A "disk$i"
declare -n currDisk="disk$i"
while IFS= read -r line || (( ${#currDisk[#]} )); do : "line=$line"
if [[ $line ]]; then
if [[ $line = *=* ]]; then
currDisk[${line%%=*}]=${line#*=}
else
printf 'WARNING: Ignoring unrecognized line: %q\n' "$line" >&2
fi
else
if [[ ${#currDisk[#]} ]]; then
declare -p "disk$i" >&2 # for debugging/demo: print out what we created
(( ++i ))
unset -n currDisk
declare -g -A "disk$i=( )"
declare -n currDisk="disk$i"
fi
fi
done < <(blkid -o export)
This gives you something like:
declare -g -A disk0=( [PARTLABEL]="primary" [UUID]="1111-2222-3333" [TYPE]=btrfs ...)
declare -g -A disk1=( [PARTLABEL]="esp" [LABEL]="boot" [TYPE]="vfat" ...)
...so you can write code iterating over them doing whatever search/comparison/etc you want. For example:
for _diskVar in "${!disk#}"; do # iterates over variable names starting with "disk"
declare -n _currDisk="$_diskVar" # refer to each such variable as _currDisk in turn
# replace the below with your actual application logic, whatever that is
if [[ ${_currDisk[LABEL]} = "something" ]] && [[ ${_currDisk[TYPE]} = "something_else" ]]; then
echo "Found ${_currDisk[DEVNAME]}"
fi
unset -n _currDisk # clear the nameref when done with it
done

Shell (sh) script array

Have been trying to find away to do array in sh shell script. Other than BASH etc. have not found much, other than arrays not supported in sh.
Here's what I've come up with using setvar and eval. Is there a better way? Any way to eliminate setvar and/or eval?
#!/bin/sh
# FreeBSD 11.1
# Kind of an array workaround.
echo -e "Simulated array creation and element assignment using dynamic index."
array() {
i=0
for x in $2
do
setvar ${1}_${i} $x # any way to do without setvar?
i=$((i+1))
done
setvar ${1}_cnt $i
}
array "my_arry" "a b c"
echo -e "\nSimulated array element access using dynamic index."
i=0
while [ $i -lt $my_arry_cnt ]
do
eval aev=\${my_arry_${i}} # any way to do without eval?
i=$((i+1))
echo $aev
done
echo -e "\nSimulated array element access using static index."
echo ${my_arry_0}
echo ${my_arry_1}
echo ${my_arry_2}

Resources