Here is an example that runs how I want it to in bash
declare -A price_oracle=(
[name]="price_oracle_name"
[active]=true
)
declare -A market=(
[name]="market_name"
[active]=true
)
declare -a contract_list=(
price_oracle
market
)
for contract in "${contract_list[#]}"; do
declare -n lst="$contract"
if ${lst[is_active]}; then
echo "Importing schema for $contract contract"
wasm-cli import -s --name $contract contracts/$contract/schema
fi
done
How can this same functionality be accomplished in zsh? declare -n is not valid in zsh.
The nested parameter expansion flag (P) can be used instead of a bash nameref:
#!/usr/bin/env zsh
declare -A price_oracle=(
[name]=price_oracle_name
[active]=true
)
declare -A market=(
[name]=market_name
[active]=true
)
declare -a contract_list=(
price_oracle
market
)
for contract in $contract_list; do
local -A lst=("${(Pkv#)contract}")
if ${lst[active]}; then
echo "Importing schema for $contract contract"
wasm-cli import -s --name ${lst[name]} contracts/$contract/schema
fi
done
To work with the associative array, this also uses the key (k), value (v), and array separation (#) expansion flags. These are described in the zshexpn man page and the online zsh documentation.
Related
I want to pass an array and an associative array between bash script.
I tried to send the argument as in the example bellow, but I get the erro message:
./b.sh: line 3: ${1[#]}: bad substitution
How can I do this?
Example:
First script a.sh that call other script b.sh
a.sh
#!/usr/bin/bash -x
declare -a array=("a" "b")
declare -A associative_array
associative_array[10]="Hello world"
./b.sh "${array[#]}" $associative_array
b.sh
#!/usr/bin/bash
declare -a array="${1[#]}"
declare -A associative_array="$2"
echo "${array[#]}"
echo "${associative_array[10]}"
This is going to be flame-bait since it uses the evil eval, but you may find it useful until a better solution comes along. Don't use it without reading Why should eval be avoided in Bash, and what should I use instead? first.
a.sh
declare -a array=("a" "b")
declare -A associative_array
associative_array[10]="Hello world"
./b.sh "$(declare -p array)" "$(declare -p associative_array)"
b.sh
eval "$1"
eval "$2"
echo "${array[#]}"
echo "${associative_array[10]}"
I'm trying to write a backup script that takes a specific list of disk's UUIDs, mounts them to specified points, rsyncs the data to a specified end point, and then also does a bunch of other conditional checks when that's done. But since there will be a lot of checking after rsync, it would be nice to set each disk's UUID and associated/desired mount point as strings to save hardcoding them throughout the script, and also if say the UUIDs change in future (if a drive is updated or swapped out), this will be easier to maintain the script...
I've been looking at arrays in bash, to make the list of wanted disks, but have questions about how to make this possible, as my experience with arrays is non-existent!
The list of disks---in order of priority wanted to backup---are:
# sdc1 2.7T UUID 8C1CC0C19D012E29 /media/user/Documents/
# sdb1 1.8T UUID 39CD106C6FDA5907 /media/user/Photos/
# sdd1 3.7T UUID 5104D5B708E102C0 /media/user/Video/
... and notice I want to go sdc1, sdb1, sdd1 and so on, (i.e. custom order).
Is it possible to create this list, in order of priority, so it's something like this?
DisksToBackup
└─1
└─UUID => '8C1CC0C19D012E29'
└─MountPoint => '/media/user/Documents/'
└─2
└─UUID => '39CD106C6FDA5907'
└─MountPoint => '/media/user/Photos/'
└─3
└─UUID => '5104D5B708E102C0'
└─MountPoint => '/media/user/Video/'
OR some obviously better idea than this...
And then how to actually use this?
Let's say for example, how to go through our list and mount each disk (I know this is incorrect syntax, but again, I know nothing about arrays:
mount --uuid $DisksToBackup[*][UUID] $DisksToBackup[*][MountPoint]?
Update: Using Linux Mint 19.3
Output of bash --version gives: GNU bash, version 4.4.20(1)
Bash starting with version 4 provides associative arrays but only using a single dimension. Multiple dimensions you would have to simulate using keys such as 'sdc1-uuid' as shown in the following interactive bash examples (remove leading $ and > and bash output when putting into a script).
$ declare -A disks
$ disks=([0-uuid]=8C1CC0C19D012E29 [0-mount]=/media/user/Documents/
> [1-uuid]=39CD106C6FDA5907 [1-mount]=/media/user/Photos/)
$ echo ${disks[sdc1-uuid]}
8C1CC0C19D012E29
$ echo ${disks[*]}
/media/user/Documents/ 39CD106C6FDA5907 8C1CC0C19D012E29 /media/user/Photos/
$ echo ${!disks[*]}
0-mount 0-uuid 1-uuid 1-mount
However, there is no ordering for the keys (the order of the keys differs from the order in which we defined them). You may want to use a second array as in the following example which allows you to break down the multiple dimensions as well:
$ disks_order=(0 1)
$ for i in ${disks_order[*]}; do
> echo "${disks[$i-uuid]} ${disks[$i-mount]}"
> done
8C1CC0C19D012E29 /media/user/Documents/
39CD106C6FDA5907 /media/user/Photos/
In case you use bash version 3, you need to simulate the associative array using other means. See the question on associative arrays in bash 3 or simply represent your structure in a simple array such as which makes everything more readable anyway:
$ disks=(8C1CC0C19D012E29=/media/user/Documents/
> 39CD106C6FDA5907=/media/user/Photos/)
$ for disk in "${disks[#]}"; do
> uuid="${disk%=*}"
> path="${disk##*=}"
> echo "$uuid $path"
> done
8C1CC0C19D012E29 /media/user/Documents/
39CD106C6FDA5907 /media/user/Photos/
The %=* is a fancy way of saying remove everything after (and including) the = sign. And ##*= to remove everything before (and including) the = sign.
As an example of how one could read this data into a series of arrays:
#!/usr/bin/env bash
i=0
declare -g -A "disk$i"
declare -n currDisk="disk$i"
while IFS= read -r line || (( ${#currDisk[#]} )); do : "line=$line"
if [[ $line ]]; then
if [[ $line = *=* ]]; then
currDisk[${line%%=*}]=${line#*=}
else
printf 'WARNING: Ignoring unrecognized line: %q\n' "$line" >&2
fi
else
if [[ ${#currDisk[#]} ]]; then
declare -p "disk$i" >&2 # for debugging/demo: print out what we created
(( ++i ))
unset -n currDisk
declare -g -A "disk$i=( )"
declare -n currDisk="disk$i"
fi
fi
done < <(blkid -o export)
This gives you something like:
declare -g -A disk0=( [PARTLABEL]="primary" [UUID]="1111-2222-3333" [TYPE]=btrfs ...)
declare -g -A disk1=( [PARTLABEL]="esp" [LABEL]="boot" [TYPE]="vfat" ...)
...so you can write code iterating over them doing whatever search/comparison/etc you want. For example:
for _diskVar in "${!disk#}"; do # iterates over variable names starting with "disk"
declare -n _currDisk="$_diskVar" # refer to each such variable as _currDisk in turn
# replace the below with your actual application logic, whatever that is
if [[ ${_currDisk[LABEL]} = "something" ]] && [[ ${_currDisk[TYPE]} = "something_else" ]]; then
echo "Found ${_currDisk[DEVNAME]}"
fi
unset -n _currDisk # clear the nameref when done with it
done
I have a little bash script that I had working before. Where I used Unix IFS to import a key/value from a text file:
#!/bin/bash
KEY=/home/myusr/.keyinfo
IFS="
"
set -A arr $(cat $KEY)
echo "This is ${arr[0]}"
echo "This is ${arr[1]}"
Input .keyinfo file:
ABC 123
However, I'm trying to get this to work on a different flavor of Linux and I'm getting this error message:
./tst3.sh: line 7: set: -A: invalid option
set: usage: set [--abefhkmnptuvxBCHP] [-o option-name] [arg ...]
This is
This is
Question:
Is this the better way to use IFS? Best practice?
I'd like to dump the key/value pairs into an array and then call those out later in my script.
You can use your code by modifying it like this:
#!/bin/bash
KEY=/home/myusr/.keyinfo
IFS=$' ' # omitting this line will do too, as IFS is defaulted to space
declare -a arr=($(cat $KEY))
echo "This is ${arr[0]}"
echo "This is ${arr[1]}"
Use declare to declare variables, not set.
-A option is for associative array, -a for indexed array.
Instead of using cat you should consider using: declare -a arr=($(< $KEY))
I'm writing a Bash script to simplify file copies from our main site to multiple agencies.
In this script, I'm trying to use a variable as an associative array name but I optain an error, here is the code :
#!/bin/bash
declare -A GROUP1
declare -A GROUP2
declare -A GROUP3
declare -A ARRAY
GROUP1["SITE1"]="x.x.x.x"
GROUP1["SITE2"]="y.y.y.y"
GROUP1["SITE3"]="z.z.z.z"
GROUP2["SITE1"]="1.1.1.1"
GROUP2["SITE2"]="2.2.2.2"
GROUP2["SITE3"]="3.3.3.3"
GROUP2["SITE1"]="a.a.a.a"
GROUP2["SITE2"]="b.b.b.b"
GROUP2["SITE3"]="c.c.c.c"
read -p "Choose a group of sites : " group
case $group in
1 ) DEST="GROUP1" ;;
2 ) DEST="GROUP2" ;;
3 ) DEST="GROUP3" ;;
esac
eval "ARRAY=(\${$DEST[#]})"
for elem in "${!ARRAY[#]}"
do
echo $elem
echo ${ARRAY[$elem]}
done
Here is the error :
./test: line28: TAB : 3.3.3.3 : must use subscript when assigning associative array
./test: line28: TAB : 2.2.2.2 : must use subscript when assigning associative array
./test: line28: TAB : 1.1.1.1 : must use subscript when assigning associative array
Is what I am trying to do possible ?
Thanks in advance.
It's possible but it's not easy unless you have bash v4.3. With 4.3, you can use a "nameref":
declare -A GROUP1
declare -A GROUP2
declare -A GROUP3
GROUP1["SITE1"]="x.x.x.x"
#...
# Yuk. Use command-line options, not stdin
read -p "Choose a group of sites : " group
for g in GROUP1 GROUP2 GROUP3; do if [[ $group == $g ]]; then
# Here's the nameref: After this executes, ARRAY is an alias for
# the indicated variable.
declare -n ARRAY=$group
break
fi
### This is documented but might not work depending on bash version
### Instead you could test for existence of a known key:
### if [[ ! -v ${ARRAY[SITE1]} ]]; then
if [[ ! -R ARRAY ]]; then
echo "No such group" >> /dev/stderr; exit 1
fi
OK, you probably don't yet have bash 4.3 but in the future the above will be useful to someone. So without it, you could follow a strategy like the one which you propose, which is to copy the indicated associative array. That's not too bad if the arrays aren't big. To do that, use the above but replace the nameref line (declare -n ARRAY=$group) with the following:
defn=$(declare -p $group)
eval "${defn/$group/ARRAY}"
unset defn
In this case, you'll have the use the alternative test for success.
Note: It's really not a good idea to use all caps for bash variable names. The convention is that system-generated environment variables and shell builtin variables use all caps. User variables should be lower-cased in order to not collide with these predefined variables.
Assume the following in Bash:
declare -A ar='([one]="1" [two]="2" )'
declare -a ari='([0]="one" [1]="two")'
for i in ${!ari[#]}; do
echo $i ${ari[i]} ${ar[${ari[i]}]}
done
0 one 1
1 two 2
Can the same be done with GNU Parallel, making sure to use the index of the associative array, not the sequence? Does the fact that arrays can't be exported make this difficult, if not impossible?
Yes, it makes it trickier. But not impossible.
You can't export an array directly. However, you can turn an array into a description of that same array using declare -p, and you can store that description in an exportable variable. In fact, you can store that description in a function and export the function, although it's a bit of a hack, and you have to deal with the fact that executing a declare command inside a function makes the declared variables local, so you need to
introduce a -g flag into the generated declare functions.
UPDATE: Since shellshock, the above hack doesn't work. A small variation on the theme does work. So if your bash has been updated, please skip down to the subtitle "ShellShock Version".
So, here's one possible way of generating the exportable function:
make_importer () {
local func=$1; shift;
export $func='() {
'"$(for arr in $#; do
declare -p $arr|sed '1s/declare -./&g/'
done)"'
}'
}
Now we can create our arrays and build an exported importer for them:
$ declare -A ar='([one]="1" [two]="2" )'
$ declare -a ari='([0]="one" [1]="two")'
$ make_importer ar_importer ar ari
And see what we've built
$ echo "$ar_importer"
() {
declare -Ag ar='([one]="1" [two]="2" )'
declare -ag ari='([0]="one" [1]="two")'
}
OK, the formatting is a bit ugly, but this isn't about whitespace. Here's the hack, though. All we've got there is an ordinary (albeit exported) variable, but when it gets imported into a subshell, a bit of magic happens [Note 1]:
$ bash -c 'echo "$ar_importer"'
$ bash -c 'type ar_importer'
ar_importer is a function
ar_importer ()
{
declare -Ag ar='([one]="1" [two]="2" )';
declare -ag ari='([0]="one" [1]="two")'
}
And it looks prettier, too.
Now we can run it in the command we give to parallel:
$ printf %s\\n ${!ari[#]} |
parallel \
'ar_importer; echo "{}" "${ari[{}]}" "${ar[${ari[{}]}]}"'
0 one 1
1 two 2
Or, for execution on a remote machine:
$ printf %s\\n ${!ari[#]} |
parallel -S localhost --env ar_importer \
'ar_importer; echo "{}" "${ari[{}]}" "${ar[${ari[{}]}]}"'
0 one 1
1 two 2
ShellShock version.
Unfortunately the flurry of fixes to shellshock make it a little harder to accomplish the same task. In particular, it is now necessary to export a function named foo as the environment variable named BASH_FUNC_foo%%, which is an invalid name (because of the percent signs). But we can still define the function (using eval) and export it, as follows:
make_importer () {
local func=$1; shift;
# An alternative to eval is:
# . /dev/stdin <<< ...
# but that is neither less nor more dangerous than eval.
eval "$func"'() {
'"$(for arr in $#; do
declare -p $arr|sed '1s/declare -./&g/'
done)"'
}'
export -f "$func"
}
As above, we can then build the arrays and make an exporter:
$ declare -A ar='([one]="1" [two]="2" )'
$ declare -a ari='([0]="one" [1]="two")'
$ make_importer ar_importer ar ari
But now, the function actually exists in our environment as a function:
$ type ar_importer
ar_importer is a function
ar_importer ()
{
declare -Ag ar='([one]="1" [two]="2" )';
declare -ag ari='([0]="one" [1]="two")'
}
Since it has been exported, we can run it in the command we give to parallel:
$ printf %s\\n ${!ari[#]} |
parallel \
'ar_importer; echo "{}" "${ari[{}]}" "${ar[${ari[{}]}]}"'
0 one 1
1 two 2
Unfortunately, it no longer works on a remote machine (at least with the version of parallel I have available) because parallel doesn't know how to export functions. If this gets fixed, the following should work:
$ printf %s\\n ${!ari[#]} |
parallel -S localhost --env ar_importer \
'ar_importer; echo "{}" "${ari[{}]}" "${ar[${ari[{}]}]}"'
However, there is one important caveat: you cannot export a function from a bash with the shellshock patch to a bash without the patch, or vice versa. So even if parallel gets fixed, the remote machine(s) must be running the same bash version as the local machine. (Or at least, either both or neither must have the shellshock patches.)
Note 1: The magic is that the way bash marks an exported variable as a function is that the exported value starts exactly with () {. So if you export a variable which starts with those characters and is a syntactically correct function, then bash subshells will treat it as a function. (Don't expect non-bash subshells to understand, though.)
A lot has happened in 4 years. GNU Parallel 20190222 comes with env_parallel. This is a shell function that makes it possible to export the most of the environment to the commands run by GNU Parallel.
It is supported in ash, bash, csh, dash, fish, ksh, mksh, pdksh, sh, tcsh, and zsh. The support varies from shell to shell (see details on https://www.gnu.org/software/parallel/env_parallel.html). For bash you would do:
# Load the env_parallel function
. `which env_parallel.bash`
# Ignore variables currently defined
env_parallel --session
[... define your arrays, functions, aliases, and variables here ...]
env_parallel my_command ::: values
# The environment is also exported to remote systems if they use the same shell
(echo value1; echo value2) | env_parallel -Sserver1,server2 my_command
# Optional cleanup
env_parallel --end-session
So in your case something like this:
env_parallel --session
declare -A ar='([one]="1" [two]="2" )'
declare -a ari='([0]="one" [1]="two")'
foo() {
for i in ${!ari[#]}; do
echo $i ${ari[i]} ${ar[${ari[i]}]}
done;
}
env_parallel foo ::: dummy
env_parallel --end-session
As you might expect env_parallel is a bit slower than pure parallel.
GNU Parallel is a perl program. If the perl program cannot access the variables, then I do not see a way that the variables can be passed on by the perl program.
So if you want to parallelize the loop I see two options:
declare -A ar='([one]="1" [two]="2" )'
declare -a ari='([0]="one" [1]="two")'
for i in ${!ari[#]}; do
sem -j+0 echo $i ${ari[i]} ${ar[${ari[i]}]}
done
The sem solution will not guard against mixed output.
declare -A ar='([one]="1" [two]="2" )'
declare -a ari='([0]="one" [1]="two")'
for i in ${!ari[#]}; do
echo echo $i ${ari[i]} ${ar[${ari[i]}]}
done | parallel