Is there a way in Bash to make a pointer to the value of a key in an associate array? Like this:
declare -A mapp
mapp=( ["key"]="${value}" )
for k in "${!mapp[#]}"; do
pointer="${mapp["${k}"]}" # How do I do this?
done
Usually, you do not need to use a pointer, but I'm curious to see if there's a way to make one.
In a simpler situation (i.e., for normal/string variables), I would make a pointer like this:
pointer=b
read -p "Enter something: " b
eval pointer=\$${pointer}
How would I do this for an associate array? This doesn't work (skip the strikethroughed code):
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp["${k}"]
read -p "Enter ${k}: " new
eval v=\$${v} # Doesn't work
done
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp["${k}"]
read -p "Enter ${k}: " k
eval v=\$${v} # Doesn't work
done
This doesn't work either (skip the strikethroughed code):
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp
read -p "Enter ${k}: " new
eval v=\$${v["${k}"]} # Doesn't work (and has terrible readability)
done
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp
read -p "Enter ${k}: " k
eval v=\$${v["${k}"]} # Doesn't work (and has terrible readability)
done
In bash 4.3, you can use a nameref:
$ mapp=([key]=value)
$ declare -n x=mapp[key] # NO dollar sign!
$ x=7
$ echo ${mapp[key]}
7
Before 4.3, you need to use the declare command differently to do the indirection.
$ mapp=([key]=value)
$ x=mapp[key] # NO dollar sign!
$ declare "$x=7"
$ echo ${mapp[key]}
7
No problem:
$ declare -A ary=([foo]=bar [baz]=qux)
$ key=foo
$ pointer="ary[$key]"
$ echo "$pointer"
ary[foo]
$ echo "${!pointer}"
bar
A "pointer" in this sense is an indirect variable
Related
Given an array (in Bash), is there a command that prints the contents of the array according to indices?
Something like that: arr[0]=... , arr[1]=... ,...
I know that I can print it in a for loop, but I am looking for a command that does it.
Given an array with contiguous indices starting at 0:
$ arr=(one two three)
And non-contiguous indices:
$ declare -a arr2='([0]="one" [2]="two" [5]="three")'
You can print the values:
$ echo ${arr[*]} # same with arr2
one two three
Or, use a C style loop for arr:
$ for (( i=0;i<${#arr[#]};i++ )); do echo "arr[$i]=${arr[$i]}"; done
arr[0]=one
arr[1]=two
arr[2]=three
But that won't work for arr2.
So, you can expand the indices (contiguous or not) and print index, value like so:
$ for i in "${!arr2[#]}"; do echo "arr2[$i]=${arr2[$i]}"; done
arr2[0]=one
arr2[2]=two
arr2[5]=three
Or inspect it with declare -p:
$ declare -p arr
declare -a arr='([0]="one" [1]="two" [2]="three")'
Which also works if the array has non-contiguous indices (where the C loop would break):
$ declare -p arr2
declare -a arr2='([0]="one" [2]="two" [5]="three")'
Note: A common mistake is to use the sigil $ and thinking you are addressing that array or named value. It is the unadorned name that is used since the sigil will dereference and tell details of the name contained instead of that name:
$ k=arr
$ declare -p $k
declare -a arr='([0]="one" [1]="two" [2]="three")' # note this is 'arr' , not 'k'
$ declare -p k
declare -- k="arr"
Since declare with no arguments will print the entire Bash environment at that moment, you can also use utilities such as sed grep or awk against that output:
$ declare | grep 'arr'
arr=([0]="one" [1]="two" [2]="three")
k=arr
With bash, it is possible to store an array in a dictionary? I have shown some sample code of fetching an array from the dictionary but it seems to lose the fact that it is an array.
I expect it is the dict+=(["pos"]="${array[#]}") command but am unsure of how to do this or if it is even possible.
# Normal array behaviour (just an example)
array=(1 2 3)
for a in "${array[#]}"
do
echo "$a"
done
# Outputs:
# 1
# 2
# 3
# Array in a dictionary
declare -A dict
dict+=(["pos"]="${array[#]}")
# When I fetch the array, it is not an array anymore
posarray=("${dict[pos]}")
for a in "${posarray[#]}"
do
echo "$a"
done
# Outputs:
# 1 2 3
# but I want
# 1
# 2
# 3
No, but there are workarounds.
Using printf '%q ' + eval
You can flatten your array into a string:
printf -v array_str '%q ' "${array[#]}"
dict["pos"]=$array_str
...and then use eval to expand that array back:
# WARNING: Only safe if array was populated with eval-safe strings, as from printf %q
key=pos; dest=array
printf -v array_cmd "%q=( %s )" "$dest" "${dict[$key]}"
eval "$array_cmd"
Note that this is only safe if your associative array is populated through the code using printf '%q ' to escape the values before they're added; content that avoids this process is potentially unsafe to eval.
Using base64 encoding
Slower but safer (if you can't prevent modification of your dictionary's contents by untrusted code), another approach is to store a base64-encoded NUL-delimited list:
dict["pos"]=$(printf '%s\0' "${array[#]}" | openssl enc base64)
...and read it out the same way:
array=( )
while IFS= read -r -d '' item; do
array+=( "$item" )
done < <(openssl enc -d base64 <<<"${dict["pos"]}"
Using Multiple Variables + Indirect Expansion
This one's actually symmetric, though it requires bash 4.3 or newer. That said, it restricts your key names to those which are permissible as shell variable names.
key=pos
array=( "first value" "second value" )
printf -v var_name 'dict_%q' "$key"
declare -n var="$var_name"
var=( "${array[#]}" )
unset -n var
...whereafter declare -p dict_pos will emit declare -a dict_pos=([0]="first value" [1]="second value"). On the other end, for retrieval:
key=pos
printf -v var_name 'dict_%q' "$key"
declare -n var="$var_name"
array=( "${var[#]}" )
unset -n var
...whereafter declare -p array will emit declare -a array=([0]="first value" [1]="second value").
Dictionaries are associative arrays, so the question rephrased is: "Is it possible to store an array inside another array?"
No, it's not. Arrays cannot be nested.
dict+=(["pos"]="${array[#]}")
For this to work you'd need an extra set of parentheses to capture the value as an array and not a string:
dict+=(["pos"]=("${array[#]}"))
But that's not legal syntax.
I've looked, but have only seen answers to one array being passed in a script.
I want to pass multiple arrays to a bash script that assigns them as individual variables as follows:
./myScript.sh ${array1[#]} ${array2[#]} ${array3[#]}
such that: var1=array1 and var2=array2 and var3=array3
I've tried multiple options, but doing variableName=("$#") combines all arrays together into each variable. I hope to have in my bash script a variable that represents each array.
The shell passes a single argument vector (that is to say, a simple C array of strings) off to a program being run. This is an OS-level limitation: There exists no method to pass structured data between two programs (any two programs, written in any language!) in an argument list, except by encoding that structure in the contents of the members of this array of C strings.
Approach: Length Prefixes
If efficiency is a goal (both in terms of ease-of-parsing and amount of space used out of the ARG_MAX limit on command-line and environment storage), one approach to consider is prefixing each array with an argument describing its length.
By providing length arguments, however, you can indicate which sections of that argument list are supposed to be part of a given array:
./myScript \
"${#array1[#]}" "${array1[#]}" \
"${#array2[#]}" "${array2[#]}" \
"${#array3[#]}" "${array3[#]}"
...then, inside the script, you can use the length arguments to split content back into arrays:
#!/usr/bin/env bash
array1=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
array2=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
array3=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
declare -p array1 array2 array3
If run as ./myScript 3 a b c 2 X Y 1 z, this has the output:
declare -a array1='([0]="a" [1]="b" [2]="c")'
declare -a array2='([0]="X" [1]="Y")'
declare -a array3='([0]="z")'
Approach: Per-Argument Array Name Prefixes
Incidentally, a practice common in the Python world (particularly with users of the argparse library) is to allow an argument to be passed more than once to amend to a given array. In shell, this would look like:
./myScript \
"${array1[#]/#/--array1=}" \
"${array2[#]/#/--array2=}" \
"${array3[#]/#/--array3=}"
and then the code to parse it might look like:
#!/usr/bin/env bash
declare -a args array1 array2 array3
while (( $# )); do
case $1 in
--array1=*) array1+=( "${1#*=}" );;
--array2=*) array2+=( "${1#*=}" );;
--array3=*) array3+=( "${1#*=}" );;
*) args+=( "$1" );;
esac
shift
done
Thus, if your original value were array1=( one two three ) array2=( aye bee ) array3=( "hello world" ), the calling convention would be:
./myScript --array1=one --array1=two --array1=three \
--array2=aye --array2=bee \
--array3="hello world"
Approach: NUL-Delimited Streams
Another approach is to pass a filename for each array from which a NUL-delimited list of its contents can be read. One chief advantage of this approach is that the size of array contents does not count against ARG_MAX, the OS-enforced command-line length limit. Moreover, with an operating system where such is available, the below does not create real on-disk files but instead creates /dev/fd-style links to FIFOs written to by subshells writing the contents of each array.
./myScript \
<( (( ${#array1[#]} )) && printf '%s\0' "${array1[#]}") \
<( (( ${#array2[#]} )) && printf '%s\0' "${array2[#]}") \
<( (( ${#array3[#]} )) && printf '%s\0' "${array3[#]}")
...and, to read (with bash 4.4 or newer, providing mapfile -d):
#!/usr/bin/env bash
mapfile -d '' array1 <"$1"
mapfile -d '' array2 <"$2"
mapfile -d '' array3 <"$3"
...or, to support older bash releases:
#!/usr/bin/env bash
declare -a array1 array2 array3
while IFS= read -r -d '' entry; do array1+=( "$entry" ); done <"$1"
while IFS= read -r -d '' entry; do array2+=( "$entry" ); done <"$2"
while IFS= read -r -d '' entry; do array3+=( "$entry" ); done <"$3"
Charles Duffy's response works perfectly well, but I would go about it a different way that makes it simpler to initialize var1, var2 and var3 in your script:
./myScript.sh "${#array1[#]} ${#array2[#]} ${#array3[#]}" \
"${array1[#]}" "${array2[#]}" "${array3[#]}"
Then in myScript.sh
#!/bin/bash
declare -ai lens=($1);
declare -a var1=("${#:2:lens[0]}") var2=("${#:2+lens[0]:lens[1]}") var3=("${#:2+lens[0]+lens[1]:lens[2]}");
Edit: Since Charles has simplified his solution, it is probably a better and more clear solution than mine.
Here is a code sample, which shows how to pass 2 arrays to a function. There is nothing more than in previous answers except it provides a full code example.
This is coded in bash 4.4.12, i.e. after bash 4.3 which would require a different coding approach. One array contains the texts to be colorized, and the other array contains the colors to be used for each of the text elements :
function cecho_multitext () {
# usage : cecho_multitext message_array color_array
# what it does : Multiple Colored-echo.
local -n array_msgs=$1
local -n array_colors=$2
# printf '1: %q\n' "${array_msgs[#]}"
# printf '2: %q\n' "${array_colors[#]}"
local i=0
local coloredstring=""
local normalcoloredstring=""
# check array counts
# echo "msg size : "${#array_msgs[#]}
# echo "col size : "${#array_colors[#]}
[[ "${#array_msgs[#]}" -ne "${#array_colors[#]}" ]] && exit 2
# build the colored string
for msg in "${array_msgs[#]}"
do
color=${array_colors[$i]}
coloredstring="$coloredstring $color $msg "
normalcoloredstring="$normalcoloredstring $msg"
# echo -e "coloredstring ($i): $coloredstring"
i=$((i+1))
done
# DEBUG
# echo -e "colored string : $coloredstring"
# echo -e "normal color string : $normal $normalcoloredstring"
# use either echo or printf as follows :
# echo -e "$coloredstring"
printf '%b\n' "${coloredstring}"
return
}
Calling the function :
#!/bin/bash
green='\E[32m'
cyan='\E[36m'
white='\E[37m'
normal=$(tput sgr0)
declare -a text=("one" "two" "three" )
declare -a color=("$white" "$green" "$cyan")
cecho_multitext text color
Job done :-)
I do prefer using base64 to encode and decode arrays like:
encode_array(){
local array=($#)
echo -n "${array[#]}" | base64
}
decode_array(){
echo -n "$#" | base64 -d
}
some_func(){
local arr1=($(decode_array $1))
local arr2=($(decode_array $2))
local arr3=($(decode_array $3))
echo arr1 has ${#arr1[#]} items, the second item is ${arr1[2]}
echo arr2 has ${#arr2[#]} items, the third item is ${arr2[3]}
echo arr3 has ${#arr3[#]} items, the here the contents ${arr3[#]}
}
a1=(ab cd ef)
a2=(gh ij kl nm)
a3=(op ql)
some_func "$(encode_array "${a1[#]}")" "$(encode_array "${a2[#]}")" "$(encode_array "${a3[#]}")"
The output is
arr1 has 3 items, the second item is cd
arr2 has 4 items, the third item is kl
arr3 has 2 items, the here the contents op ql
Anyway, that will not work with values that have tabs or spaces. If required, we need a more elaborated solution. something like:
encode_array()
{
for item in "$#";
do
echo -n "$item" | base64
done | paste -s -d , -
}
decode_array()
{
local IFS=$'\2'
local -a arr=($(echo "$1" | tr , "\n" |
while read encoded_array_item;
do
echo "$encoded_array_item" | base64 -d;
echo "$IFS"
done))
echo "${arr[*]}";
}
test_arrays_step1()
{
local IFS=$'\2'
local -a arr1=($(decode_array $1))
local -a arr2=($(decode_array $2))
local -a arr3=($(decode_array $3))
unset IFS
echo arr1 has ${#arr1[#]} items, the second item is ${arr1[1]}
echo arr2 has ${#arr2[#]} items, the third item is ${arr2[2]}
echo arr3 has ${#arr3[#]} items, the here the contents ${arr3[#]}
}
test_arrays()
{
local a1_2="$(echo -en "c\td")";
local a1=("a b" "$a1_2" "e f");
local a2=(gh ij kl nm);
local a3=(op ql );
a1_size=${#a1[#])};
resp=$(test_arrays_step1 "$(encode_array "${a1[#]}")" "$(encode_array "${a2[#]}")" "$(encode_array "${a3[#]}")");
echo -e "$resp" | grep arr1 | grep "arr1 has $a1_size, the second item is $a1_2" || echo but it should have only $a1_size items, with the second item as $a1_2
echo "$resp"
}
Based on the answers to this question you could try the following.
Define the arrays as variable on the shell:
array1=(1 2 3)
array2=(3 4 5)
array3=(6 7 8)
Have a script like this:
arg1=("${!1}")
arg2=("${!2}")
arg3=("${!3}")
echo "arg1 array=${arg1[#]}"
echo "arg1 #elem=${#arg1[#]}"
echo "arg2 array=${arg2[#]}"
echo "arg2 #elem=${#arg2[#]}"
echo "arg3 array=${arg3[#]}"
echo "arg3 #elem=${#arg3[#]}"
And call it like this:
. ./test.sh "array1[#]" "array2[#]" "array3[#]"
Note that the script will need to be sourced (. or source) so that it is executed in the current shell environment and not a sub shell.
i have an array of zeros
declare -a MY_ARRAY=( $(for i in {1..100}; do echo 0; done) )
how to set, for example, 12-25th to "1"? i've tried:
MY_ARRAY[12..25]=1
MY_ARRAY[12:25]=1
MY_ARRAY[12-25]=1
all not working..
the range 12-25 will be variables obtain from another file.
I am looking for a simple solution better not involve in looping
please help
You can use eval here, in this manner:
eval MY_ARRAY[{12..25}]=1\;
If you want to know what is being evaled, replace eval by echo.
Using eval is generally considered as a no-no. But this use of eval here should be completely safe.
On another note,
for i in {1..100}; do echo 0; done
can also be re-written as
printf '%.1s\n' 0{1..100}
EDIT: For start & end being stored in variables, this could work:
$ declare -i start=12
$ declare -i end=12
$ eval $(eval echo "MY_ARRAY[{$start..$end}]=1;")
But in that case, you should really use loops. This answer is only for demonstration/information.
Simple one-liner:-
for i in {12..25}; do MY_ARRAY[$i]=1; done
Refer page Arrays for more manipulation examples.
If the start & end values are stored in variables, the brace expansion would not work. In that case, you should use for loop like this:
$ declare -i start=12
$ declare -i end=25
$ for ((i=$start;i<=$end;i++)); do MY_ARRAY[$i]=1; done
declare -a MY_ARRAY=(
$(printf "%.2s" 0' '{1..11}) # 11 first zeroes
$(printf "%.2s" 1' '{12..25}) # 14 ones
$(printf "%.2s" 0' '{26..100}) # remaining zeroes
)
update
If the values 12 and 25 are in two variables, let's say, From and To:
declare -a MY_ARRAY=(
$( eval "{ printf %.2s 0_{1..$((From-1))};
printf %.2s 1_{$From..$To};
printf %.2s 0_{$((To+1))..100}; }" |
tr _ ' '
)
)
I want to remove the last entry in my array, and I want the array to show me that it has 1 less entry when I am using the ${#array[#]}. This is the current line I am using:
unset GreppedURLs[${#GreppedURLs[#]} -1]
Please correct me and show me the right way.
The answer you have is (nearly) correct for non-sparse indexed arrays¹:
unset 'arr[${#arr[#]}-1]'
Bash 4.3 or higher added this new syntax to do the same:
unset arr[-1]
(Note the single quotes: they prevent pathname expansion).
Demo:
arr=( a b c )
echo ${#arr[#]}
3
for a in "${arr[#]}"; do echo "$a"; done
a
b
c
unset 'arr[${#arr[#]}-1]'
for a in "${arr[#]}"; do echo "$a"; done
a
b
Punchline
echo ${#arr[#]}
2
(GNU bash, version 4.2.8(1)-release (x86_64-pc-linux-gnu))
¹ #Wil provided an excellent answer that works for all kinds of arrays
You must remove the blank before -1.
If you'd like an answer which won't eat your kittens, try this:
array=([1]=1 {2..5} [10]=6);
# declare -a array='([1]="1" [2]="2" [3]="3" [4]="4" [5]="5" [10]="6}")'
index=("${!array[#]}");
# declare -a index='([0]="1" [1]="2" [2]="3" [3]="4" [4]="5" [5]="10")'
unset 'array[${index[#]: -1}]';
# declare -a array='([1]="1" [2]="2" [3]="3" [4]="4" [5]="5")'
And there you have it - removing the last element. Now I'll present a much easier answer which probably meets your needs but has a caveat:
array=([1]=1 {2..5} [10]=6);
# declare -a array='([1]="1" [2]="2" [3]="3" [4]="4" [5]="5" [10]="6}")'
array=("${array[#]::${#array[#]}-1}");
# declare -a array='([0]="1" [1]="2" [2]="3" [3]="4" [4]="5")'
This version takes a shortcut. It re-indexes the array and drops the last element. Unfortunately you can also see that the index has not been maintained. The values and their order has been. If you don't care about the index then this is probably the answer you wanted.
Both of the above answers will also work on bash 4 Associative Arrays.
--
The chosen answer is not safe. Here's an example:
array=([1]=1 {2..5} [10]=6);
# declare -a array='([1]="1" [2]="2" [3]="3" [4]="4" [5]="5" [10]="6")'
unset 'arr[${#arr[#]}-1]';
# declare -a array='([1]="1" [2]="2" [3]="3" [4]="4" [10]="6")'
Okay so as you can see it is unsetting the element with index 5 because it incorrectly calculated the index of the last element of the array. It failed because it operated on an assumption that all arrays are zero-based, and not sparse. This answer will fail on arrays starting with anything other than zero, arrays which are sparse, and obviously must fail for an associative array with 'fubar' for the last element.
For any indexed array (sparse or not), since bash 4.3+ (and ksh93+), this is the simplest of solutions:
unset 'array[-1]'
The quotes are needed to avoid shell expansion in bash if the -1 is an arithmetic expression or a variable. This also works correctly:
a=3; unset 'arr[ a - 4 * 1 ]'
But will not work if unquoted ('') as the * will be expanded to the list of files in the present working directory ($pwd).
For older bash versions: this works since bash 3.0 for non-sparse arrays:
unset 'arr[ ${#arr[#]}-1 ]'
Example:
$ arr=( {a..i} ); declare -p arr
declare -a arr=([0]="a" [1]="b" [2]="c" [3]="d" [4]="e" [5]="f" [6]="g" [7]="h")
$ unset 'arr[ ${#arr[#]}-1 ]'; declare -p arr
declare -a arr=([0]="a" [1]="b" [2]="c" [3]="d" [4]="e" [5]="f" [6]="g")
This will not work for sparse arrays (with some holes):
$ arr=( {a..g} [9]=i ); declare -p arr
declare -a arr=([0]="a" [1]="b" [2]="c" [3]="d" [4]="e" [5]="f" [6]="g" [9]="i")
$ unset 'arr[ ${#arr[#]}-1 ]'; declare -p arr
declare -a arr=([0]="a" [1]="b" [2]="c" [3]="d" [4]="e" [5]="f" [6]="g" [9]="i")
This happens because the count of elements (${#arr[#]}) is 8 and 8-1 is 7.
So, the command will unset arr[7], which doesn't exist. Nothing is done.
A solution, that also work for Associative arrays (in whatever it could mean "the last element" in an unsorted list) is to generate a new array of indexes.
Then use the last index to unset that element.
Assuming arr is already defined (for bash 3.0+):
$ index=( "${!arr[#]}" ) # makes index non-sparse.
$ unset 'arr[${index[#]}-1]' # unset the last index.
$ declare -p arr
declare -a arr=([0]="a" [1]="b" [2]="c" [3]="d" [4]="e" [5]="f" [6]="g")
A slightly more portable (works in ksh93), that looks ugly, solution is:
$ arr=( {a..e} [9]=i )
$ index=( "${!arr[#]}" )
$ unset "arr[ ${index[${#index[#]}-1]} ]" # Yes, double quotes.
$ declare -p arr
declare -a arr=([0]="a" [1]="b" [2]="c" [3]="d" [4]="e")
Or (again, double quotes for ksh):
$ unset "arr[${index[#]: -1}]"
If you want to avoid the space and the negative number, make it a variable:
$ a="-1"; unset "arr[${index[#]:a}]"
The following works fine for Mac/bash#3.x and Linux (ubuntu/bash#4.x)
unset arr[$[${#arr[#]}-1]] # non-sparse array only
in more details:
len=${#arr[#]}
idx=$[$len-1] # <=> $(($len-1))
unset arr[$idx]
In your function, you could add the following:
target="${#:$(($#)):1}"
set -- "${#:1:$(($#-1))}"