Deleting specific values from an array in ksh - arrays

I have a customized .profile that I use in ksh and below is a function that I created to skip back and forth from directories with overly complicated or long names.
As you can see, the pathnames are stored in an array (BOOKMARKS[]) to keep track of them and reference them at a later time. I want to be able to delete certain values from the array, using a case statement (or OPTARG if necessary) so that I can just type bmk -d # to remove the path at the associated index.
I have fiddled around with array +A and -A, but it just wound up screwing up my array (what is left in the commented out code may not be pretty...I didn't proofread it).
Any suggestions/tips on how to create that functionality? Thanks!
# To bookmark the current directory you are in for easy navigation back and forth from multiple non-aliased directories
# Use like 'bmk' (sets the current directory to a bookmark number) to go back to this directory, i.e. type 'bmk 3' (for the 3rd)
# To find out what directories are linked to which numbers, type 'bmk -l' (lowercase L)
# For every new directory bookmarked, the number will increase so the first time you run 'bmk' it will be 1 then 2,3,4...etc. for every consecutive run therea
fter
# TODO: finish -d (delete bookmark entry) function
make_bookmark()
{
if [[ $# -eq 0 ]]; then
BOOKMARKS[${COUNTER}]=${PWD}
(( COUNTER=COUNTER+1 ))
else
case $1 in
-l) NUM_OF_ELEMENTS=${#BOOKMARKS[*]}
while [[ ${COUNTER} -lt ${NUM_OF_ELEMENTS} ]]
do
(( ACTUAL_NUM=i+1 ))
echo ${ACTUAL_NUM}":"${BOOKMARKS[${i}]}
(( COUNTER=COUNTER+1 ))
done
break ;;
#-d) ACTUAL_NUM=$2
#(( REMOVE=${ACTUAL_NUM}-1 ))
#echo "Removing path ${BOOKMARKS[${REMOVE}]} from 'bmk'..."
#NUM_OF_ELEMENTS=${#BOOKMARKS[*]}
#while [[ ${NUM_OF_ELEMENTS} -gt 0 ]]
#do
#if [[ ${NUM_OF_ELEMENTS} -ne ${ACTUAL_NUM} ]]; then
# TEMP_ARR=$(echo "${BOOKMARKS[*]}")
# (( NUM_OF_ELEMENTS=${NUM_OF_ELEMENTS}-1 ))
#fi
#echo $TEMP_ARR
#done
#break
#for VALUE in ${TEMP_ARR}
#do
# set +A BOOKMARK ${TEMP_ARR}
#done
#echo ${BOOKMARK[*]}
#break ;;
*) (( INDEX=$1-1 ))
cd ${BOOKMARKS[${INDEX}]}
break ;;
esac
fi
}

Arrays in the Korn shell (and Bash and others) are sparse, so if you use unset to delete members of the array, you won't be able to use the size of the array as an index to the last member and other limitations.
Here are some useful snippets (the second for loop is something you might be able to put to use right away):
array=(1 2 3)
unset array[2]
echo ${array[2]} # null
indices=(${!array[#]}) # create an array of the indices of "array"
size=${#indices[#]} # the size of "array" is the number of indices into it
size=${#array[#]} # same
echo ${array[#]: -1} # you can use slices to get array elements, -1 is the last one, etc.
for element in ${array[#]}; do # iterate over the array without an index
for index in ${indices[#]} # iterate over the array WITH an index
do
echo "Index: ${index}, Element: ${array[index]}"
done
for index in ${!array[#]} # iterate over the array WITH an index, directly
That last one can eliminate the need for a counter.
Here are a couple more handy techniques:
array+=("new element") # append a new element without referring to an index
((counter++)) # shorter than ((counter=counter+1)) or ((counter+=1))
if [[ $var == 3 ]] # you can use the more "natural" comparison operators inside double square brackets
while [[ $var < 11 ]] # another example
echo ${array[${index}-1] # math inside an array subscript
This all assumes ksh93, some things may not work in earlier versions.

you can use unset. eg to delete array element 1
unset array[0]
to delete entire array
unset array

A few caveats regarding the previous answer:
First: I see this error all the time. When you provide an array element to "unset", you have to quote it. Consider:
$ echo foo > ./a2
$ ls a[2]
a2
$ a2="Do not delete this"
$ a=(this is not an array)
$ unset -v a[2]
$ echo "a2=${a2-UNSET}, a[]=${a[#]}"
a2=UNSET a[]=this is not an array
What happened? Globbing. You obviously wanted to delete element 2 of a[], but shell syntax being what it is, the shell first checked the current directory for a file that matched the glob pattern "a[2]". If it finds a match, it replaces the glob pattern with that filename, and you wind up making a decision about which variable to delete based on what files exist in your current directory.
This is profoundly stupid. But it's not something anyone has bothered to fix, apparently, and the error turns up in all kinds of documentation and example code from the last 3 decades.
Next is a related problem: it's easy to insert elements in your associative array with any key you like. But it's harder to remove these elements:
typeset -A assoc
key="foo] bar"
assoc[$key]=3 #No problem!
unset -v "assoc[$key]" #Problem!
In bash you can do this:
unset -v "assoc[\$key]"
In Korn Shell, you have to do this:
unset -v "assoc[foo\]\ bar]"
So it gets a bit more complicated in the case where your keys contain syntax characters.

Related

Array not expanding when using in if

i am creating a script that will help updating the GoLang compile binary in a GNU/Linux system.
but it fail
#!/usr/bin/env bash
# -*- coding: utf-8 -*-
# set -e
# ==============================================================================
# title : Semi Automatic Update GoLang
# description : Script for Install, setup path, permission and update golang
# author : Walddys Emmanuel Dorrejo Céspedes
# usage : bash up_install.sh
# notes : Execute this Script will ask sudo password
# dependencies : wget awk sed curl tar
# ==============================================================================
## Get Golang Versions from repository
declare -a go_jversion go_sversion
readarray -t go_jversion < <(curl -s https://go.googlesource.com/go/+refs?format=JSON | grep -Eo "go[0-9]\.[^\"]+" | sort -V)
## Delete go_versions RC and Beta from the pool.
for i in "${!go_jversion[#]}"; do
if [[ "${go_jversion[i]}" =~ (rc|beta) ]]; then
unset "go_jversion[i]"
fi
done
unset go_sversion # Not allow to grow indefinitely the pool when re-execute the scripts
for i in "${!go_jversion[#]}"; do
set -vx
## Create an array of the stables versions (Those versions that repeat more than or equal to 2 are stables)
# if [[ "${go_jversion[i]}" == "${go_jversion[i + 1]}" ]] && [[ "${go_sversion[i - 1]}" != "${go_jversion[i + 1]}" ]]; then
# go_sversion+=("${go_jversion[i]}")
# fi
In this section i am comparing major version + minimum version, to exclude the patch version of the array, but the condition after the "&&", the array "${go_sversion[$i -1]}" is expanding null in each cycle of the loop, when i am assigning a value in a cycle before.
## Create an array of the stables versions (Those versions that repeat more than or equal to 2 are stables) (second version)
if [[ "${go_jversion[$i]}" == "${go_jversion[$i + 1]}" && "${go_sversion[$i - 1]}" != "${go_jversion[$i]}" ]]; then
go_sversion+=("${go_jversion[$i]}")
echo "${!go_sversion[$i]}"
fi
set +vx
done
echo "${go_sversion[#]}"
echo "${!go_sversion[#]}"
My issue is in the section where "${go_sversion[$i -1]}", why is not expanding?
assign value to "${go_sversion[$i -1]}" the value display in the next cycle of the loop
Arrays in bash are allowed to be sparse, meaning their indices are not required to be strictly sequential. For example:
arr=(1 2 3)
echo "${arr[#]}" # prints 1 2 3
echo "${!arr[#]}" # prints 0 1 2
unset arr\[1\]
echo "${arr[#]}" # prints 1 3
echo "${!arr[#]}" # prints 0 2
When you unset the RC and Beta values you could be creating these types of gaps in your jversion array, but you're assigning to the sversion array sequentially. This means the indices do not align between the arrays.
If your jversion looks like my array above, you might put something into sversion[0] from jversion[0], then process jversion[2] and attempt to match it against sversion[1] which doesn't exist yet.
One simple way to de-sparsify the array is to reassign it:
go_jversion=( "${go_jversion[#]}" )
This will reassign the contents of the array to itself in sequential order without any gaps in the indices.
If this is unviable for some reason, you'll have to write code that is aware of the possible sparseness of the array. For example, instead of blinding looking at go_sversion[i-1] you could look at go_sversion[-1] which will always give you the last item in the array.

How do you unset all empty array elements in bash? [duplicate]

I need to remove an element from an array in bash shell.
Generally I'd simply do:
array=("${(#)array:#<element to remove>}")
Unfortunately the element I want to remove is a variable so I can't use the previous command.
Down here an example:
array+=(pluto)
array+=(pippo)
delete=(pluto)
array( ${array[#]/$delete} ) -> but clearly doesn't work because of {}
Any idea?
The following works as you would like in bash and zsh:
$ array=(pluto pippo)
$ delete=pluto
$ echo ${array[#]/$delete}
pippo
$ array=( "${array[#]/$delete}" ) #Quotes when working with strings
If need to delete more than one element:
...
$ delete=(pluto pippo)
for del in ${delete[#]}
do
array=("${array[#]/$del}") #Quotes when working with strings
done
Caveat
This technique actually removes prefixes matching $delete from the elements, not necessarily whole elements.
Update
To really remove an exact item, you need to walk through the array, comparing the target to each element, and using unset to delete an exact match.
array=(pluto pippo bob)
delete=(pippo)
for target in "${delete[#]}"; do
for i in "${!array[#]}"; do
if [[ ${array[i]} = $target ]]; then
unset 'array[i]'
fi
done
done
Note that if you do this, and one or more elements is removed, the indices will no longer be a continuous sequence of integers.
$ declare -p array
declare -a array=([0]="pluto" [2]="bob")
The simple fact is, arrays were not designed for use as mutable data structures. They are primarily used for storing lists of items in a single variable without needing to waste a character as a delimiter (e.g., to store a list of strings which can contain whitespace).
If gaps are a problem, then you need to rebuild the array to fill the gaps:
for i in "${!array[#]}"; do
new_array+=( "${array[i]}" )
done
array=("${new_array[#]}")
unset new_array
You could build up a new array without the undesired element, then assign it back to the old array. This works in bash:
array=(pluto pippo)
new_array=()
for value in "${array[#]}"
do
[[ $value != pluto ]] && new_array+=($value)
done
array=("${new_array[#]}")
unset new_array
This yields:
echo "${array[#]}"
pippo
This is the most direct way to unset a value if you know it's position.
$ array=(one two three)
$ echo ${#array[#]}
3
$ unset 'array[1]'
$ echo ${array[#]}
one three
$ echo ${#array[#]}
2
This answer is specific to the case of deleting multiple values from large arrays, where performance is important.
The most voted solutions are (1) pattern substitution on an array, or (2) iterating over the array elements. The first is fast, but can only deal with elements that have distinct prefix, the second has O(n*k), n=array size, k=elements to remove. Associative array are relative new feature, and might not have been common when the question was originally posted.
For the exact match case, with large n and k, possible to improve performance from O(nk) to O(n+klog(k)). In practice, O(n) assuming k much lower than n. Most of the speed up is based on using associative array to identify items to be removed.
Performance (n-array size, k-values to delete). Performance measure seconds of user time
N K New(seconds) Current(seconds) Speedup
1000 10 0.005 0.033 6X
10000 10 0.070 0.348 5X
10000 20 0.070 0.656 9X
10000 1 0.043 0.050 -7%
As expected, the current solution is linear to N*K, and the fast solution is practically linear to K, with much lower constant. The fast solution is slightly slower vs the current solution when k=1, due to additional setup.
The 'Fast' solution: array=list of input, delete=list of values to remove.
declare -A delk
for del in "${delete[#]}" ; do delk[$del]=1 ; done
# Tag items to remove, based on
for k in "${!array[#]}" ; do
[ "${delk[${array[$k]}]-}" ] && unset 'array[k]'
done
# Compaction
array=("${array[#]}")
Benchmarked against current solution, from the most-voted answer.
for target in "${delete[#]}"; do
for i in "${!array[#]}"; do
if [[ ${array[i]} = $target ]]; then
unset 'array[i]'
fi
done
done
array=("${array[#]}")
Here's a one-line solution with mapfile:
$ mapfile -d $'\0' -t arr < <(printf '%s\0' "${arr[#]}" | grep -Pzv "<regexp>")
Example:
$ arr=("Adam" "Bob" "Claire"$'\n'"Smith" "David" "Eve" "Fred")
$ echo "Size: ${#arr[*]} Contents: ${arr[*]}"
Size: 6 Contents: Adam Bob Claire
Smith David Eve Fred
$ mapfile -d $'\0' -t arr < <(printf '%s\0' "${arr[#]}" | grep -Pzv "^Claire\nSmith$")
$ echo "Size: ${#arr[*]} Contents: ${arr[*]}"
Size: 5 Contents: Adam Bob David Eve Fred
This method allows for great flexibility by modifying/exchanging the grep command and doesn't leave any empty strings in the array.
Partial answer only
To delete the first item in the array
unset 'array[0]'
To delete the last item in the array
unset 'array[-1]'
To expand on the above answers, the following can be used to remove multiple elements from an array, without partial matching:
ARRAY=(one two onetwo three four threefour "one six")
TO_REMOVE=(one four)
TEMP_ARRAY=()
for pkg in "${ARRAY[#]}"; do
for remove in "${TO_REMOVE[#]}"; do
KEEP=true
if [[ ${pkg} == ${remove} ]]; then
KEEP=false
break
fi
done
if ${KEEP}; then
TEMP_ARRAY+=(${pkg})
fi
done
ARRAY=("${TEMP_ARRAY[#]}")
unset TEMP_ARRAY
This will result in an array containing:
(two onetwo three threefour "one six")
Here's a (probably very bash-specific) little function involving bash variable indirection and unset; it's a general solution that does not involve text substitution or discarding empty elements and has no problems with quoting/whitespace etc.
delete_ary_elmt() {
local word=$1 # the element to search for & delete
local aryref="$2[#]" # a necessary step since '${!$2[#]}' is a syntax error
local arycopy=("${!aryref}") # create a copy of the input array
local status=1
for (( i = ${#arycopy[#]} - 1; i >= 0; i-- )); do # iterate over indices backwards
elmt=${arycopy[$i]}
[[ $elmt == $word ]] && unset "$2[$i]" && status=0 # unset matching elmts in orig. ary
done
return $status # return 0 if something was deleted; 1 if not
}
array=(a 0 0 b 0 0 0 c 0 d e 0 0 0)
delete_ary_elmt 0 array
for e in "${array[#]}"; do
echo "$e"
done
# prints "a" "b" "c" "d" in lines
Use it like delete_ary_elmt ELEMENT ARRAYNAME without any $ sigil. Switch the == $word for == $word* for prefix matches; use ${elmt,,} == ${word,,} for case-insensitive matches; etc., whatever bash [[ supports.
It works by determining the indices of the input array and iterating over them backwards (so deleting elements doesn't screw up iteration order). To get the indices you need to access the input array by name, which can be done via bash variable indirection x=1; varname=x; echo ${!varname} # prints "1".
You can't access arrays by name like aryname=a; echo "${$aryname[#]}, this gives you an error. You can't do aryname=a; echo "${!aryname[#]}", this gives you the indices of the variable aryname (although it is not an array). What DOES work is aryref="a[#]"; echo "${!aryref}", which will print the elements of the array a, preserving shell-word quoting and whitespace exactly like echo "${a[#]}". But this only works for printing the elements of an array, not for printing its length or indices (aryref="!a[#]" or aryref="#a[#]" or "${!!aryref}" or "${#!aryref}", they all fail).
So I copy the original array by its name via bash indirection and get the indices from the copy. To iterate over the indices in reverse I use a C-style for loop. I could also do it by accessing the indices via ${!arycopy[#]} and reversing them with tac, which is a cat that turns around the input line order.
A function solution without variable indirection would probably have to involve eval, which may or may not be safe to use in that situation (I can't tell).
Using unset
To remove an element at particular index, we can use unset and then do copy to another array. Only just unset is not required in this case. Because unset does not remove the element it just sets null string to the particular index in array.
declare -a arr=('aa' 'bb' 'cc' 'dd' 'ee')
unset 'arr[1]'
declare -a arr2=()
i=0
for element in "${arr[#]}"
do
arr2[$i]=$element
((++i))
done
echo "${arr[#]}"
echo "1st val is ${arr[1]}, 2nd val is ${arr[2]}"
echo "${arr2[#]}"
echo "1st val is ${arr2[1]}, 2nd val is ${arr2[2]}"
Output is
aa cc dd ee
1st val is , 2nd val is cc
aa cc dd ee
1st val is cc, 2nd val is dd
Using :<idx>
We can remove some set of elements using :<idx> also. For example if we want to remove 1st element we can use :1 as mentioned below.
declare -a arr=('aa' 'bb' 'cc' 'dd' 'ee')
arr2=("${arr[#]:1}")
echo "${arr2[#]}"
echo "1st val is ${arr2[1]}, 2nd val is ${arr2[2]}"
Output is
bb cc dd ee
1st val is cc, 2nd val is dd
http://wiki.bash-hackers.org/syntax/pe#substring_removal
${PARAMETER#PATTERN} # remove from beginning
${PARAMETER##PATTERN} # remove from the beginning, greedy match
${PARAMETER%PATTERN} # remove from the end
${PARAMETER%%PATTERN} # remove from the end, greedy match
In order to do a full remove element, you have to do an unset command with an if statement. If you don't care about removing prefixes from other variables or about supporting whitespace in the array, then you can just drop the quotes and forget about for loops.
See example below for a few different ways to clean up an array.
options=("foo" "bar" "foo" "foobar" "foo bar" "bars" "bar")
# remove bar from the start of each element
options=("${options[#]/#"bar"}")
# options=("foo" "" "foo" "foobar" "foo bar" "s" "")
# remove the complete string "foo" in a for loop
count=${#options[#]}
for ((i = 0; i < count; i++)); do
if [ "${options[i]}" = "foo" ] ; then
unset 'options[i]'
fi
done
# options=( "" "foobar" "foo bar" "s" "")
# remove empty options
# note the count variable can't be recalculated easily on a sparse array
for ((i = 0; i < count; i++)); do
# echo "Element $i: '${options[i]}'"
if [ -z "${options[i]}" ] ; then
unset 'options[i]'
fi
done
# options=("foobar" "foo bar" "s")
# list them with select
echo "Choose an option:"
PS3='Option? '
select i in "${options[#]}" Quit
do
case $i in
Quit) break ;;
*) echo "You selected \"$i\"" ;;
esac
done
Output
Choose an option:
1) foobar
2) foo bar
3) s
4) Quit
Option?
Hope that helps.
There is also this syntax, e.g. if you want to delete the 2nd element :
array=("${array[#]:0:1}" "${array[#]:2}")
which is in fact the concatenation of 2 tabs. The first from the index 0 to the index 1 (exclusive) and the 2nd from the index 2 to the end.
POSIX shell script does not have arrays.
So most probably you are using a specific dialect such as bash, korn shells or zsh.
Therefore, your question as of now cannot be answered.
Maybe this works for you:
unset array[$delete]
What I do is:
array="$(echo $array | tr ' ' '\n' | sed "/itemtodelete/d")"
BAM, that item is removed.
This is a quick-and-dirty solution that will work in simple cases but will break if (a) there are regex special characters in $delete, or (b) there are any spaces at all in any items. Starting with:
array+=(pluto)
array+=(pippo)
delete=(pluto)
Delete all entries exactly matching $delete:
array=(`echo $array | fmt -1 | grep -v "^${delete}$" | fmt -999999`)
resulting in
echo $array -> pippo, and making sure it's an array:
echo $array[1] -> pippo
fmt is a little obscure: fmt -1 wraps at the first column (to put each item on its own line. That's where the problem arises with items in spaces.) fmt -999999 unwraps it back to one line, putting back the spaces between items. There are other ways to do that, such as xargs.
Addendum: If you want to delete just the first match, use sed, as described here:
array=(`echo $array | fmt -1 | sed "0,/^${delete}$/{//d;}" | fmt -999999`)
Actually, I just noticed that the shell syntax somewhat has a behavior built-in that allows for easy reconstruction of the array when, as posed in the question, an item should be removed.
# let's set up an array of items to consume:
x=()
for (( i=0; i<10; i++ )); do
x+=("$i")
done
# here, we consume that array:
while (( ${#x[#]} )); do
i=$(( $RANDOM % ${#x[#]} ))
echo "${x[i]} / ${x[#]}"
x=("${x[#]:0:i}" "${x[#]:i+1}")
done
Notice how we constructed the array using bash's x+=() syntax?
You could actually add more than one item with that, the content of a whole other array at once.
In ZSH this is dead easy (note this uses more bash compatible syntax than necessary where possible for ease of understanding):
# I always include an edge case to make sure each element
# is not being word split.
start=(one two three 'four 4' five)
work=(${(#)start})
idx=2
val=${work[idx]}
# How to remove a single element easily.
# Also works for associative arrays (at least in zsh)
work[$idx]=()
echo "Array size went down by one: "
[[ $#work -eq $(($#start - 1)) ]] && echo "OK"
echo "Array item "$val" is now gone: "
[[ -z ${work[(r)$val]} ]] && echo OK
echo "Array contents are as expected: "
wanted=("${start[#]:0:1}" "${start[#]:2}")
[[ "${(j.:.)wanted[#]}" == "${(j.:.)work[#]}" ]] && echo "OK"
echo "-- array contents: start --"
print -l -r -- "-- $#start elements" ${(#)start}
echo "-- array contents: work --"
print -l -r -- "-- $#work elements" "${work[#]}"
Results:
Array size went down by one:
OK
Array item two is now gone:
OK
Array contents are as expected:
OK
-- array contents: start --
-- 5 elements
one
two
three
four 4
five
-- array contents: work --
-- 4 elements
one
three
four 4
five
To avoid conflicts with array index using unset - see https://stackoverflow.com/a/49626928/3223785 and https://stackoverflow.com/a/47798640/3223785 for more information - reassign the array to itself: ARRAY_VAR=(${ARRAY_VAR[#]}).
#!/bin/bash
ARRAY_VAR=(0 1 2 3 4 5 6 7 8 9)
unset ARRAY_VAR[5]
unset ARRAY_VAR[4]
ARRAY_VAR=(${ARRAY_VAR[#]})
echo ${ARRAY_VAR[#]}
A_LENGTH=${#ARRAY_VAR[*]}
for (( i=0; i<=$(( $A_LENGTH -1 )); i++ )) ; do
echo ""
echo "INDEX - $i"
echo "VALUE - ${ARRAY_VAR[$i]}"
done
exit 0
[Ref.: https://tecadmin.net/working-with-array-bash-script/ ]
How about something like:
array=(one two three)
array_t=" ${array[#]} "
delete=one
array=(${array_t// $delete / })
unset array_t
#/bin/bash
echo "# define array with six elements"
arr=(zero one two three 'four 4' five)
echo "# unset by index: 0"
unset -v 'arr[0]'
for i in ${!arr[*]}; do echo "arr[$i]=${arr[$i]}"; done
arr_delete_by_content() { # value to delete
for i in ${!arr[*]}; do
[ "${arr[$i]}" = "$1" ] && unset -v 'arr[$i]'
done
}
echo "# unset in global variable where value: three"
arr_delete_by_content three
for i in ${!arr[*]}; do echo "arr[$i]=${arr[$i]}"; done
echo "# rearrange indices"
arr=( "${arr[#]}" )
for i in ${!arr[*]}; do echo "arr[$i]=${arr[$i]}"; done
delete_value() { # value arrayelements..., returns array decl.
local e val=$1; new=(); shift
for e in "${#}"; do [ "$val" != "$e" ] && new+=("$e"); done
declare -p new|sed 's,^[^=]*=,,'
}
echo "# new array without value: two"
declare -a arr="$(delete_value two "${arr[#]}")"
for i in ${!arr[*]}; do echo "arr[$i]=${arr[$i]}"; done
delete_values() { # arraydecl values..., returns array decl. (keeps indices)
declare -a arr="$1"; local i v; shift
for v in "${#}"; do
for i in ${!arr[*]}; do
[ "$v" = "${arr[$i]}" ] && unset -v 'arr[$i]'
done
done
declare -p arr|sed 's,^[^=]*=,,'
}
echo "# new array without values: one five (keep indices)"
declare -a arr="$(delete_values "$(declare -p arr|sed 's,^[^=]*=,,')" one five)"
for i in ${!arr[*]}; do echo "arr[$i]=${arr[$i]}"; done
# new array without multiple values and rearranged indices is left to the reader

Passing named arrays to another bash script

I would like to pass multiple named arrays to another bash script as they are.
For example, given
outer.sh
echo "outer"
a=(1 2)
b=(3 4 5)
echo ${#a[#]}
echo ${#b[#]}
a=${a[#]} b=${b[#]} sh inner.sh
and inner.sh
echo "inner"
echo ${a[#]}
echo ${b[#]}
echo ${#a[#]}
echo ${#b[#]}
running the outer.sh gives
$ sh outer.sh
outer
2
3
inner
1 2
3 4 5
1
1
That is, even though values are preserved, but their lengths change, which means, they are not arrays anymore, but strings.
How do I pass multiple named arrays to another bash script as they are?
There are several approaches available, each with their own disadvantages.
The Easy Way: Run The Inner Script Inside A Subshell
This means all variables are inherited, including ones (like arrays!) that can't be passed through the environment.
a=(1 2)
b=(3 4 5)
(. inner)
Of course, it also means that shell settings (IFS, set -e, etc) are inherited too, so inner.sh needs to be written robustly to handle whatever setup it may happen to receive; and you can't rewrite it in a different / non-shell language later.
The Unsafe Way: Pass eval-able code (and trust your caller!)
Modify inner.sh to run [[ $setup ]] && eval "$setup", and then invoke it as:
setup=$(declare -p a b) ./inner
Obviously, this is a severe security risk if you don't control the process environment.
The Hard Way: Deserialize into individual elements
Here, we pass each array as its name, its length, and then its original elements.
inner needs to be modified to copy items off its command-argument list back into the arrays, as in the following example:
while (( $# )); do # iterating over our argument list:
dest_name=$1; shift # expect the variable name first
dest_size=$1; shift # then its size
declare -g -a "$dest_name=( )" # initialize our received variable as empty
declare -n dest="$dest_name" # bash 4.3: make "dest" point to our target name
while (( dest_size )) && (( $# )); do # and then, for up to "size" arguments...
dest+=( "$1" ); shift # pop an argument off the list onto an array
(( dest_size -= 1 )) # and decrease the count left in "size"
done
unset -n dest # end that redirection created above
done
...and then expand into that format in outer:
./inner a "${#a[#]}" "${a[#]}" b "${#b[#]}" "${b[#]}"

Create associative array from grep output

I have a grep output and I'm trying to make an associative array from the output that I get.
Here is my grep output:
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
I want to use that output to define an associative array, like this:
array[123456789101]=devid1234
array[11213141516]=devid5678
Is that possible? I'm new at making arrays. I hope someone could help me in my problem.
Either pipe your grep output to a helper script with a while loop containing a simple "0/1" toggle to read two lines taking the last field of each to fill your array, e.g.
#!/bin/bash
declare -A array
declare -i n=0
arridx=
while read -r label value; do # read 2 fields
if [ "$n" -eq 0 ]
then
arridx="${value:1}" # strip 1st and lst 2 chars
arridx="${arridx:0:(-2)}" # save in arridx (array index)
((n++)) # increment toggle
else
arrval="${value:1}" # strip 1st and lst 2 chars
arrval="${arrval:0:(-2)}" # save in arrval (array value)
array[$arridx]="$arrval" # assign to associative array
n=0 # zero toggle
fi
done
for i in ${!array[#]}; do # output array
echo "array[$i] ${array[$i]}"
done
Or you can use process substitution containing the grep command within the script to do the same thing, e.g.
done < <( your grep command )
You can also add a check under the else clause that if [[ $label =~ DeviceId ]] to validate you are on the right line and catch any variation in the grep output content.
Example Input
$ cat dat/grepout.txt
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
Example Use/Output
$ cat dat/grepout.txt | bash parsegrep2array.sh
array[123456789101] devid1234
array[111213141516] devid5678
Parsing out the values is easy, and once you have them you can certainly use those values to build up an array. The trickiest part comes from the fact that you need to combine input from separate lines. Here is one approach; note that this script is verbose on purpose, to show what's going on; once you see what's happening, you can eliminate most of the output:
so.input
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
so.sh
#!/bin/bash
declare -a hardwareInfo
while [[ 1 ]]; do
# read in two lines of input
# if either line is the last one, we don't have enough input to proceed
read lineA < "${1:-/dev/stdin}"
# if EOF or empty line, exit
if [[ "$lineA" == "" ]]; then break; fi
read lineB < "${1:-/dev/stdin}"
# if EOF or empty line, exit
if [[ "$lineB" == "" ]]; then break; fi
echo "$lineA"
echo "$lineB"
hwsn=$lineA
hwsn=${hwsn//HardwareSerialNumber/}
hwsn=${hwsn//\"/}
hwsn=${hwsn//:/}
hwsn=${hwsn//,/}
echo $hwsn
# some checking could be done here to test that the value is numeric
devid=$lineB
devid=${devid//DeviceId/}
devid=${devid//\"/}
devid=${devid//:/}
devid=${devid//,/}
echo $devid
# some checking could be done here to make sure the value is valid
# populate the array
hardwareInfo[$hwsn]=$devid
done
# spacer, for readability of the output
echo
# display the array; in your script, you would do something different and useful
for key in "${!hardwareInfo[#]}"; do echo $key --- ${hardwareInfo[$key]}; done
cat so.input | ./so.sh
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
123456789101
devid1234
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
111213141516
devid5678
111213141516 --- devid5678
123456789101 --- devid1234
I created the input file so.input just for convenience. You would probably pipe your grep output into the bash script, like so:
grep-command | ./so.sh
EDIT #1: There are lots of choices for parsing out the key and value from the strings fed in by grep; the answer from #David C. Rankin shows another way. The best way depends on what you can rely on about the content and structure of the grep output.
There are also several choices for reading two separate lines that are related to each other; David's "toggle" approach is also good, and commonly used; I considered it myself, before going with "read two lines and stop if either is blank".
EDIT #2: I see declare -A in David's answer and in examples on the web; I used declare -a because that's what my version of bash wants (I'm using a Mac). So, just be aware that there can be differences.

Bash; creating arrays which names are computed from other variables [duplicate]

This question already has an answer here:
Assign to a bash array variable indirectly, by dynamically constructed variable name
(1 answer)
Closed 5 years ago.
My goal is to create a list of partitions for each block device listed in /sys/block;
#!/bin/bash
block_devices_list=($(ls /sys/block))
partition_list=($(cat /proc/partitions | awk '{print $4}'))
unset partition_list[0]
for block_device in ${block_devices_list[#]}; do
for partition in ${partition_list[#]}; do
partitions+=($(echo $partition | grep $block_device))
done
# Right here?
unset partitions
done
Every time the outside 'for loop' completes it's cycle it ends up with an array of partitions for a particular block device. At that point I would like to transfer that data to a separate array, dynamically named after the device it belongs to (like 'partitions_sda' for example).
I have read a few questions/answers about 'dynamic' variable names, 'associative' arrays and whatnot but don't seem to be able to figure this out. Any help much appreciated.
As a best-practices example (for bash 4.3 or newer):
#!/bin/bash
for blockdev in /sys/block/*; do
devname=${blockdev##*/} # remove leading path elements
devname=${devname//-/_} # rename dashes in device name to underscores
declare -a "partitions_${devname}=()" # define an empty array
declare -n _current_partitions="partitions_$devname" # define a nameref
for part in "$blockdev"/*/dev; do # iterate over partitions
[[ -e $part ]] || continue # skip if no matches
part=${part%/dev} # strip trailing /dev
_current_partitions+=( "${part##*/}" ) # add match via nameref
done
unset -n _current_partitions # clear the nameref
declare -p "partitions_$devname" # print our resulting array
done
For my local test VM, this emits:
declare -a partitions_dm_0=()
declare -a partitions_dm_1=()
declare -a partitions_sda=([0]="sda1" [1]="sda2")
declare -a partitions_sdb=()
declare -a partitions_sr0=()
...which is correct, as sda is the only partitioned device.
The basic mechanism here is the nameref: declare -n name1=name2 will allow one to refer to the variable name2 under the name name1, including updates or assignments, until unset -n name1 is performed.
Not sure I understood what you are trying to do but this is an example that you can, maybe, start with:
#!/bin/bash
list1=(a b c)
list2=(j k l)
for x in ${list1[#]}; do
for y in ${list2[#]}; do
tmplist+=(${x}${y})
done
cmd="declare -a list_${x}=(${tmplist[#]})"
eval $cmd
unset tmplist
done
echo "list_a: ${list_a[#]}"
echo "list_b: ${list_b[#]}"
echo "list_c: ${list_c[#]}"
It shows how to create arrays which names are computed from other variables, how to unset a temporary array...
I would create two types of arrays. One for containing disk names, then different arrays for each disk for storing partition specific information.
#!/usr/bin/env bash
while read -r l; do
t="${l#* }"
if [[ $t == "disk" ]]; then
# disks list contains all the disk names
disks+="${l% *} "
elif [[ $t == "part" ]]; then
# for each disk 'XXX', a separate partitions array 'parts_XXX' is created
[[ $l =~ [^a-z0-9]*([a-z]*)([0-9]*)\ ]] && d="${BASH_REMATCH[1]}" && p="${BASH_REMATCH[2]}"
eval parts_$d+=\"my-value=\$d\$p \" # append to partitians array
else
echo "unknown type $t in $l"
exit 1
fi
done <<< $(lsblk -n --output=NAME,TYPE | tr -s ' ')
# arrays are created. now iterate them
for i in ${disks[#]}; do
echo "iterating partitions of disk: $i"
# following is the name of the current disks partitions array
var=parts_$i[#]
# iterate partitians array of the current disk
for j in ${!var}; do
echo ">> $j"
done
done

Resources