Populating bash array declare from awk fails - arrays

I am trying to make an array of the partitions contained in a device, here's my attempt, but it does not seem to work.
#!/bin/bash
DISK=sda
declare -a PARTS=("$(awk -v disk=$DISK '$4 ~ disk {printf "[" $2 "]=\"" $3 "\" "}' /proc/partitions)")
by themselves, the commands seem to work:
$ DISK=sda
$ awk -v disk=$DISK '$4 ~ disk {printf "[" $2 "]=\"" $3 "\" "}' /proc/partitions
[0]="7987200" [1]="7986408"
$ declare -a PARTS=([0]="7987200" [1]="7986408" )
$ echo ${PARTS[0]}
7987200
$ echo ${PARTS[1]}
7986408
but not combined:
$ DISK=sda
$ declare -a PARTS=($(awk -v disk=$DISK '$4 ~ disk {printf "[" $2 "]=\"" $3 "\" "}' /proc/partitions))
$ echo ${PARTS[0]}
[0]="7987200"
$ echo ${PARTS[1]}
[1]="7986408"
Any help greatly appreciated!

For the evaluation to proceed with the declare command, you must pass a string with the whole content embraced by parentheses, which is the proper syntax for array declaration in bash (declare -a VAR=([key]=val ...)). For example, your command should be:
$ DISK=sda
$ declare -a PARTS='('$(awk -v disk=$DISK \
'$4 ~ disk {printf "[" $2 "]=\"" $3 "\" "}' /proc/partitions)')'
You may as well check out what the proper syntax is by simply dumping the array. This is the result after running the awk command in my machine:
$ declare -p PARTS
declare -a PARTS='([0]="488386584" [1]="25165824" [2]="16777216" \
[3]="8388608" [4]="438053895")'

You don't need awk for this; the code is much cleaner in pure bash:
DISK=sda
declare -a parts # Optional
while read maj min blks name; do
[[ $name =~ ^$DISK ]] && parts[$min]=$blks
done < /proc/partitions

Related

bad substitution when creating nested array variables in bash

I don't know what I am doing and could use some assistance with my script.
$ ./mysql.sh LOCALIP 'SELECT LOCALIP FROM Host'
mysql.sh
#!/bin/bash
source $PWD/data/login
mapfile -t "$1" < <(mysql -N ""$DB"" -h""$HOST"" -u""$USER"" -p""$PASS"" -se "$2")
echo ${$1[0]}
echo ${$1[1]}
echo ${$1[2]}
echo ${$1[3]}
fi
Output
[シ]owner#gwpi ~/scriptdir $./mysql.sh LOCALIP 'SELECT LOCALIP FROM Host'
./mysql.sh: line 10: ${$1[0]}: bad substitution
Simply replace the varible $1 with var here.It works.
$ mapfile -t var < <(mysql -N ""$DB"" -h""$HOST"" -u""$USER"" -p""$PASS"" -se "$2")
$ echo" ${var[#]}"
Original script has two problems.
You cannot change $1 arg in this way.Right style refers to this
should be better using more meaningful variables than $1

Pass multiple arrays as arguments to a Bash script?

I've looked, but have only seen answers to one array being passed in a script.
I want to pass multiple arrays to a bash script that assigns them as individual variables as follows:
./myScript.sh ${array1[#]} ${array2[#]} ${array3[#]}
such that: var1=array1 and var2=array2 and var3=array3
I've tried multiple options, but doing variableName=("$#") combines all arrays together into each variable. I hope to have in my bash script a variable that represents each array.
The shell passes a single argument vector (that is to say, a simple C array of strings) off to a program being run. This is an OS-level limitation: There exists no method to pass structured data between two programs (any two programs, written in any language!) in an argument list, except by encoding that structure in the contents of the members of this array of C strings.
Approach: Length Prefixes
If efficiency is a goal (both in terms of ease-of-parsing and amount of space used out of the ARG_MAX limit on command-line and environment storage), one approach to consider is prefixing each array with an argument describing its length.
By providing length arguments, however, you can indicate which sections of that argument list are supposed to be part of a given array:
./myScript \
"${#array1[#]}" "${array1[#]}" \
"${#array2[#]}" "${array2[#]}" \
"${#array3[#]}" "${array3[#]}"
...then, inside the script, you can use the length arguments to split content back into arrays:
#!/usr/bin/env bash
array1=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
array2=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
array3=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
declare -p array1 array2 array3
If run as ./myScript 3 a b c 2 X Y 1 z, this has the output:
declare -a array1='([0]="a" [1]="b" [2]="c")'
declare -a array2='([0]="X" [1]="Y")'
declare -a array3='([0]="z")'
Approach: Per-Argument Array Name Prefixes
Incidentally, a practice common in the Python world (particularly with users of the argparse library) is to allow an argument to be passed more than once to amend to a given array. In shell, this would look like:
./myScript \
"${array1[#]/#/--array1=}" \
"${array2[#]/#/--array2=}" \
"${array3[#]/#/--array3=}"
and then the code to parse it might look like:
#!/usr/bin/env bash
declare -a args array1 array2 array3
while (( $# )); do
case $1 in
--array1=*) array1+=( "${1#*=}" );;
--array2=*) array2+=( "${1#*=}" );;
--array3=*) array3+=( "${1#*=}" );;
*) args+=( "$1" );;
esac
shift
done
Thus, if your original value were array1=( one two three ) array2=( aye bee ) array3=( "hello world" ), the calling convention would be:
./myScript --array1=one --array1=two --array1=three \
--array2=aye --array2=bee \
--array3="hello world"
Approach: NUL-Delimited Streams
Another approach is to pass a filename for each array from which a NUL-delimited list of its contents can be read. One chief advantage of this approach is that the size of array contents does not count against ARG_MAX, the OS-enforced command-line length limit. Moreover, with an operating system where such is available, the below does not create real on-disk files but instead creates /dev/fd-style links to FIFOs written to by subshells writing the contents of each array.
./myScript \
<( (( ${#array1[#]} )) && printf '%s\0' "${array1[#]}") \
<( (( ${#array2[#]} )) && printf '%s\0' "${array2[#]}") \
<( (( ${#array3[#]} )) && printf '%s\0' "${array3[#]}")
...and, to read (with bash 4.4 or newer, providing mapfile -d):
#!/usr/bin/env bash
mapfile -d '' array1 <"$1"
mapfile -d '' array2 <"$2"
mapfile -d '' array3 <"$3"
...or, to support older bash releases:
#!/usr/bin/env bash
declare -a array1 array2 array3
while IFS= read -r -d '' entry; do array1+=( "$entry" ); done <"$1"
while IFS= read -r -d '' entry; do array2+=( "$entry" ); done <"$2"
while IFS= read -r -d '' entry; do array3+=( "$entry" ); done <"$3"
Charles Duffy's response works perfectly well, but I would go about it a different way that makes it simpler to initialize var1, var2 and var3 in your script:
./myScript.sh "${#array1[#]} ${#array2[#]} ${#array3[#]}" \
"${array1[#]}" "${array2[#]}" "${array3[#]}"
Then in myScript.sh
#!/bin/bash
declare -ai lens=($1);
declare -a var1=("${#:2:lens[0]}") var2=("${#:2+lens[0]:lens[1]}") var3=("${#:2+lens[0]+lens[1]:lens[2]}");
Edit: Since Charles has simplified his solution, it is probably a better and more clear solution than mine.
Here is a code sample, which shows how to pass 2 arrays to a function. There is nothing more than in previous answers except it provides a full code example.
This is coded in bash 4.4.12, i.e. after bash 4.3 which would require a different coding approach. One array contains the texts to be colorized, and the other array contains the colors to be used for each of the text elements :
function cecho_multitext () {
# usage : cecho_multitext message_array color_array
# what it does : Multiple Colored-echo.
local -n array_msgs=$1
local -n array_colors=$2
# printf '1: %q\n' "${array_msgs[#]}"
# printf '2: %q\n' "${array_colors[#]}"
local i=0
local coloredstring=""
local normalcoloredstring=""
# check array counts
# echo "msg size : "${#array_msgs[#]}
# echo "col size : "${#array_colors[#]}
[[ "${#array_msgs[#]}" -ne "${#array_colors[#]}" ]] && exit 2
# build the colored string
for msg in "${array_msgs[#]}"
do
color=${array_colors[$i]}
coloredstring="$coloredstring $color $msg "
normalcoloredstring="$normalcoloredstring $msg"
# echo -e "coloredstring ($i): $coloredstring"
i=$((i+1))
done
# DEBUG
# echo -e "colored string : $coloredstring"
# echo -e "normal color string : $normal $normalcoloredstring"
# use either echo or printf as follows :
# echo -e "$coloredstring"
printf '%b\n' "${coloredstring}"
return
}
Calling the function :
#!/bin/bash
green='\E[32m'
cyan='\E[36m'
white='\E[37m'
normal=$(tput sgr0)
declare -a text=("one" "two" "three" )
declare -a color=("$white" "$green" "$cyan")
cecho_multitext text color
Job done :-)
I do prefer using base64 to encode and decode arrays like:
encode_array(){
local array=($#)
echo -n "${array[#]}" | base64
}
decode_array(){
echo -n "$#" | base64 -d
}
some_func(){
local arr1=($(decode_array $1))
local arr2=($(decode_array $2))
local arr3=($(decode_array $3))
echo arr1 has ${#arr1[#]} items, the second item is ${arr1[2]}
echo arr2 has ${#arr2[#]} items, the third item is ${arr2[3]}
echo arr3 has ${#arr3[#]} items, the here the contents ${arr3[#]}
}
a1=(ab cd ef)
a2=(gh ij kl nm)
a3=(op ql)
some_func "$(encode_array "${a1[#]}")" "$(encode_array "${a2[#]}")" "$(encode_array "${a3[#]}")"
The output is
arr1 has 3 items, the second item is cd
arr2 has 4 items, the third item is kl
arr3 has 2 items, the here the contents op ql
Anyway, that will not work with values that have tabs or spaces. If required, we need a more elaborated solution. something like:
encode_array()
{
for item in "$#";
do
echo -n "$item" | base64
done | paste -s -d , -
}
decode_array()
{
local IFS=$'\2'
local -a arr=($(echo "$1" | tr , "\n" |
while read encoded_array_item;
do
echo "$encoded_array_item" | base64 -d;
echo "$IFS"
done))
echo "${arr[*]}";
}
test_arrays_step1()
{
local IFS=$'\2'
local -a arr1=($(decode_array $1))
local -a arr2=($(decode_array $2))
local -a arr3=($(decode_array $3))
unset IFS
echo arr1 has ${#arr1[#]} items, the second item is ${arr1[1]}
echo arr2 has ${#arr2[#]} items, the third item is ${arr2[2]}
echo arr3 has ${#arr3[#]} items, the here the contents ${arr3[#]}
}
test_arrays()
{
local a1_2="$(echo -en "c\td")";
local a1=("a b" "$a1_2" "e f");
local a2=(gh ij kl nm);
local a3=(op ql );
a1_size=${#a1[#])};
resp=$(test_arrays_step1 "$(encode_array "${a1[#]}")" "$(encode_array "${a2[#]}")" "$(encode_array "${a3[#]}")");
echo -e "$resp" | grep arr1 | grep "arr1 has $a1_size, the second item is $a1_2" || echo but it should have only $a1_size items, with the second item as $a1_2
echo "$resp"
}
Based on the answers to this question you could try the following.
Define the arrays as variable on the shell:
array1=(1 2 3)
array2=(3 4 5)
array3=(6 7 8)
Have a script like this:
arg1=("${!1}")
arg2=("${!2}")
arg3=("${!3}")
echo "arg1 array=${arg1[#]}"
echo "arg1 #elem=${#arg1[#]}"
echo "arg2 array=${arg2[#]}"
echo "arg2 #elem=${#arg2[#]}"
echo "arg3 array=${arg3[#]}"
echo "arg3 #elem=${#arg3[#]}"
And call it like this:
. ./test.sh "array1[#]" "array2[#]" "array3[#]"
Note that the script will need to be sourced (. or source) so that it is executed in the current shell environment and not a sub shell.

How to split input string into multiple variable

I'm working on shell script and trying to split user input into multiple variable and use them at different places.
User input is not fixed so can't really assign fixed number of variable, input is separated by comma ,
./user_input.ksh -string /m01,/m02,/m03
#!/bin/ksh
STR=$2
function showMounts {
echo "$STR"
arr=($(tr ',' ' ' <<< "$STR"))
printf "%s\n" "$(arr[#]}"
for x in "$(arr[#]}"
do
free_space=`df -h "$x" | grep -v "Avail" | awk '{print $4}'`
echo "$x": free_space "$free_space"
done
#total_free_space = <total of $free_space>
#echo "$total_free_space"
}
Basically $STR* variable value is filesystem mount points
Host output if run separate df -h command
$ df -h /m01 | grep -v "Avail" | awk '{print $4}'
***Output***
150
Current problems:
(working)1. How to get free space available for each /m* using df -h?
Easiest thing to do is to use shell array here like this:
#!/bin/ksh
str='/m01,/m02,/m03'
arr=($(tr ',' ' ' <<< "$str"))
printf "%s\n" "${arr[#]}"
Output:
/m01
/m02
/m03
To read elements individually you can use:
"${arr[0]}"
"${arr[1]}"
...
Update: Here is your corrected script:
#!/bin/ksh
STR="$2"
arr=($(tr ',' ' ' <<< "$STR"))
printf "<%s>\n" "${arr[#]}"
for x in "${arr[#]}"; do
echo "$x"
free_space=`df -h "$x" | awk '!/Avail/{print $4}'`
echo "$free_space"
done
you can try,
#!/bin/ksh
STR=/m01,/m02,/m03
read STR1 STR2 STR3 <<<`echo $STR | awk 'BEGIN{FS=","; OFS=" "}{$1=$1; print}'`
echo $STR1 - $STR2 - $STR3
you get:
/m01 - /m02 - /m03
A variation on the theme:
# cat user_input.ksh
#!/bin/ksh
c=1
for i in $(echo ${#} | tr "," " ")
do
eval STR$c="$i"
((c=c+1))
done
printf "\$STR1 = %s; \$STR2 = %s; \$STR3 = %s; ...\n" "$STR1" "$STR2" "$STR3"
Which gives you:
# ksh ./user_input.ksh /m01,/m02,/m03,/m04
$STR1 = /m01; $STR2 = /m02; $STR3 = /m03; ...
Hope that helps..
--ab1
$ cat tst.sh
str='/m01,/m02,/m03'
IFS=,
set -- $str
for i
do
echo "$i"
done
$ ./tst.sh
/m01
/m02
/m03
Don't use all-upper-case for variable names unless you are going to export them (by convention and to avoid clashes with built in names like HOME, PATH, IFS, etc.).
For your overall script, you should simply be doing something like this:
df -h "${str//,/ }" | awk '/^ /{print $5, $3; sum+=$3} END{print sum}'
depending on what your df -h output looks like and what you're final output is supposed to be.

Bash - Pointer to Value in Associate Array?

Is there a way in Bash to make a pointer to the value of a key in an associate array? Like this:
declare -A mapp
mapp=( ["key"]="${value}" )
for k in "${!mapp[#]}"; do
pointer="${mapp["${k}"]}" # How do I do this?
done
Usually, you do not need to use a pointer, but I'm curious to see if there's a way to make one.
In a simpler situation (i.e., for normal/string variables), I would make a pointer like this:
pointer=b
read -p "Enter something: " b
eval pointer=\$${pointer}
How would I do this for an associate array? This doesn't work (skip the strikethroughed code):
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp["${k}"]
read -p "Enter ${k}: " new
eval v=\$${v} # Doesn't work
done
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp["${k}"]
read -p "Enter ${k}: " k
eval v=\$${v} # Doesn't work
done
This doesn't work either (skip the strikethroughed code):
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp
read -p "Enter ${k}: " new
eval v=\$${v["${k}"]} # Doesn't work (and has terrible readability)
done
declare -A mapp
mapp=( ["first"]="${a}" ["second"]="${b}" )
for k in "${!mapp[#]}"; do
v=mapp
read -p "Enter ${k}: " k
eval v=\$${v["${k}"]} # Doesn't work (and has terrible readability)
done
In bash 4.3, you can use a nameref:
$ mapp=([key]=value)
$ declare -n x=mapp[key] # NO dollar sign!
$ x=7
$ echo ${mapp[key]}
7
Before 4.3, you need to use the declare command differently to do the indirection.
$ mapp=([key]=value)
$ x=mapp[key] # NO dollar sign!
$ declare "$x=7"
$ echo ${mapp[key]}
7
No problem:
$ declare -A ary=([foo]=bar [baz]=qux)
$ key=foo
$ pointer="ary[$key]"
$ echo "$pointer"
ary[foo]
$ echo "${!pointer}"
bar
A "pointer" in this sense is an indirect variable

How to pass bash parameter to awk script?

I have awk file:
#!/bin/awk -f
BEGIN {
}
{
filetime[$'$colnumber']++;
}
END {
for (i in filetime) {
print filetime[i],i;
}
}
And bash script:
#!/bin/bash
var1=$1
awk -f myawk.awk
When I run:
ls -la | ./countPar.sh 5
I receive error:
ls -la | ./countPar.sh 5
awk: myawk.awk:6: filetime[$'$colnumber']++;
awk: myawk.awk:6: ^ invalid char ''' in expression
Why? $colnumber must be replaced with 5, so awk should read 5th column of ls ouput.
Thanks.
You can pass variables to your awk script directly from the command line.
Change this line:
filetime[$'$colnumber']++;
To:
filetime[colnumber]++;
And run:
ls -al | awk -f ./myawk.awk -v colnumber=5
If you really want to use a bash wrapper:
#!/bin/bash
var1=$1
awk -f myawk.awk colnumber=$var1
(with the same change in your script as above.)
If you want to use environment variables use:
#!/bin/bash
export var1=$1
awk -f myawk.awk
and:
filetime[ENVIRON["var1"]]++;
(I really don't understand what the purpose of your awk script is though. The last part could be simplified to:
END { print filetime[colnumber],colnumber; }
and parsing the output of ls is generally a bad idea.)
The easiest way to do it:
#!/bin/bash
var=$1
awk -v colnumber="${var}" -f /your/script
But within your awk script, you don't need the $ in front of colnumber.
HTH
Passing 3 variable to script myscript.sh
var1 is the column number on which condition has set.
While var2 & var3 are input and temp file.
#!/bin/ksh
var1=$1
var2=$2
var3=$3
awk -v col="${var1}" -f awkscript.awk ${var2} > $var3
mv ${var3} ${var2}
execute it like below -
./myscript.sh 2 file.txt temp.txt

Resources