Loop through dynamically generated array in bash - arrays

I'm trying to write a script that performs actions of files with different extensions. To make it as easy to add different actions as possible, the extensions are read from an array, files are found via the "find" command, and results returned to a dynamically generated array named after the file extension that was searched.
To add a new extension to search for I can simply add to the file_ext array.
I create the array like this:
file_exts=("dsd" "dsdd" "dmesg")
for ext in "${file_exts[#]}"
do
echo "Finding $ext files"
eval var="exts_$ext"
declare -a $var="($( find "$dirs_target" -name "*.$ext"))"
done
The arrays are created correctly, and I can manually echo "${exts_dsd[0]} ${exts_dsd[1]}" and see the entries, However, I can't find a way of looping through each entry in the dynamically assigned arrays.
I have tried a few combinations using eval, and I can print out the first entry in the array, IE just referencing "$exts_dsd" Here are two things I've already tried:
for varname in "${!exts_#}"
do
for entry in ${varname[#]}
do
echo "$varname : $entry"
done
eval value=\$${varname[#]}
echo "$varname=$value"
done
How can I loop through each entry in the above for loop, so I can print out all the entries in all the dynamically created arrays?
Here is a complete test script:
#! /bin/bash
file_exts=("dsd" "dsdd" "dmesg")
dirs_target="/tmp/arraytest/"
echo "Creating $dirs_target"
if [[ ! -d "$dirs_target" ]]; then
if ! mkdir "$dirs_target"; then
echo "Couldn't create temp dir"
exit 1
fi
fi
echo "Creating test files"
for tmpfile in $( seq 0 5 )
do
echo -e "\tCreating $dirs_target$tmpfile.dsd"
if ! touch "$dirs_target/$tmpfile.dsd"; then
echo "Coudn't create $dirs_target/test$tmpfile.dsd"
exit 1
fi
done
echo ""
echo "-----Finding Files-----"
for ext in "${file_exts[#]}"
do
echo "Finding $ext files"
eval var="exts_$ext"
declare -a $var="($( find "$dirs_target" -name "*.$ext"))"
done
echo ""
echo "-----File Extensions-----"
for varname in "${!exts_#}"
do
for entry in ${varname[#]}
do
echo "$varname : $entry"
done
eval value=\$${varname[#]}
#echo "$varname=$value"
done
echo ""
echo "Finishing."
rm -rf "$dirs_target"

To loop over the entries, you have to use the same trick as when creating them: just store the variable name in a variable. The point is to include the [#] index, too, which will be correctly recognized in the indirection:
for varname in "${!exts_#}" ; do
arr=$varname'[#]'
for entry in "${!arr}" ; do
echo "$varname : $entry"
done
done
Also note that eval isn't needed in
# eval var="exts_$ext"
var=exts_$ext # works even better!

I've found the answer. I had the eval statement slightly wrong.
echo "-----File Extensions-----"
for varname in "${!exts_#}"
do
echo "varname=$varname"
eval testvalue="\${$varname[#]}"
for entry in $testvalue
do
echo -e "\tFile: $entry"
done
done
As a bonus, I've also figured out how to add to a dynamically created array
var="table_$entry"
declare -a $var
while read -r line
do
eval $var+=\(\'"$line"\'\)
done < "$dirs_table"

Related

How to replace an array variable of one shell script from another shell script's array variable?

I have two shell scripts, fruits_original.sh and appending_fruits.sh. In the fruits_original.sh I have one array variable: fruits=('Apple' 'Mango' 'Guava').
What I want to do is I have to write a shell script appending_fruits.sh that will take an argument some new fruits name is Orange and will append that new fruit name to the fruits_original.sh fruits array variable.
After script run fruits array should be remain an array only and its value should be fruits=('Apple' 'Mango' 'Guava' 'Orange').
The file fruits_original.sh has this. Below is the appending_fruits.sh script by this my variable is changing into this fruits= ('Apple' 'Mango' 'Guava' 'Orange'). But when I am trying to do echo "${fruits[#]}" I am getting this error:
line 1: syntax error near unexpected token `('
Any luck ?
fruits= ('Apple' 'Mango' 'Guava')
echo "${fruits[#]}"
declare -a var=$(awk -F'=' '/^fruits=/ {print $2}' fruits_original.sh)
echo "${var[#]}"
var[${#var[#]}]='Orange'
joined=$(printf " '%s'" "${var[#]}")
echo ${joined:1}
echo "${joined[#]}"
sed -i "s/fruits=.*/fruits= ($( echo ${joined:1})) /" fruits_original.sh
Do not modify the script file. Instead, create another file and source the dynamic data from it. I have chosen the location of configuration to be in /tmp directory.
# fruits_original.sh
fruits=()
if [[ -e /tmp/fruits_original.rc ]]; then
. /tmp/fruits_original.rc
fi
some stuff
Then generate the config file. Use declare -p to safely output properly quoted variables.
# appending_fruits.sh
fruits=()
if [[ -e /tmp/fruits_original.rc ]]; then
. /tmp/fruits_original.rc
fi
fruits+=("new fruit")
decalre -p fruits > /tmp/fruits_original.rc
Put a uuid inside fruits_original.sh to recognize where is your snippet that you want to work with.
# fruits_original.sh
# snip 419d0df3-5f08-4511-ad5a-ad24db45aa6c
fruits=()
# snip 419d0df3-5f08-4511-ad5a-ad24db45aa6c
some stuff
Then extract the relevant parts with sed or other tool, declare "$part" it into a variable, append normally and then capture output from declare -p and replace the content between the marks again.
If not going with any of the above and this is only a very toy example to test some stuff, you could:
# read the line from another script
declare "$(sed '/fruits=/!d' fruits_original.sh)"
# append element
fruits+=(Orange)
# create source-able output
new="$(declare -p fruits)"
# remove declare -- in front
new="fruits=${new%*fruits=}"
# Replace the line with declare -p output.
sed -i "s/fruits=.*/fruits=$new/" fruits_original.sh
Notes:
var[${#var[#]}]='Orange' - just var+=(Orange). No need for ${#.
$( echo ${joined:1}) is a useless use of echo (unless you want word splitting and filename expansion).
check your scripts with https://shellcheck.net
fruits= ( is not an assignment and will run a subshell and could cause syntax error. There is no space in assignment around =.
declare -a var=$( - var is not an array (or, it's an array with one element).

can't print array values in bash

I need to generate ipset rules dynamically. so i created following script and having error try to get the urls from array.
#!/bin/bash
# urlList
allow_SMTP_OUT_URLS=(mtp.sendgrid.net)
allow_HTTP_OUT_URLS=(archive.mariadb.org)
allow_HTTPS_OUT_URLS=(ppa.launchpad.net repo.mongodb.org www.google.com)
allow_SSH_OUT_URLS=(bitbucket.org)
ipsetNames=(allow_HTTP_OUT allow_HTTPS_OUT allow_SMTP_OUT allow_SSH_OUT)
for ipSET in "${ipsetNames[#]}"
do
ipset create -exist $ipSET hash:ip timeout 86400 comment family inet
chkIPSETexsit="$(ipset list -n | grep $ipSET)"
# Check / Adding IPSET rules
if [ -z "$chkIPSETexsit" ]; then
echo "$ipSET is empty"
exit 1
else
echo "$ipSET is present. Adding URLs to ipset"
urlList=$ipSET
urlList+="_URLS"
echo "urlList: $urlList"
echo URLs: ${(echo $urlList)[#]}
fi
done
its gives error as bad substitution
allow_HTTP_OUT is present. Adding URLs to ipset
urlList: allow_HTTP_OUT_URLS
/root/salt-cron-scripts/test2.sh: line 30: ${(echo $urlList)[#]}: bad substitution
Any suggestions to correct this please
You can use a nameref declared with declare -n. This gives you a slightly better and more intuitive manipulation than using indirect parameter expansion, such as using: ${urls[#]}, ${urls[0]}, ${urls[5]//a/b} etc.
#!/bin/bash
# urlList
allow_SMTP_OUT_URLS=(mtp.sendgrid.net)
allow_HTTP_OUT_URLS=(archive.mariadb.org)
allow_HTTPS_OUT_URLS=(ppa.launchpad.net repo.mongodb.org www.google.com)
allow_SSH_OUT_URLS=(bitbucket.org)
ipsetNames=(allow_HTTP_OUT allow_HTTPS_OUT allow_SMTP_OUT allow_SSH_OUT)
for ipSET in "${ipsetNames[#]}"
do
ipset create -exist $ipSET hash:ip timeout 86400 comment family inet
chkIPSETexsit="$(ipset list -n | grep $ipSET)"
# Check / Adding IPSET rules
if [ -z "$chkIPSETexsit" ]; then
echo "$ipSET is empty"
exit 1
else
echo "$ipSET is present. Adding URLs to ipset"
urlList=${ipSET}_URLS
echo "urlList: $urlList"
declare -n urls=$urlList
echo "URLs: ${urls[#]}"
fi
done
You need to use indirect parameter expansion to expand the array named by urlList.
allow_SMTP_OUT_URLS=(mtp.sendgrid.net)
allow_HTTP_OUT_URLS=(archive.mariadb.org)
allow_HTTPS_OUT_URLS=(ppa.launchpad.net repo.mongodb.org www.google.com)
allow_SSH_OUT_URLS=(bitbucket.org)
ipsetNames=(allow_HTTP_OUT allow_HTTPS_OUT allow_SMTP_OUT allow_SSH_OUT)
for ipSET in "${ipsetNames[#]}"
do
ipset create -exist "$ipSET" hash:ip timeout 86400 comment family inet
chkIPSETexsit="$(ipset list -n | grep $ipSET)"
# Check / Adding IPSET rules
if [ -z "$chkIPSETexsit" ]; then
echo "$ipSET is empty"
exit 1
fi
echo "$ipSET is present. Adding URLs to ipset"
urlList=${ipSET}_URLS
echo "urlList: $urlList"
urlList=$urlList[#]
echo "URLs: ${!urlList}"
done

Shell (sh) script array

Have been trying to find away to do array in sh shell script. Other than BASH etc. have not found much, other than arrays not supported in sh.
Here's what I've come up with using setvar and eval. Is there a better way? Any way to eliminate setvar and/or eval?
#!/bin/sh
# FreeBSD 11.1
# Kind of an array workaround.
echo -e "Simulated array creation and element assignment using dynamic index."
array() {
i=0
for x in $2
do
setvar ${1}_${i} $x # any way to do without setvar?
i=$((i+1))
done
setvar ${1}_cnt $i
}
array "my_arry" "a b c"
echo -e "\nSimulated array element access using dynamic index."
i=0
while [ $i -lt $my_arry_cnt ]
do
eval aev=\${my_arry_${i}} # any way to do without eval?
i=$((i+1))
echo $aev
done
echo -e "\nSimulated array element access using static index."
echo ${my_arry_0}
echo ${my_arry_1}
echo ${my_arry_2}

Bash script - how to fill array?

Let's say I have this directory structure:
DIRECTORY:
.........a
.........b
.........c
.........d
What I want to do is: I want to store elements of a directory in an array
something like : array = ls /home/user/DIRECTORY
so that array[0] contains name of first file (that is 'a')
array[1] == 'b' etc.
Thanks for help
You can't simply do array = ls /home/user/DIRECTORY, because - even with proper syntax - it wouldn't give you an array, but a string that you would have to parse, and Parsing ls is punishable by law. You can, however, use built-in Bash constructs to achieve what you want :
#!/usr/bin/env bash
readonly YOUR_DIR="/home/daniel"
if [[ ! -d $YOUR_DIR ]]; then
echo >&2 "$YOUR_DIR does not exist or is not a directory"
exit 1
fi
OLD_PWD=$PWD
cd "$YOUR_DIR"
i=0
for file in *
do
if [[ -f $file ]]; then
array[$i]=$file
i=$(($i+1))
fi
done
cd "$OLD_PWD"
exit 0
This small script saves the names of all the regular files (which means no directories, links, sockets, and such) that can be found in $YOUR_DIR to the array called array.
Hope this helps.
Option 1, a manual loop:
dirtolist=/home/user/DIRECTORY
shopt -s nullglob # In case there aren't any files
contentsarray=()
for filepath in "$dirtolist"/*; do
contentsarray+=("$(basename "$filepath")")
done
shopt -u nullglob # Optional, restore default behavior for unmatched file globs
Option 2, using bash array trickery:
dirtolist=/home/user/DIRECTORY
shopt -s nullglob
contentspaths=("$dirtolist"/*) # This makes an array of paths to the files
contentsarray=("${contentpaths[#]##*/}") # This strips off the path portions, leaving just the filenames
shopt -u nullglob # Optional, restore default behavior for unmatched file globs
array=($(ls /home/user/DIRECTORY))
Then
echo ${array[0]}
will equal to the first file in that directory.

Looping through an Array in bash

I am currently attempting to create a bash script that will check inside of each users /Library/Mail folder to see if a folder named V2 exists. The script should create an array with each item in the array being a user and then iterate through each of these users checking their home folder for the above captioned contents. This is what I have so far:
#!/bin/bash
cd /Users
array=($(ls))
for i in ${array[#]}
do
if [ -d /$i/Library/Mail/V2 ]
then
echo "$i mail has been upgraded."
else
echo "$i FAIL"
fi
done
Populating your array from the output of ls is going to make for serious problems when you have a username with spaces. Use a glob expression instead. Also, using [ -d $i/... ] will similarly break on names with spaces -- either use [[ -d $i/... ]] (the [[ ]] construct has its own syntax rules and doesn't require quoting) or [ -d "$i/..." ] (with the quotes).
Similarly, you need to double-quote "${array[#]}" to avoid string-splitting from splitting names with spaces in two, as follows:
cd /Users
array=(*)
for i in "${array[#]}"; do
if [[ -d $i/Library/Mail/V2 ]]; then
echo "$i mail has been upgraded."
else
echo "$i FAIL"
fi
done
That said, you don't really need an array here at all:
for i in *; do
...check for $i/Library/Mail/V2...
done

Resources