I am still quite new to bash scripting and I am somehow stuck.
I am looking for a clean and easy way to parse a settingsfile, where some special (and known) settings are arrays.
So the settings file looks like this.
foo=(1 2 3 4)
bar="foobar"
The best solution I came up with so far is:
#!/bin/bash
while IFS== read -r k v; do
if [ "$k" = "foo" ]
then
IFS=' ' read -r -a $k <<< "$v"
else
declare "$k"="$(echo $v | tr -d '""')"
fi
done < settings.txt
But I am obviously mixing up array types. As far is I understood and tried out for the bar="foobar" part this actually declares an array, and could be accessed by echo ${bar[0]} but as well as echo $bar. So I thought this would be a indexed array, but the error log clearly states something different:
cannot convert associative to indexed array
Would be glad if somebody could explain me a little bit how to find a proper solution.
Is it safe for you to just source the file?
. settings.txt
That will insert all the lines of the file as if they were lines of your current script. Obviously, there are security concerns if the file isn't as secure as the script file itself.
Related
I have a JSON file which holds data like this: "path/to/git/directory/location": "path/to/local/location". A minimum example of the file might be this:
{
"${HOME}/dotfiles/.bashrc": "${HOME}/.bashrc",
"${HOME}/dotfiles/.atom/": "${HOME}/.atom/"
}
I have a script that systematically reads the above JSON (called locations.json) and creates an array, and then prints elements of the array that are directories. MWE:
#!/usr/bin/env bash
unset sysarray
declare -A sysarray
while IFS=: read -r field data
do
sysarray["${field}"]="${data}"
done <<< $(sed '/^[{}]$/d;s/\s*"/"/g;s/,$//' locations.json)
for file in "${sysarray[#]}"
do
if [ -d "${file}" ]
then
echo "${file}"
fi
done
However, this does not print the directory (i.e., ${HOME}/.atom).
I don't understand why this is happening, because
I have tried creating an array manually (i.e., not from a JSON) and checking if its elements are directories, and that works fine.
I have tried echoing each element in the array into a temporary file and reading each line in the file to see if it was a product of how the information was stored in the array, but no luck.
I have tried adding | tr -d "[:blank:]" | tr -d '\"' after using sed on the JSON (to see if it was a product of unintended whitespace or quotes), but no luck.
I have tried simply running [ -d "${HOME}/.atom/" ] && echo '.atom is a directory', and that works (so indeed it is a directory). I'm unsure what might be causing this.
Help on this would be great!
You could use a tool to process json files properly, which will deal with any valid json.
#!/usr/bin/env bash
unset sysarray
declare -A sysarray
while IFS="=" read -r field data
do
sysarray["${field}"]=$(eval echo "${data}")
done <<< $(jq -r 'keys[] as $k | "\($k)=\(.[$k])"' locations.json)
for file in "${sysarray[#]}"
do
if [ -d "${file}" ]
then
echo "${file}"
fi
done
Another problem is that, once the extra quote signs are properly processed, we have a literal ${HOME} that is not expanded. The only solution I came is using eval to force the expansion. It is not the nicest way, but right now I cannot find a better solution.
I am trying to create files from an array called columnHeaders[]
Within the array, I have test values that are currently:
id, source, EN, EN-GB, French-FR, French-DE
When I run my code I get
EN
EN.xml
EN-GB
EN-GB.xml
French-FR
French-FR.xml
French-DE
.xmlch-DE
NOTE that the FRENCH-DE filename gets morphed into .xmlch-DE
Why is this? I can't for the life of me figure out whey only that file ends up looking like this. It's driving me crazy!
Thanks for any help.
below is the snippet of my code that is causing me problems:
# take all the languages and either find the files or create new files for them. The language options
# should be stored in the columnHeader array in positions 3 - n
# cycle through all the output languages (so exclude "id, source' inputs)
# in my example, numWordsInLine is 6
c=2
while [ $c -lt $numWordsInLine ]; do
OUTPUT_LANG="${columnHeaders[$c]}"
echo "$OUTPUT_LANG"
OUTPUT_FILE="$OUTPUT_LANG".xml
# HERE'S WHERE YOU CAN SEE OUTPUT_FILE IS WRONG FOR FRENCH_DE
echo "$OUTPUT_FILE"
OUTPUT_BAK="$OUTPUT_LANG".bak
TMP_FILE="~tmp.xml"
if [ -f "$OUTPUT_BAK" ]; then
rm "$OUTPUT_BAK"
fi
# make a backup of the original language.xml file in case of program error or interruption
if [ -f "$OUTPUT_FILE" ]; then
mv "$OUTPUT_FILE" "$OUTPUT_BAK"
fi
if [ -f "$TMP_FILE" ]; then
rm "$TMP_FILE"
fi
c=$(expr $c + 1)
done
I'm betting that you are reading that line of data from a file with DOS newlines.
I'm also betting that the contents of the variable are "fine" but include a trailing carriage return.
Try printf %q\\n "$OUTPUT_FILE" or echo "$OUTPUT_FILE" | cat -v to see.
Then use something like dos2unix on the file to convert it.
Extra (unrelated) comments:
There's also no reason to use expr. ((c++)) will do what you want.
You could even turn the loop itself into for ((c=2;c < $numWordsInLine; c++)); do if you wanted to.
$numWordsInLine is also unnecessary if $columnHeaders is already split into the right "words" since you can use ${#columnHeaders} to get the length.
I am able to read file into a regular array with a single statement:
local -a ary
readarray -t ary < $fileName
Not happening is reading a file into assoc. array.
I have control over file creation and so would like to do as simply as possible w/o loops if possible at all.
So file content can be following to be read in as:
keyname=valueInfo
But I am willing to replace = with another string if cuts down on code, especially in a single line code as above.
And ...
So would it be possible to read such a file into an assoc array using something like an until or from - i.e. read into an assoc array until it hits a word, or would I have to do this as part of loop?
This will allow me to keep a lot of similar values in same file, but read into separate arrays.
I looked at mapfile as well, but does same as readarray.
Finally ...
I am creating an options list - to select from - as below:
local -a arr=("${!1}")
select option in ${arr[*]}; do
echo ${option}
break
done
Works fine - however the list shown is not sorted. I would like to have it sorted if possible at all.
Hope it is ok to put all 3 questions into 1 as the questions are similar - all on arrays.
Thank you.
First thing, associative arrays are declared with -A not -a:
local -A ary
And if you want to declare a variable on global scope, use declare outside of a function:
declare -A ary
Or use -g if BASH_VERSION >= 4.2.
If your lines do have keyname=valueInfo, with readarray, you can process it like this:
readarray -t lines < "$fileName"
for line in "${lines[#]}"; do
key=${line%%=*}
value=${line#*=}
ary[$key]=$value ## Or simply ary[${line%%=*}]=${line#*=}
done
Using a while read loop can also be an option:
while IFS= read -r line; do
ary[${line%%=*}]=${line#*=}
done < "$fileName"
Or
while IFS== read -r key value; do
ary[$key]=$value
done < "$fileName"
Noble StackOverflow readers,
I have a comma seperated file, each line of which I am putting into an array.
Data looks as so...
25455410,GROU,AJAXa,GROU1435804437
25455410,AING,EXS3d,AING4746464646
25455413,TRAD,DLGl,TRAD7176202067
There are 103 lines and I am able to generate the 103 arrays without issue.
n=1; while read -r OrdLine; do
IFS=',' read -a OrdLineArr${n} <<< "$OrdLine"
let n++
done < $WkOrdsFile
HOWEVER, I can only access the arrays as so...
echo "${OrdLineArr3[0]} <---Gives 25455413
I cannot access it with the number 1-103 as a variable - for example the following doesn't work...
i=3
echo "${OrdLineArr${i}[0]}
That results in...
./script2.sh: line 24: ${OrdLineArr${i}[0]}: bad substitution
I think that the answer might involve 'eval' but I cannot seem to find a fitting example to borrow. If somebody can fix this then the above code makes for a very easy to handle 2d array replacement in bash!
Thanks so much for you help in advance!
Dan
You can use indirect expansion. For example, if $key is OrdLineArr4[7], then ${!key} (with an exclamation point) means ${OrdLineArr4[7]}. (See ยง3.5.3 "Shell Parameter Expansion" in the Bash Reference Manual, though admittedly that passage doesn't really explain how indirect expansion interacts with arrays.)
I'd recommend wrapping this in a function:
function OrdLineArr () {
local -i i="$1" # line number (1-103)
local -i j="$2" # field number (0-3)
local key="OrdLineArr$i[$j]"
echo "${!key}"
}
Then you can write:
echo "$(OrdLineArr 3 0)" # prints 25455413
i=3
echo "$(OrdLineArr $i 0)" # prints 25455413
This obviously isn't a total replacement for two-dimensional arrays, but it will accomplish what you need. Without using eval.
eval is usually a bad idea, but you can do it with:
eval echo "\${OrdLineArr$i[0]}"
I would store each line in an array, but split it on demand:
readarray OrdLineArr < $WkOrdsFile
...
OrdLine=${OrdLineArr[i]}
IFS=, read -a Ord <<< "$OrdLine"
However, bash isn't really equipped for data processing; it's designed to facilitate process and file management. You should consider using a different language.
I'm trying build an sort of property set in ksh.
Thought the easiest way to do so was using arrays but the syntax is killing me.
What I want is to
Build an arbitrary sized array in a config file with a name and a property.
Iterate for each item in that list and get that property.
I theory what I wish I could do is something like
MONITORINGSYS={
SYS1={NAME="GENERATOR" MONITORFUNC="getGeneratorStatus"}
SYS2={NAME="COOLER" MONITORFUNC="getCoolerStatus"}
}
Then later on, be able to do something like:
for CURSYS in $MONITORINGSYS
do
CSYSNAME=$CURSYS.NAME
CSYSFUNC=$CURSYS.MONITORFUNC
REPORT="$REPORT\n$CSYSNAME"
CSYSSTATUS=CSYSFUNC $(date)
REPORT="$REPORT\t$CSYSSTATUS"
done
echo $REPORT
Well, that's not real programming, but I guess you got the point..
How do I do that?
[EDIT]
I do not mean I want to use associative arrays. I only put this way to make my question more clear... I.e. It would not be a problem if the loop was something like:
for CURSYS in $MONITORINGSYS
do
CSYSNAME=${CURSYS[0]}
CSYSFUNC=${CURSYS[1]}
REPORT="$REPORT\n$CSYSNAME"
CSYSSTATUS=CSYSFUNC $(date)
REPORT="$REPORT\t$CSYSSTATUS"
done
echo $REPORT
Same applies to the config file.. I'm just looking for a syntax that makes it minimally readable.
cheers
Not exactly sure what you want... Kornshell can handle both associative and indexed arrays.
However, Kornshell arrays are one dimensional. It might be possible to use indirection to emulate a two dimensional array via the use of $() and eval. I did this a couple of times in the older Perl 4.x and Perl 3.x, but it's a pain. If you want multidimensional arrays, use Python or Perl.
The only thing is that you must declare arrays via the typedef command:
$ typeset -A foohash #foohash is an associative array
$ typeset -a foolist #foolist is an integer indexed array.
Maybe your script can look something like this
typeset -a sysname
typeset -a sysfunct
sysname[1] = "GENERATOR"
sysname[2] = "COOLER"
sysfunc[1] = "getGeneratorStatus"
sysfunc[2] = "getCoolerStatus"
for CURSYS in {1..2}
do
CSYSNAME="${sysname[$CURSYS]}"
CSYSFUNC="${sysfunc[$CURSYS]}"
REPORT="$REPORT\n$CSYSNAME"
CSYSSTATUS=$(eval "CSYSFUNC $(date)")
REPORT="$REPORT\t$CSYSSTATUS"
done
echo $REPORT
ksh93 now has compound variables which can contain a mixture of indexed and associative arrays. No need to declare it as ksh will work it out itself.
#!/bin/ksh
MONITORINGSYS=(
[SYS1]=(NAME="GENERATOR" MONITORFUNC="getGeneratorStatus")
[SYS2]=(NAME="COOLER" MONITORFUNC="getCoolerStatus")
)
echo MONITORING REPORT
echo "-----------------"
for sys in ${!MONITORINGSYS[*]}; do
echo "System: $sys"
echo "Name: ${MONITORINGSYS[$sys].NAME}"
echo "Generator: ${MONITORINGSYS[$sys].MONITORFUNC}"
echo
done
Output:
MONITORING REPORT
-----------------
System: SYS1
Name: GENERATOR
Generator: getGeneratorStatus
System: SYS2
Name: COOLER
Generator: getCoolerStatus