This question already has answers here:
How do I split a string on a delimiter in Bash?
(37 answers)
Closed last year.
STATUS QUO
i have an external properties file, where a couple of variables are stored. one of these has a list of values (min 1 value, max X values).
when declaring this single list within the shell script, it would look like this:
NODES=(
"node"
"node-2"
"node-3"
)
I read the values from the properties file like this:
# checks, if file exists and is readable
file="./properties.credo"
if [ ! -r "$file" ]
then
echo "fatal: properties file $file not found / readable."
exit 2
fi
# loads properties into variables
. $file
[[ -z "$VALUEA" ]] && echo "fatal: VALUEA not specified in .credo" && exit 2
...
PROBLEM
When defining the NODES values in the properties like this:
NODES=node,node-2,node-3
... and reading with that:
...
[[ -z "$NODES" ]] && echo "fatal: NODES not specified in .credo" && exit 2
...
... it will be read from the file as a single string node,node-2,node-3, but not as a list or one-dimensional array.
Beware! This is dangerous as the config file can contain PATH=/some/path and the script can then execute commands over which you have no control.
You can use read -a to populate an array. Set IFS to the separator and send the values read from the file to read's standard input.
#! /bin/bash
file=$1
IFS== read var values < "$file"
IFS=, read -a $var <<< "$values"
echo "${NODES[#]}"
For a multiline config, I tried the following:
nodes=node1,node2,node3
servers=server1,server2
and modified the script to loop over the input:
#! /bin/bash
file=$1
while IFS== read var values ; do
IFS=, read -a $var <<< "$values"
done < "$file"
echo "${nodes[#]}"
echo "${servers[#]}"
You might need to skip over lines that don't follow the var=val1,val2,... pattern.
Related
I have text file like this:
src_dir=source1
src_dir=source2
dst_dir=dest1
whatever_thing=thing1
whatever_thing=thing2
I want a script that will create arrays with names from the left part of a line and fill it with elements from the right part of a line. So basically it should do:
src_dir=(source1 source2)
dst_dir=(dest1)
whatever_thing=(thing1 thing2)
I've tried so far:
while read -r -a line
do
IFS='= ' read -r -a array <<< "$line"
"${array[0]}"+="${array[1]}"
done < file.txt
If your bash version is 4.3 or newer, declare has an -n option
to define a rerefence to the variable name which works as a reference in C++.
Then please try the following:
while IFS== read -r key val; do
declare -n name=$key
name+=("$val")
done < file.txt
# test
echo "${src_dir[#]}"
echo "${dst_dir[#]}"
echo "${whatever_thing[#]}"
try this:
#!/bin/bash
mapfile -t arr < YourDataFile.txt
declare -A dict
for line in "${arr[#]}"; do
key="${line%%=*}"
value="${line#*=}"
[ ${dict["$key"]+X} ] && dict["$key"]+=" $value" || dict["$key"]="$value"
done
for key in "${!dict[#]}"; do
printf "%s=(%s)\n" "$key" "${dict["$key"]}"
done
explanation
# read file into array
mapfile -t arr < YourDataFile.txt
# declare associative array
declare -A dict
# loop over the data array
for line in "${arr[#]}"; do
# extract key
key="${line%%=*}"
# extract value
value="${line#*=}"
# write into associative array
# - if key exists ==> append value
# - else initialize entry
[ ${dict["$key"]+X} ] && dict["$key"]+=" $value" || dict["$key"]="$value"
done
# loop over associative array
for key in "${!dict[#]}"; do
# print key=value pairs
printf "%s=(%s)\n" "$key" "${dict["$key"]}"
done
output
dst_dir=(dest1)
src_dir=(source1 source2)
whatever_thing=(thing1 thing2)
So for some reason my script isnt adding the extra data to the arrays inside the while loops. The data is there and correctly matching but not appending to the arrays.
The input file contents is:
name:passwordhash
while read -r lines;
do
HASH_COMMON=$(echo -n "$lines" | sha256sum | awk '{print $1}');
while read -r line ;
do
# change seperator to colon
IFS=: read -r NAME PASSWORD <<< "$line"
# compare hashed password against hashed password list
if [ "$HASH_COMMON" == "$PASSWORD" ] ;
then
CRACKED_RESULT+=("$lines")
HASH_RESULT+=("$HASH_COMMON")
NAME_RESULT+=("$NAME")
PASSWORD_RESULT+=("$PASSWORD")
fi
done < "$PASSED_FILE"
done < "$COMMON_PW"
output if i print the arrays outside the loop is just a single pass of data using for loop
for i in "${NAME_RESULT[#]}"
do
echo "$i"
echo ""
done
then I am comparing the arrays later with a for loop as well
for isCracked in "${CRACKED_RESULT[#]}";
do
for checkCrack in "${PASSWORD_RESULT[#]}";
do
one="${HASH_RESULT[isCracked]}"
two="${PASSWORD_RESULT[checkCracked]}"
if [ "$one" == "$two" ];
then
RESULT_ARRAY+=("USERNAME: ${NAME_RESULT[checkCracked]} PASSWORD: ${CRACKED_RESULT[checkCracked]}")
fi
checkCrack=+1
done
isCracked=+1
done
This question already has answers here:
How can I store the "find" command results as an array in Bash
(8 answers)
Closed 4 years ago.
How do I put the result of find $1 into an array?
In for loop:
for /f "delims=/" %%G in ('find $1') do %%G | cut -d\/ -f6-
I want to cry.
In bash:
file_list=()
while IFS= read -d $'\0' -r file ; do
file_list=("${file_list[#]}" "$file")
done < <(find "$1" -print0)
echo "${file_list[#]}"
file_list is now an array containing the results of find "$1
What's special about "field 6"? It's not clear what you were attempting to do with your cut command.
Do you want to cut each file after the 6th directory?
for file in "${file_list[#]}" ; do
echo "$file" | cut -d/ -f6-
done
But why "field 6"? Can I presume that you actually want to return just the last element of the path?
for file in "${file_list[#]}" ; do
echo "${file##*/}"
done
Or even
echo "${file_list[#]##*/}"
Which will give you the last path element for each path in the array. You could even do something with the result
for file in "${file_list[#]##*/}" ; do
echo "$file"
done
Explanation of the bash program elements:
(One should probably use the builtin readarray instead)
find "$1" -print0
Find stuff and 'print the full file name on the standard output, followed by a null character'. This is important as we will split that output by the null character later.
<(find "$1" -print0)
"Process Substitution" : The output of the find subprocess is read in via a FIFO (i.e. the output of the find subprocess behaves like a file here)
while ...
done < <(find "$1" -print0)
The output of the find subprocess is read by the while command via <
IFS= read -d $'\0' -r file
This is the while condition:
read
Read one line of input (from the find command). Returnvalue of read is 0 unless EOF is encountered, at which point while exits.
-d $'\0'
...taking as delimiter the null character (see QUOTING in bash manpage). Which is done because we used the null character using -print0 earlier.
-r
backslash is not considered an escape character as it may be part of the filename
file
Result (first word actually, which is unique here) is put into variable file
IFS=
The command is run with IFS, the special variable which contains the characters on which read splits input into words unset. Because we don't want to split.
And inside the loop:
file_list=("${file_list[#]}" "$file")
Inside the loop, the file_list array is just grown by $file, suitably quoted.
arrayname=( $(find $1) )
I don't understand your loop question? If you look how to work with that array then in bash you can loop through all array elements like this:
for element in $(seq 0 $((${#arrayname[#]} - 1)))
do
echo "${arrayname[$element]}"
done
This is probably not 100% foolproof, but it will probably work 99% of the time (I used the GNU utilities; the BSD utilities won't work without modifications; also, this was done using an ext4 filesystem):
declare -a BASH_ARRAY_VARIABLE=$(find <path> <other options> -print0 | sed -e 's/\x0$//' | awk -F'\0' 'BEGIN { printf "("; } { for (i = 1; i <= NF; i++) { printf "%c"gensub(/"/, "\\\\\"", "g", $i)"%c ", 34, 34; } } END { printf ")"; }')
Then you would iterate over it like so:
for FIND_PATH in "${BASH_ARRAY_VARIABLE[#]}"; do echo "$FIND_PATH"; done
Make sure to enclose $FIND_PATH inside double-quotes when working with the path.
Here's a simpler pipeless version, based on the version of user2618594
declare -a names=$(echo "("; find <path> <other options> -printf '"%p" '; echo ")")
for nm in "${names[#]}"
do
echo "$nm"
done
To loop through a find, you can simply use find:
for file in "`find "$1"`"; do
echo "$file" | cut -d/ -f6-
done
It was what I got from your question.
I read the files of a directory and put each file name into an array (SEARCH)
Then I use a loop to go through each file name in the array (SEARCH) and open them up with a while read line loop and read each line into another array (filecount). My problem is its one huge array with 39 lines (each file has 13 lines) and I need it to be 3 seperate arrays, where
filecount1[line1] is the first line from the 1st file and so on. here is my code so far...
typeset -A files
for file in ${SEARCH[#]}; do
while read line; do
files["$file"]+="$line"
done < "$file"
done
So, Thanks Ivan for this example! However I'm not sure I follow how this puts it into a seperate array because with this example wouldnt all the arrays still be named "files"?
If you're just trying to store the file contents into an array:
declare -A contents
for file in "${!SEARCH[#]}"; do
contents["$file"]=$(< $file)
done
If you want to store the individual lines in a array, you can create a pseudo-multi-dimensional array:
declare -A contents
for file in "${!SEARCH[#]}"; do
NR=1
while read -r line; do
contents["$file,$NR"]=$line
(( NR++ ))
done < "$file"
done
for key in "${!contents[#]}"; do
printf "%s\t%s\n" "$key" "${contents["$key"]}"
done
line 6 is
$filecount[$linenum]}="$line"
Seems it is missing a {, right after the $.
Should be:
${filecount[$linenum]}="$line"
If the above is true, then it is trying to run the output as a command.
Line 6 is (after "fixing" it above):
${filecount[$linenum]}="$line"
However ${filecount[$linenum]} is a value and you can't have an assignment on a value.
Should be:
filecount[$linenum]="$line"
Now I'm confused, as in whether the { is actually missing, or } is the actual typo :S :P
btw, bash supports this syntax too
filecount=$((filecount++)) # no need for $ inside ((..)) and use of increment operator ++
This should work:
typeset -A files
for file in ${SEARCH[#]}; do # foreach file
while read line; do # read each line
files["$file"]+="$line" # and place it in a new array
done < "$file" # reading each line from the current file
done
a small test shows it works
# set up
mkdir -p /tmp/test && cd $_
echo "abc" > a
echo "foo" > b
echo "bar" > c
# read files into arrays
typeset -A files
for file in *; do
while read line; do
files["$file"]+="$line"
done < "$file"
done
# print arrays
for file in *; do
echo ${files["$file"]}
done
# same as:
echo ${files[a]} # prints: abc
echo ${files[b]} # prints: foo
echo ${files[c]} # prints: bar
Posted my code below, wondering if I can search one array for a match... or if theres a way I can search a unix file inside of an argument.
#!/bin/bash
# store words in file
cat $1 | ispell -l > file
# move words in file into array
array=($(< file))
# remove temp file
rm file
# move already checked words into array
checked=($(< .spelled))
# print out words & ask for corrections
for ((i=0; i<${#array[#]}; i++ ))
do
if [[ ! ${array[i]} = ${checked[#]} ]]; then
read -p "' ${array[i]} ' is mispelled. Press "Enter" to keep
this spelling, or type a correction here: " input
if [[ ! $input = "" ]]; then
correction[i]=$input
else
echo ${array[i]} >> .spelled
fi
fi
done
echo "MISPELLED: CORRECTIONS:"
for ((i=0; i<${#correction[#]}; i++ ))
do
echo ${array[i]} ${correction[i]}
done
otherwise, i would need to write a for loop to check each array indice, and then somehow make a decision statement whether to go through the loop and print/take input
The ususal shell incantation to do this is:
cat $1 | ispell -l |while read -r ln
do
read -p "$ln is misspelled. Enter correction" corrected
if [ ! x$corrected = x ] ; then
ln=$corrected
fi
echo $ln
done >correctedwords.txt
The while;do;done is kind of like a function and you can pipe data into and out of it.
P.S. I didn't test the above code so there may be syntax errors