I'm trying to read the information of a structured file into an associative array using Bash script. The file contains in each line the name of a person and its address, separated by a "|". For example:
person1|address of person1
person2|address of person2
...
personN|address of personN
I tried to do this using the script below. Within the WHILE loop, the information is being printed. However, in the FOR loop the information is not being printed. It seems that the information is not being stored in the associative array outside of the WHILE loop.
What am I doing wrong? Why this is not working? Is there more efficient ways to do that?
#!/bin/bash
declare -A address
cat adresses.txt | while read line
do
name=`echo $line | cut -d '|' -f 1`
add=`echo $line | cut -d '|' -f 2`
address[$name]=$add
echo "$name - ${address[$name]}"
done
for name in ${!address[*]}
do
echo "$name - ${address[$name]}"
done
Wrong and useless usage of cut
#!/bin/bash
declare -A address
while IFS=\| read name add
do
address[$name]=$add
done < adresses.txt
for name in ${!address[*]}
do
echo "$name - ${address[$name]}"
done
cat addresses.txt | while read line
do
...
done
Shell commands in a pipelines are executed in subshells. Variables set in subshells aren't visible the parent shell.
You can fix this by replacing the pipelines with a redirection.
while read line
do
...
done < addresses.txt
Extending the accepted answer to resolve the OP's comment:
#!/bin/bash
declare -A address
while IFS='|' read name add
do
address[$name]=$add
echo "$name - ${address[$name]}"
done < adresses.txt
for name in "${!address[#]}"
do
echo "$name - ${address[$name]}"
done
Related
The following script produces no output for me when I run it. I am really confused as to why this isn't working.
#!/bin/bash
i=0
OLDIFS=$IFS
IFS=$'\n'
read -p 'Search history for? ' string
arr=( "$(history | grep "$string" | cut -c8-)" )
for item in ${arr[#]}
do
echo "$(( i++))) $item"
done
However this exact same thing (at least it seems the same to me) works fine when typed directly into my terminal in a single line:
i=0; OLDIFS=$IFS; IFS=$'\n'; read -p 'Search history for? ' string; arr=( "$(history | grep "$string" | cut -c8-)" ); for item in ${arr[#]}; do echo "$(( i++))) $item"; done
I've made the script executable. I've saved it as both a multi line and a single line script. Yet none of the saved scripts produce any output. Why doesn't this work when saved as a script but works fine typed directly into my terminal?
The line echo "$(( i++))) $item" has one closing parentheses in excess.
echo "$(( i++ )) $item"
If you try to use history in a script, it will fail.
Try running this script:
#!/bin/bash
history
It will print nothing because there is no history stored (for this instance of the shell). To read history you need to provide the file with the stored history, call the builtin history to read -r and finally you can list the history from memory:
#!/bin/bash
HISTFILE="$HOME/.bash_history"
history -r
history
That doesn't mean that commands will be written to the file, that's controlled by a different option.
#!/bin/bash
read -p 'Search history for? ' string
i=0
OLDIFS=$IFS
IFS=$'\n'
HISTFILE="$HOME/.bash_history"
history -r
IFS=$'\n' read -d '' -a arr <<<"$(history | grep "$string" | cut -c8-)"
for item in ${arr[#]}
do echo "$(( i++ )) $item"
done
Have a look at this. Apparently the bash history command is disabled in shell programs. But you can get around it according to that link:
#!/bin/bash
#Add this line in to set the history file to your.bash_history
HISTFILE=~/.bash_history
set -o history
history
A script takes a URL, parses it for the required fields, and redirects its output to be saved in a file, file.txt. The output is saved on a new line each time a field has been found.
file.txt
A Cat
A Dog
A Mouse
etc...
I want to take file.txt and create an array from it in a new script, where every line gets to be its own string variable in the array. So far I have tried:
#!/bin/bash
filename=file.txt
declare -a myArray
myArray=(`cat "$filename"`)
for (( i = 0 ; i < 9 ; i++))
do
echo "Element [$i]: ${myArray[$i]}"
done
When I run this script, whitespace results in words getting split and instead of getting
Desired output
Element [0]: A Cat
Element [1]: A Dog
etc...
I end up getting this:
Actual output
Element [0]: A
Element [1]: Cat
Element [2]: A
Element [3]: Dog
etc...
How can I adjust the loop below such that the entire string on each line will correspond one-to-one with each variable in the array?
Use the mapfile command:
mapfile -t myArray < file.txt
The error is using for -- the idiomatic way to loop over lines of a file is:
while IFS= read -r line; do echo ">>$line<<"; done < file.txt
See BashFAQ/005 for more details.
mapfile and readarray (which are synonymous) are available in Bash version 4 and above. If you have an older version of Bash, you can use a loop to read the file into an array:
arr=()
while IFS= read -r line; do
arr+=("$line")
done < file
In case the file has an incomplete (missing newline) last line, you could use this alternative:
arr=()
while IFS= read -r line || [[ "$line" ]]; do
arr+=("$line")
done < file
Related:
Need alternative to readarray/mapfile for script on older version of Bash
You can do this too:
oldIFS="$IFS"
IFS=$'\n' arr=($(<file))
IFS="$oldIFS"
echo "${arr[1]}" # It will print `A Dog`.
Note:
Filename expansion still occurs. For example, if there's a line with a literal * it will expand to all the files in current folder. So use it only if your file is free of this kind of scenario.
Use mapfile or read -a
Always check your code using shellcheck. It will often give you the correct answer. In this case SC2207 covers reading a file that either has space separated or newline separated values into an array.
Don't do this
array=( $(mycommand) )
Files with values separated by newlines
mapfile -t array < <(mycommand)
Files with values separated by spaces
IFS=" " read -r -a array <<< "$(mycommand)"
The shellcheck page will give you the rationale why this is considered best practice.
You can simply read each line from the file and assign it to an array.
#!/bin/bash
i=0
while read line
do
arr[$i]="$line"
i=$((i+1))
done < file.txt
This answer says to use
mapfile -t myArray < file.txt
I made a shim for mapfile if you want to use mapfile on bash < 4.x for whatever reason. It uses the existing mapfile command if you are on bash >= 4.x
Currently, only options -d and -t work. But that should be enough for that command above. I've only tested on macOS. On macOS Sierra 10.12.6, the system bash is 3.2.57(1)-release. So the shim can come in handy. You can also just update your bash with homebrew, build bash yourself, etc.
It uses this technique to set variables up one call stack.
Make sure set the Internal File Separator (IFS)
variable to $'\n' so that it does not put each word
into a new array entry.
#!/bin/bash
# move all 2020 - 2022 movies to /backup/movies
# put list into file 1 line per dir
# dirs are "movie name (year)/"
ls | egrep 202[0-2] > 2020_movies.txt
OLDIFS=${IFS}
IFS=$'\n' #fix separator
declare -a MOVIES # array for dir names
MOVIES=( $( cat "${1}" ) ) // load into array
for M in ${MOVIES[#]} ; do
echo "[${M}]"
if [ -d "${M}" ] ; then # if dir name
mv -v "$M" /backup/movies/
fi
done
IFS=${OLDIFS} # restore standard separators
# not essential as IFS reverts when script ends
#END
Just wondering, assuming I am storing my data in a file called BookDB.txt in the following format :
C++ for dummies:Jared:10.52:5:6
Java for dummies:David:10.65:4:6
whereby each field is seperated by the delimeter ":".
How would I preserve whitespace in the first field and have an array with the following contents : ('C++ for dummies' 'Java for dummies')?
Any help is very much appreciated!
Ploutox's solution is almost correct, but without setting IFS, you will not get the array that you seek, with two elements in this case.
Note: He corrected his solution after this post.
IFS=$'\n': arr=( $(awk -F':' '{print $1 }' Input.txt ) )
echo ${#arr[#]}
echo ${arr[0]}
echo ${arr[1]}
Output:
2
C++ for dummies
Java for dummies
Just use a while loop:
#!/bin/bash
# create and populate the array
a=()
while IFS=':' read -r field _
do
a+=("$field")
done < file
# print the array contents
printf "%s\n" "${a[#]}"
I totally misunderstood your question on my 1st attempt to answer. awk seems more suited for your need though. You can get what you want with simple scripting :
IFS=$'\n' : MYARRAY=($(awk -F ":" '{print $1}' myfile))
the -F flag forces : as the field separator.
echo ${MYARRAY[0]} will print :
C++ for dummies
$ yes sed -i "s/:/\'\'/" BookDB.txt | head -n100 | bash
this command while work. this is a linux command, run it on shell in same path with BookDB.txt
I have a text file which consists of say ..following information say test.text:
an
apple of
one's eye
I want to read these lines in an array using shell scripting by doing a cat test.text. I have tried using a=(`cat test.text`), but that doesn't work as it considers space as a delimiter. I need the values as a[0]=an , a[1]=apple of , a[2]=one's eye. I don't want to use IFS. Need help, thanks in advance..!!
In bash 4 or later
readarray a < test.text
This will include an empty element for each blank line, so you might want to remove the empty lines from the input file first.
In earlier versions, you'll need to build the array manually.
a=()
while read; do a+=("$REPLY"); done < test.text
One of various options you have is to use read with bash. Set IFS to the newline and line separator to NUL
IFS=$'\n' read -d $'\0' -a a < test.txt
Plain sh
IFS='
'
set -- $(< test.txt)
unset IFS
echo "$1"
echo "$2"
echo "$#"
bash
IFS=$'\n' a=($(< test.txt))
echo "${a[0]}"
echo "${a[1]}"
echo "${a[#]}"
I'm inclined to say these are the best of the available solutions because they do not involve looping.
Let's say:
cat file
an
apple of
one's eye
Use this while loop:
arr=()
while read -r l; do
[[ -n "$l" ]] && arr+=("$l")
done < file
TEST
set | grep arr
arr=([0]="an" [1]="apple of" [2]="one's eye")
I'm trying to create an alias that will get all "Modified" files and run the php syntax check on them...
function gitphpcheck () {
filearray=()
git diff --name-status | while read line; do
if [[ $line =~ ^M ]]
then
filename="`echo $line | awk '{ print $2 }'`"
echo "$filename" # correct output
filearray+=($filename)
fi
done
echo "--------------FILES"
echo ${filearray[#]}
# will do php check here, but echo of array is blank
}
As Wrikken says, the while body runs in a subshell, so all changes to the filearray array will disappear when the subshell ends. A couple of different solutions come to mind:
Process substitution (less readable but does not require a subshell)
while read line; do
:
done < <(git diff --name-status)
echo "${filearray[#]}"
Use the modified variable in the subshell using command grouping
git diff --name-status | {
while read line; do
:
done
echo "${filearray[#]}"
}
# filearray is empty here
You've piped | things to while, which is essentially another process, so the filearray variable is a different one (not the same scope).