I have a file containing config information and a shell script that reads that file. I want to hand over values to a bash script.
file.txt
varNumber=1.1.1
varName=testThis
varFile=~/myDir/mySubDir/output.zip
myShellScript.sh
FILENAME="~/myDir/mySubDir/output.zip" <- this is what I expect from grep/awk
startNextScript.sh -f $FILENAME
I would like to extract the variables either as an associated array or - if easier - grep for them,
but as I'm not used to writing commands like this in bash I am asking for help!
Using associative array in bash:
#!/bin/bash
declare -A vars
while read -r line ; do
var=${line%%=*} # Remove everything after the first =.
value=${line#*=} # Remove everything before the first =.
vars[$var]=$value
done < file.txt
echo Number: ${vars[varNumber]}
echo Name: ${vars[varName]}
echo File: ${vars[varFile]}
Related
I have two shell scripts, fruits_original.sh and appending_fruits.sh. In the fruits_original.sh I have one array variable: fruits=('Apple' 'Mango' 'Guava').
What I want to do is I have to write a shell script appending_fruits.sh that will take an argument some new fruits name is Orange and will append that new fruit name to the fruits_original.sh fruits array variable.
After script run fruits array should be remain an array only and its value should be fruits=('Apple' 'Mango' 'Guava' 'Orange').
The file fruits_original.sh has this. Below is the appending_fruits.sh script by this my variable is changing into this fruits= ('Apple' 'Mango' 'Guava' 'Orange'). But when I am trying to do echo "${fruits[#]}" I am getting this error:
line 1: syntax error near unexpected token `('
Any luck ?
fruits= ('Apple' 'Mango' 'Guava')
echo "${fruits[#]}"
declare -a var=$(awk -F'=' '/^fruits=/ {print $2}' fruits_original.sh)
echo "${var[#]}"
var[${#var[#]}]='Orange'
joined=$(printf " '%s'" "${var[#]}")
echo ${joined:1}
echo "${joined[#]}"
sed -i "s/fruits=.*/fruits= ($( echo ${joined:1})) /" fruits_original.sh
Do not modify the script file. Instead, create another file and source the dynamic data from it. I have chosen the location of configuration to be in /tmp directory.
# fruits_original.sh
fruits=()
if [[ -e /tmp/fruits_original.rc ]]; then
. /tmp/fruits_original.rc
fi
some stuff
Then generate the config file. Use declare -p to safely output properly quoted variables.
# appending_fruits.sh
fruits=()
if [[ -e /tmp/fruits_original.rc ]]; then
. /tmp/fruits_original.rc
fi
fruits+=("new fruit")
decalre -p fruits > /tmp/fruits_original.rc
Put a uuid inside fruits_original.sh to recognize where is your snippet that you want to work with.
# fruits_original.sh
# snip 419d0df3-5f08-4511-ad5a-ad24db45aa6c
fruits=()
# snip 419d0df3-5f08-4511-ad5a-ad24db45aa6c
some stuff
Then extract the relevant parts with sed or other tool, declare "$part" it into a variable, append normally and then capture output from declare -p and replace the content between the marks again.
If not going with any of the above and this is only a very toy example to test some stuff, you could:
# read the line from another script
declare "$(sed '/fruits=/!d' fruits_original.sh)"
# append element
fruits+=(Orange)
# create source-able output
new="$(declare -p fruits)"
# remove declare -- in front
new="fruits=${new%*fruits=}"
# Replace the line with declare -p output.
sed -i "s/fruits=.*/fruits=$new/" fruits_original.sh
Notes:
var[${#var[#]}]='Orange' - just var+=(Orange). No need for ${#.
$( echo ${joined:1}) is a useless use of echo (unless you want word splitting and filename expansion).
check your scripts with https://shellcheck.net
fruits= ( is not an assignment and will run a subshell and could cause syntax error. There is no space in assignment around =.
declare -a var=$( - var is not an array (or, it's an array with one element).
I'm stucked on a bash script.
I'm having a config.ini files like this :
#Username
username=user
#Userpassword
userpassword=password
And i'm looking in a bash script to extract this information and put it in a associative array. My script looks like :
declare -A array
OIFS=$IFS
IFS='='
grep -vE '^(\s*$|#)' file | while read -r var1 var2
do
array+=([$var1]=$var2)
done
echo ${array[#]}
But the array seems to be empty because the commande echo ${array[#]} gives no output.
Any idea why me script don't work ? Thanks for your help and sorry for my bad english.
Common error - "grep | while" causes the while loop to be executed in a separate shell and the variables inside the loop are not global to your shell. Use a here string instead:
while read -r var1 var2
do
array+=([$var1]=$var2)
done <<< $(grep -vE '^(\s*$|#)' file)
Assuming the file can be trusted (ie the content is regulated and known), the simplest method would be to source the ini file and then directly use the variable names within the script:
. config.ini
You can either use the period (.) as above or the source builtin command
I have the following find command with the following output:
$ find -name '*.jpg'
./public_html/github/screencasts-gh-pages/reactiveDataVis/presentation/images/telescope.jpg
./public_html/github/screencasts-gh-pages/introToBackbone/presentation/images/telescope.jpg
./public_html/github/StarCraft-master/img/Maps/(6)Thin Ice.jpg
./public_html/github/StarCraft-master/img/Maps/Snapshot.jpg
./public_html/github/StarCraft-master/img/Maps/Map_Grass.jpg
./public_html/github/StarCraft-master/img/Maps/(8)TheHunters.jpg
./public_html/github/StarCraft-master/img/Maps/(2)Volcanis.jpg
./public_html/github/StarCraft-master/img/Maps/(3)Trench wars.jpg
./public_html/github/StarCraft-master/img/Maps/(8)BigGameHunters.jpg
./public_html/github/StarCraft-master/img/Maps/(8)Turbo.jpg
./public_html/github/StarCraft-master/img/Maps/(4)Blood Bath.jpg
./public_html/github/StarCraft-master/img/Maps/(2)Switchback.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(6)Thin Ice.jpg
./public_html/github/StarCraft-master/img/Maps/Original/Map_Grass.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(8)TheHunters.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(2)Volcanis.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(3)Trench wars.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(8)BigGameHunters.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(8)Turbo.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(4)Blood Bath.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(2)Switchback.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(4)Orbital Relay.jpg
./public_html/github/StarCraft-master/img/Maps/(4)Orbital Relay.jpg
./public_html/github/StarCraft-master/img/Bg/GameLose.jpg
./public_html/github/StarCraft-master/img/Bg/GameWin.jpg
./public_html/github/StarCraft-master/img/Bg/GameStart.jpg
./public_html/github/StarCraft-master/img/Bg/GamePlay.jpg
./public_html/github/StarCraft-master/img/Demo/Demo.jpg
./public_html/github/flot/examples/image/hs-2004-27-a-large-web.jpg
./public_html/github/minicourse-ajax-project/other/GameLose.jpg
How do I store this output in an array? I want it to handle filenames with spaces
I have tried this arrayname=($(find -name '*.jpg')) but this just stores the first element. # I am doing the following which seems to be just the first element?
$ arrayname=($(find -name '*.jpg'))
$ echo "$arrayname"
./public_html/github/screencasts-gh-pages/reactiveDataVis/presentation/images/telescope.jpg
$
I have tried here but again this just stores the 1st element
Other similar Qs
How do I capture the output from the ls or find command to store all file names in an array?
How do i store the output of a bash command in a variable?
If you know with certainty that your filenames will not contain newlines, then
mapfile -t arrayname < <(find ...)
If you want to be able to handle any file
arrayname=()
while IFS= read -d '' -r filename; do
arrayname+=("$filename")
done < <(find ... -print0)
echo "$arrayname" will only show the first element of the array. It is equivalent to echo "${arrayname[0]}". To dump an array:
printf "%s\n" "${arrayname[#]}"
# ............^^^^^^^^^^^^^^^^^ must use exactly this form, with the quotes.
arrayname=($(find ...)) is still wrong. It will store the file ./file with spaces.txt as 3 separate elements in the array.
If you have a sufficiently recent version of bash, you can save yourself a lot of trouble by just using a ** glob.
shopt -s globstar
files=(**/*.jpg)
The first line enables the feature. Once enabled, ** in a glob pattern will match any number (including 0) of directories in the path.
Using the glob in the array definition makes sure that whitespace is handled correctly.
To view an array in a form which could be used to define the array, use the -p (print) option to the declare builtin:
declare -p files
A script takes a URL, parses it for the required fields, and redirects its output to be saved in a file, file.txt. The output is saved on a new line each time a field has been found.
file.txt
A Cat
A Dog
A Mouse
etc...
I want to take file.txt and create an array from it in a new script, where every line gets to be its own string variable in the array. So far I have tried:
#!/bin/bash
filename=file.txt
declare -a myArray
myArray=(`cat "$filename"`)
for (( i = 0 ; i < 9 ; i++))
do
echo "Element [$i]: ${myArray[$i]}"
done
When I run this script, whitespace results in words getting split and instead of getting
Desired output
Element [0]: A Cat
Element [1]: A Dog
etc...
I end up getting this:
Actual output
Element [0]: A
Element [1]: Cat
Element [2]: A
Element [3]: Dog
etc...
How can I adjust the loop below such that the entire string on each line will correspond one-to-one with each variable in the array?
Use the mapfile command:
mapfile -t myArray < file.txt
The error is using for -- the idiomatic way to loop over lines of a file is:
while IFS= read -r line; do echo ">>$line<<"; done < file.txt
See BashFAQ/005 for more details.
mapfile and readarray (which are synonymous) are available in Bash version 4 and above. If you have an older version of Bash, you can use a loop to read the file into an array:
arr=()
while IFS= read -r line; do
arr+=("$line")
done < file
In case the file has an incomplete (missing newline) last line, you could use this alternative:
arr=()
while IFS= read -r line || [[ "$line" ]]; do
arr+=("$line")
done < file
Related:
Need alternative to readarray/mapfile for script on older version of Bash
You can do this too:
oldIFS="$IFS"
IFS=$'\n' arr=($(<file))
IFS="$oldIFS"
echo "${arr[1]}" # It will print `A Dog`.
Note:
Filename expansion still occurs. For example, if there's a line with a literal * it will expand to all the files in current folder. So use it only if your file is free of this kind of scenario.
Use mapfile or read -a
Always check your code using shellcheck. It will often give you the correct answer. In this case SC2207 covers reading a file that either has space separated or newline separated values into an array.
Don't do this
array=( $(mycommand) )
Files with values separated by newlines
mapfile -t array < <(mycommand)
Files with values separated by spaces
IFS=" " read -r -a array <<< "$(mycommand)"
The shellcheck page will give you the rationale why this is considered best practice.
You can simply read each line from the file and assign it to an array.
#!/bin/bash
i=0
while read line
do
arr[$i]="$line"
i=$((i+1))
done < file.txt
This answer says to use
mapfile -t myArray < file.txt
I made a shim for mapfile if you want to use mapfile on bash < 4.x for whatever reason. It uses the existing mapfile command if you are on bash >= 4.x
Currently, only options -d and -t work. But that should be enough for that command above. I've only tested on macOS. On macOS Sierra 10.12.6, the system bash is 3.2.57(1)-release. So the shim can come in handy. You can also just update your bash with homebrew, build bash yourself, etc.
It uses this technique to set variables up one call stack.
Make sure set the Internal File Separator (IFS)
variable to $'\n' so that it does not put each word
into a new array entry.
#!/bin/bash
# move all 2020 - 2022 movies to /backup/movies
# put list into file 1 line per dir
# dirs are "movie name (year)/"
ls | egrep 202[0-2] > 2020_movies.txt
OLDIFS=${IFS}
IFS=$'\n' #fix separator
declare -a MOVIES # array for dir names
MOVIES=( $( cat "${1}" ) ) // load into array
for M in ${MOVIES[#]} ; do
echo "[${M}]"
if [ -d "${M}" ] ; then # if dir name
mv -v "$M" /backup/movies/
fi
done
IFS=${OLDIFS} # restore standard separators
# not essential as IFS reverts when script ends
#END
My data set(data.txt) looks like this [imageID,sessionID,height1,height2,x,y,crop]:
1,0c66824bfbba50ee715658c4e1aeacf6fda7e7ff,1296,4234,194,1536,0
2,0c66824bfbba50ee715658c4e1aeacf6fda7e7ff,1296,4234,194,1536,0
3,0c66824bfbba50ee715658c4e1aeacf6fda7e7ff,1296,4234,194,1536,0
4,0c66824bfbba50ee715658c4e1aeacf6fda7e7ff,1296,4234,194,1536,950
These are a set of values which I wish to use. I'm new to shell script :) I read the file line by line like this ,
cat $FILENAME | while read LINE
do
string=($LINE)
# PROCESSING THE STRING
done
Now, in the code above, after getting the string, I wish to do the following :
1. Split the string into comma separated values.
2. Store these variables into arrays like imageID[],sessionID[].
I need to access these values for doing image processing using imagemagick.
However, I'm not able to perform the above steps correctly
set -A doesn't work for me (probably due to older BASH on OSX)
Posting an alternate solution using read -a in case someone needs it:
# init all your individual arrays here
imageId=(); sessionId=();
while IFS=, read -ra arr; do
imageId+=(${arr[0]})
sessionId+=(${arr[1]})
done < input.csv
# Print your arrays
echo "${imageId[#]}"
echo "${sessionId[#]}"
oIFS="$IFS"; IFS=','
set -A str $string
IFS="$oIFS"
echo "${str[0]}";
echo "${str[1]}";
echo "${str[2]}";
you can split and store like this
have a look here for more on Unix arrays.