This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 6 years ago.
I have a pretty simple sh script where I make a system cat call, collect the results and parse some relevant information before storing the information in an array, which seems to work just fine. But as soon as I exit the for loop where I store the information, the array seems to clear itself. I'm wondering if I am accessing the array incorrectly outside of the for loop. Relevant portion of my script:
#!/bin/sh
declare -a QSPI_ARRAY=()
cat /proc/mtd | while read mtd_instance
do
# split result into individiual words
words=($mtd_instance)
for word in "${words[#]}"
do
# check for uboot
if [[ $word == *"uboot"* ]]
then
mtd_num=${words[0]}
index=${mtd_num//[!0-9]/} # strip everything except the integers
QSPI_ARRAY[$index]="uboot"
echo "QSPI_ARRAY[] at index $index: ${QSPI_ARRAY[$index]}"
elif [[ $word == *"fpga_a"* ]]
then
echo "found it: "$word""
mtd_num=${words[0]}
index=${mtd_num//[!0-9]/} # strip everything except the integers
QSPI_ARRAY[$index]="fpga_a"
echo "QSPI_ARRAY[] at index $index: ${QSPI_ARRAY[$index]}"
# other items are added to the array, all successfully
fi
done
echo "length of array: ${#QSPI_ARRAY[#]}"
echo "----------------------"
done
My output is great until I exit the for loop. While within the for loop, the array size increments and I can check that the item has been added. After the for loop is complete I check the array like so:
echo "RESULTING ARRAY:"
echo "length of array: ${#QSPI_ARRAY[#]}"
for qspi in "${QSPI_ARRAY}"
do
echo "qspi instance: $qspi"
done
Here are my results, echod to my display:
dev: size erasesize name
length of array: 0
-------------
mtd0: 00100000 00001000 "qspi-fsbl-uboot"
QSPI_ARRAY[] at index 0: uboot
length of array: 1
-------------
mtd1: 00500000 00001000 "qspi-fpga_a"
QSPI_ARRAY[] at index 1: fpga_a
length of array: 2
-------------
RESULTING ARRAY:
length of array: 0
qspi instance:
EDIT: After some debugging, it seems I have two different arrays here somehow. I initialized the array like so: QSPI_ARRAY=("a" "b" "c" "d" "e" "f" "g"), and after my for-loop for parsing the array it is still a, b, c, etc. How do I have two different arrays of the same name here?
This structure:
cat /proc/mtd | while read mtd_instance
do
...
done
Means that whatever comes between do and done cannot have any effects inside the shell environment that are still there after the done.
The fact that the while loop is on the right hand side of a pipe (|) means that it runs in a subshell. Once the loop exits, so does the subshell. And all of its variable settings.
If you want a while loop which makes changes that stick around, don't use a pipe. Input redirection doesn't create a subshell, and in this case, you can just read from the file directly:
while read mtd_instance
do
...
done </proc/mtd
If you had a more complicated command than a cat, you might need to use process substitution. Still using cat as an example, that looks like this:
while read mtd_instance
do
...
done < <(cat /proc/mtd)
In the specific case of your example code, I think you could simplify it somewhat, perhaps like this:
#!/usr/bin/env bash
QSPI_ARRAY=()
while read -a words; do␣
declare -i mtd_num=${words[0]//[!0-9]/}
for word in "${words[#]}"; do
for type in uboot fpga_a; do
if [[ $word == *$type* ]]; then
QSPI_ARRAY[mtd_num]=$type
break 2
fi
done
done
done </proc/mtd
Is this potentially what you are seeing:
http://mywiki.wooledge.org/BashFAQ/024
Related
I have an array of strings to pass through a script. The script is well-behaved, and will return error code 0 if the string "passes" and non-zero if it "fails." If the string passes, it should be included in a final array to be output or written to file or etc.
The problem I'm having is that the only item ending up in my final array is the first "passing" string.
#!/bin/bash
# validator.sh
if [[ $1 -le 10 ]]; then
exit 0
else
exit 1
fi
#!/bin/bash
# main.sh
numbers=(2 4 6 8 10 12 14 16)
keep=()
for n in ${numbers[#]}; do
if ./validator.sh $n; then
keep+=("$n")
fi
done
echo $keep
Running main.sh produces:
$ ./main.sh
2
but I expect 2 4 6 8 10
Unless you meant keep to be an array of matching elements, change:
keep+=("$n")
to
keep="$keep $n"
That would work with any Bourne compatible shell and is therefore better, if you're looking for BASH specific solution, the below will also work:
keep+="${n} "
If you DO want it to be an array, then in order to output all elements, you can use:
echo ${keep[#]}
As noted by #Jetchisel and #kamilCuk in the comments.
Since you wrote you want to output all elements or save them to a file, I had assumed you don't actually need an array here but perhaps you plan to use this data in other ways later:)
I wrote a bash script that reads a file from stdin $1, and needs to read that file line by line within a loop, and based on a condition statement in each iteration, each line tested from the file will feed into one of two new arrays lets say named GOOD array and BAD array. Lastly, I'll display the total elements of each array.
#!/bin/bash
for x in $(cat $1); do
#testing something on x
if [ $? -eq 0 ]; then
#add the current value of x into array called GOOD
else
#add the current value of x into array called BAD
fi
done
echo "Total GOOD elements: ${#GOOD[#]}"
echo "Total BAD elements: ${#BAD[#]}"
What changes should i make to accomplish it?
#!/usr/bin/env bash
# here, we're checking the number of lines more than 5 characters long
# replace with your real test
testMyLine() { (( ${#1} > 5 )); }
good=( ); bad=( )
while IFS= read -r line; do
if testMyLine "$line"; then
good+=( "$line" )
else
bad+=( "$line" )
fi
done <"$1"
echo "Read ${#good[#]} good and ${#bad[#]} bad lines"
Note:
We're using a while read loop to iterate over file contents. This doesn't need to read more than one line into memory at a time (so it won't run out of RAM even with really big files), and it doesn't have unwanted side effects like changing a line containing * to a list of files in the current directory.
We aren't using $?. if foo; then is a much better way to branch on the exit status of foo than foo; if [ $? = 0 ]; then -- in particular, this avoids depending on the value of $? not being changed between when you assign it and when you need it; and it marks foo as "checked", to avoid exiting via set -e or triggering an ERR trap when your boolean returns false.
The use of lower-case variable names is intentional. All-uppercase names are used for shell-builtin variables and names with special meaning to the operating system -- and since defining a regular shell variable overwrites any environment variable with the same name, this convention applies to both types. See http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
I have a grep output and I'm trying to make an associative array from the output that I get.
Here is my grep output:
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
I want to use that output to define an associative array, like this:
array[123456789101]=devid1234
array[11213141516]=devid5678
Is that possible? I'm new at making arrays. I hope someone could help me in my problem.
Either pipe your grep output to a helper script with a while loop containing a simple "0/1" toggle to read two lines taking the last field of each to fill your array, e.g.
#!/bin/bash
declare -A array
declare -i n=0
arridx=
while read -r label value; do # read 2 fields
if [ "$n" -eq 0 ]
then
arridx="${value:1}" # strip 1st and lst 2 chars
arridx="${arridx:0:(-2)}" # save in arridx (array index)
((n++)) # increment toggle
else
arrval="${value:1}" # strip 1st and lst 2 chars
arrval="${arrval:0:(-2)}" # save in arrval (array value)
array[$arridx]="$arrval" # assign to associative array
n=0 # zero toggle
fi
done
for i in ${!array[#]}; do # output array
echo "array[$i] ${array[$i]}"
done
Or you can use process substitution containing the grep command within the script to do the same thing, e.g.
done < <( your grep command )
You can also add a check under the else clause that if [[ $label =~ DeviceId ]] to validate you are on the right line and catch any variation in the grep output content.
Example Input
$ cat dat/grepout.txt
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
Example Use/Output
$ cat dat/grepout.txt | bash parsegrep2array.sh
array[123456789101] devid1234
array[111213141516] devid5678
Parsing out the values is easy, and once you have them you can certainly use those values to build up an array. The trickiest part comes from the fact that you need to combine input from separate lines. Here is one approach; note that this script is verbose on purpose, to show what's going on; once you see what's happening, you can eliminate most of the output:
so.input
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
so.sh
#!/bin/bash
declare -a hardwareInfo
while [[ 1 ]]; do
# read in two lines of input
# if either line is the last one, we don't have enough input to proceed
read lineA < "${1:-/dev/stdin}"
# if EOF or empty line, exit
if [[ "$lineA" == "" ]]; then break; fi
read lineB < "${1:-/dev/stdin}"
# if EOF or empty line, exit
if [[ "$lineB" == "" ]]; then break; fi
echo "$lineA"
echo "$lineB"
hwsn=$lineA
hwsn=${hwsn//HardwareSerialNumber/}
hwsn=${hwsn//\"/}
hwsn=${hwsn//:/}
hwsn=${hwsn//,/}
echo $hwsn
# some checking could be done here to test that the value is numeric
devid=$lineB
devid=${devid//DeviceId/}
devid=${devid//\"/}
devid=${devid//:/}
devid=${devid//,/}
echo $devid
# some checking could be done here to make sure the value is valid
# populate the array
hardwareInfo[$hwsn]=$devid
done
# spacer, for readability of the output
echo
# display the array; in your script, you would do something different and useful
for key in "${!hardwareInfo[#]}"; do echo $key --- ${hardwareInfo[$key]}; done
cat so.input | ./so.sh
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
123456789101
devid1234
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
111213141516
devid5678
111213141516 --- devid5678
123456789101 --- devid1234
I created the input file so.input just for convenience. You would probably pipe your grep output into the bash script, like so:
grep-command | ./so.sh
EDIT #1: There are lots of choices for parsing out the key and value from the strings fed in by grep; the answer from #David C. Rankin shows another way. The best way depends on what you can rely on about the content and structure of the grep output.
There are also several choices for reading two separate lines that are related to each other; David's "toggle" approach is also good, and commonly used; I considered it myself, before going with "read two lines and stop if either is blank".
EDIT #2: I see declare -A in David's answer and in examples on the web; I used declare -a because that's what my version of bash wants (I'm using a Mac). So, just be aware that there can be differences.
I have wrote a script that throws the output of running node processes with the cwd of that process and I store the value in an array using for loop and do echo that array.
How can I able to get the user enter the index of array regarding the output that the script throws and show the output against that input generated by user
Example Myscript
array=$(netstat -nlp | grep node)
for i in ${array[*]}
do
echo $i
done
output is something like that
1056
2064
3024
I want something more advance. I want to take input from user like
Enter the regarding index from above list = 1
And lets suppose user enter 1
Then next output should be
Your selected value is 2064
Is it possible in bash
First, you're not actually using an array, you are storing a plain string in the variable "array". The string contains words separated by whitespace, so when you supply the variable in the for statement, the unquoted value is subject to Word Splitting
You need to use the array syntax for setting the array:
array=( $(netstat -nlp | grep node) )
However, the unquoted command substitution still exposes you to Filename Expansion. The best way to store the lines of a command into an array is to use the mapfile command with a process substitution:
mapfile -t array < <(netstat -nlp | grep node)
And in the for loop, make sure you quote all the variables and use index #
for i in "${array[#]}"; do
echo "$i"
done
Notes:
arrays created with mapfile will start at index 0, so be careful of off-by-one errors
I don't know how variables are implemented in bash, but there is this oddity:
if you refer to the array without an index, you'll get the first element:
array=( "hello" "world" )
echo "$array" # ==> hello
If you refer to a plain variable with array syntax and index zero, you'll get the value:
var=1234
echo "${var[0]}" # ==> 1234
I am trying to write code to break up a large array into many different small arrays. Eventually the array I would be passed is one of unknown size, this is just my test subject. I have gotten this far:
#!/bin/bash
num=(10 3 12 3 4 4)
inArray=${#num[#]}
numArrays=$(($inArray/2))
remain=$(($inArray%2))
echo $numArrays
echo $remain
nun=0
if test $remain -gt $nun; then
numArrays=$(($numArrays+1))
fi
array=(1 2)
j=0
for ((i=0;i<$numArrays;i++, j=j+2)); do
array=("${num[#]:$j:2}")
echo "The array says: ${array[#]}"
echo "The size? ${#array[#]}"
done
What I am really having a problem with is : I would like to make the variable 'array' be able to change names slightly every time, so each array is kept and has a unique name after the loop. I have tried making the name array_$i but that returns:
[Stephanie#~]$ ./tmp.sh
3
0
./tmp.sh: line 16: syntax error near unexpected token `"${num[#]:$j:2}"'
./tmp.sh: line 16: ` array_$i=("${num[#]:$j:2}")'
[Stephanie#RDT00069 ~]$ ./tmp.sh
3
0
./tmp.sh: line 16: syntax error near unexpected token `$i'
./tmp.sh: line 16: ` array($i)=("${num[#]:$j:2}")'
Does anyone have any advice?
Thanks
I don't think you can really avoid eval here, but you might be able to do it safely if you're careful. Here's my approach:
for name in "${!array_*}"; do # Get all names starting with array_
i="${name#array_*}" # Get the part after array_
if [[ $i != *[^0-9]* ]]; then # Check that it's a number.
printf '%s is not a valid subarray name\n' "$name"
else
# Create a variable named "statement" that contains code you want to eval.
printf -v statement 'cur_array=( "${%s[#]}" )' "$name"
eval "$statement"
# Do interesting things with $cur_array
fi
done
Before this, when you're just creating the array, you know what $name should be, so just use the printf -v part.
To make it even safer, you could save all the allowed array names in another array and check that $name is a member.
With simple variables, you can use the declare keyword to make indirect assignments:
v=foo
declare $v=5
echo $foo # Prints 5
This doesn't extend to arrays in the obvious (to me, anyway) sense:
i=2
# This produces a syntax error
declare -a array_$i=("${num[#]:$j:2}")
Instead, you can declare an empty array
declare -a array_$i
or assign items one at a time:
declare -a array_$i[0]=item1 array_$i[1]=item2
Here's an example of using a for-loop to copy, say, the 3rd
and 4th letters of a big array into a smaller one. We use
i as the dynamic part of the name of the smaller array, and
j as the index into that array.
letters=(a b c d e f)
i=1
j=0
for letter in "${letters[#]:2:2}"; do
# E.g., i=0 and j=1 would result in
# declare -a array_0[1]=c
declare -a array_$i[$j]=$letter
let j+=1
done
done
echo ${array_1[#]}; # c d
${foo[#]:x:y} gives us elements x, x+1, ..., x+y-1 from foo, and
You can wrap the whole thing inside another for-loop to accomplish the goal of splitting letters into 3 smaller arrays:
# We'll create array_0, array_1, and array_2
for i in 0 1 2; do
# Just like our subset above, but start at position i*2 instead of
# a constant.
for letter in "${letters[#]:$((i*2)):2}"; do
declare -a array_$i[$j]=$letter
done
done
Once you manage to populate your three arrays, how do you access them without eval? Bash has syntax for indirect access:
v=foo
foo=5
echo ${!v} # echoes 5!
The exclamation point says to use the word that follows as a variable whose value should be used as the name of the parameter to expand. Knowing that, you might think you could do the following, but you'd be wrong.
i=1
v=array_$i # array_1
echo ${!v[0]} # array_1[0] is c, so prints c, right? Wrong.
In the above, bash tries to find a variable called v[0] and expand it to get the name of a parameter to expand. We actually have to treat our array plus its index as a single name:
i=1
v=array_$i[0]
echo ${!v} # This does print c
This should work, but this is not a good solution, another language may be better bash does not support multi dimensional arrays
eval array_$i='('"${num[#]:$j:2}"')'
And then, for example
eval 'echo "${array_'$i'[0]}"'