In bash, if you have an array arr and you want to print all its values, the command
echo ${arr[#]}
will do the trick. In sh however, this command gives a bad substitution error. What is an alternative command(s) for this task in sh?
There is no such thing as a general-purpose array in the POSIX sh specification. The closest thing you have available for an arbitrary variable is a string separated by some delimiter; usually whitespace separated, but can be separated by other characters if the elements themselves can contain spaces.
$# can be treated as an array in POSIX sh, but it's a bit limited due to the fact that there's only one such variable. You can change the value of $# with set, so you can do the following:
$ set -- one "two three" four
$ echo "$#"
3
$ echo "$1"
one
$ echo "$2"
two three
$ echo "$3"
four
$ printf '"%s" "%s" "%s"\n' "$#"
"one" "two three" "four"
Couple questions:
- Can you provide any further details on the script and how the array is being initialized?
- Are you sure that you're actually using sh? On some system /bin/sh is a symlink something else like bash.
ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2013-06-04 19:52 /bin/sh -> bash
I would recommend http://www.tutorialspoint.com/unix/unix-using-arrays.htm as a starting point.
Related
The $# variable seems to maintain quoting around its arguments so that, for example:
$ function foo { for i in "$#"; do echo $i; done }
$ foo herp "hello world" derp
herp
hello world
derp
I am also aware that bash arrays, work the same way:
$ a=(herp "hello world" derp)
$ for i in "${a[#]}"; do echo $i; done
herp
hello world
derp
What is actually going on with variables like this? Particularly when I add something to the quote like "duck ${a[#]} goose". If its not space separated what is it?
Usually, double quotation marks in Bash mean "make everything between the quotation marks one word, even if it has separators in it." But as you've noticed, $# behaves differently when it's within double quotes. This is actually a parsing hack that dates back to Bash's predecessor, the Bourne shell, and this special behavior applies only to this particular variable.
Without this hack (I use the term because it seems inconsistent from a language perspective, although it's very useful), it would be difficult for a shell script to pass along its array of arguments to some other command that wants the same arguments. Some of those arguments might have spaces in them, but how would it pass them to another command without the shell either lumping them together as one big word or reparsing the list and splitting the arguments that have whitespace?
Well, you could pass an array of arguments, and the Bourne shell really only has one array, represented by $* or $#, whose number of elements is $# and whose elements are $1, $2, etc, the so-called positional parameters.
An example. Suppose you have three files in the current directory, named aaa, bbb, and cc c (the third file has a space in the name). You can initialize the array (that is, you can set the positional parameters) to be the names of the files in the current directory like this:
set -- *
Now the array of positional parameters holds the names of the files. $#, the number of elements, is three:
$ echo $#
3
And we can iterate over the position parameters in a few different ways.
1) We can use $*:
$ for file in $*; do
> echo "$file"
> done
but that re-separates the arguments on whitespace and calls echo four times:
aaa
bbb
cc
c
2) Or we could put quotation marks around $*:
$ for file in "$*"; do
> echo "$file"
> done
but that groups the whole array into one argument and calls echo just once:
aaa bbb cc c
3) Or we could use $# which represents the same array but behaves differently in double quotes:
$ for file in "$#"; do
> echo "$file"
> done
will produce
aaa
bbb
cc c
because $1 = "aaa", $2 = "bbb", and $3 = "cc c" and "$#" leaves the elements intact. If you leave off the quotation marks around $#, the shell will flatten and re-parse the array, echo will be called four times, and you'll get the same thing you got with a bare $*.
This is especially useful in a shell script, where the positional parameters are the arguments that were passed to your script. To pass those same arguments to some other command -- without the shell resplitting them on whitespace -- use "$#".
# Truncate the files specified by the args
rm "$#"
touch "$#"
In Bourne, this behavior only applies to the positional parameters because it's really the only array supported by the language. But you can create other arrays in Bash, and you can even apply the old parsing hack to those arrays using the special "${ARRAYNAME[#]}" syntax, whose at-sign feels almost like a wink to Mr. Bourne:
$ declare -a myarray
$ myarray[0]=alpha
$ myarray[1]=bravo
$ myarray[2]="char lie"
$ for file in "${myarray[#]}"; do echo "$file"; done
alpha
bravo
char lie
Oh, and about your last example, what should the shell do with "pre $# post" where you have $# within double quotes but you have other stuff in there, too? Recent versions of Bash preserve the array, prepend the text before the $# to the first array element, and append the text after the $# to the last element:
pre aaa
bb
cc c post
I need to read the output of a command in my script into an array. The command is, for example:
ps aux | grep | grep | x
and it gives the output line by line like this:
10
20
30
I need to read the values from the command output into an array, and then I will do some work if the size of the array is less than three.
The other answers will break if output of command contains spaces (which is rather frequent) or glob characters like *, ?, [...].
To get the output of a command in an array, with one line per element, there are essentially 3 ways:
With Bash≥4 use mapfile—it's the most efficient:
mapfile -t my_array < <( my_command )
Otherwise, a loop reading the output (slower, but safe):
my_array=()
while IFS= read -r line; do
my_array+=( "$line" )
done < <( my_command )
As suggested by Charles Duffy in the comments (thanks!), the following might perform better than the loop method in number 2:
IFS=$'\n' read -r -d '' -a my_array < <( my_command && printf '\0' )
Please make sure you use exactly this form, i.e., make sure you have the following:
IFS=$'\n' on the same line as the read statement: this will only set the environment variable IFS for the read statement only. So it won't affect the rest of your script at all. The purpose of this variable is to tell read to break the stream at the EOL character \n.
-r: this is important. It tells read to not interpret the backslashes as escape sequences.
-d '': please note the space between the -d option and its argument ''. If you don't leave a space here, the '' will never be seen, as it will disappear in the quote removal step when Bash parses the statement. This tells read to stop reading at the nil byte. Some people write it as -d $'\0', but it is not really necessary. -d '' is better.
-a my_array tells read to populate the array my_array while reading the stream.
You must use the printf '\0' statement after my_command, so that read returns 0; it's actually not a big deal if you don't (you'll just get an return code 1, which is okay if you don't use set -e – which you shouldn't anyway), but just bear that in mind. It's cleaner and more semantically correct. Note that this is different from printf '', which doesn't output anything. printf '\0' prints a null byte, needed by read to happily stop reading there (remember the -d '' option?).
If you can, i.e., if you're sure your code will run on Bash≥4, use the first method. And you can see it's shorter too.
If you want to use read, the loop (method 2) might have an advantage over method 3 if you want to do some processing as the lines are read: you have direct access to it (via the $line variable in the example I gave), and you also have access to the lines already read (via the array ${my_array[#]} in the example I gave).
Note that mapfile provides a way to have a callback eval'd on each line read, and in fact you can even tell it to only call this callback every N lines read; have a look at help mapfile and the options -C and -c therein. (My opinion about this is that it's a little bit clunky, but can be used sometimes if you only have simple things to do — I don't really understand why this was even implemented in the first place!).
Now I'm going to tell you why the following method:
my_array=( $( my_command) )
is broken when there are spaces:
$ # I'm using this command to test:
$ echo "one two"; echo "three four"
one two
three four
$ # Now I'm going to use the broken method:
$ my_array=( $( echo "one two"; echo "three four" ) )
$ declare -p my_array
declare -a my_array='([0]="one" [1]="two" [2]="three" [3]="four")'
$ # As you can see, the fields are not the lines
$
$ # Now look at the correct method:
$ mapfile -t my_array < <(echo "one two"; echo "three four")
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # Good!
Then some people will then recommend using IFS=$'\n' to fix it:
$ IFS=$'\n'
$ my_array=( $(echo "one two"; echo "three four") )
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # It works!
But now let's use another command, with globs:
$ echo "* one two"; echo "[three four]"
* one two
[three four]
$ IFS=$'\n'
$ my_array=( $(echo "* one two"; echo "[three four]") )
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="t")'
$ # What?
That's because I have a file called t in the current directory… and this filename is matched by the glob [three four]… at this point some people would recommend using set -f to disable globbing: but look at it: you have to change IFS and use set -f to be able to fix a broken technique (and you're not even fixing it really)! when doing that we're really fighting against the shell, not working with the shell.
$ mapfile -t my_array < <( echo "* one two"; echo "[three four]")
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="[three four]")'
here we're working with the shell!
You can use
my_array=( $(<command>) )
to store the output of command <command> into the array my_array.
You can access the length of that array using
my_array_length=${#my_array[#]}
Now the length is stored in my_array_length.
Here is a simple example. Imagine that you are going to put the files and directory names (under the current folder) to an array and count them. The script would be like;
my_array=( `ls` )
my_array_length=${#my_array[#]}
echo $my_array_length
Or, you can iterate over this array by adding the following script:
for element in "${my_array[#]}"
do
echo "${element}"
done
Please note that this is the core concept and the input must be sanitized before the processing, i.e. removing extra characters, handling empty Strings, and etc. (which is out of the topic of this thread).
It helps me all the time suppose you want to copy whole list of directories into current directory into an array
bucketlist=($(ls))
#then print them one by one
for bucket in "${bucketlist[#]}"; do
echo " here is bucket: ${bucket}"
done
I need to read the output of a command in my script into an array. The command is, for example:
ps aux | grep | grep | x
and it gives the output line by line like this:
10
20
30
I need to read the values from the command output into an array, and then I will do some work if the size of the array is less than three.
The other answers will break if output of command contains spaces (which is rather frequent) or glob characters like *, ?, [...].
To get the output of a command in an array, with one line per element, there are essentially 3 ways:
With Bash≥4 use mapfile—it's the most efficient:
mapfile -t my_array < <( my_command )
Otherwise, a loop reading the output (slower, but safe):
my_array=()
while IFS= read -r line; do
my_array+=( "$line" )
done < <( my_command )
As suggested by Charles Duffy in the comments (thanks!), the following might perform better than the loop method in number 2:
IFS=$'\n' read -r -d '' -a my_array < <( my_command && printf '\0' )
Please make sure you use exactly this form, i.e., make sure you have the following:
IFS=$'\n' on the same line as the read statement: this will only set the environment variable IFS for the read statement only. So it won't affect the rest of your script at all. The purpose of this variable is to tell read to break the stream at the EOL character \n.
-r: this is important. It tells read to not interpret the backslashes as escape sequences.
-d '': please note the space between the -d option and its argument ''. If you don't leave a space here, the '' will never be seen, as it will disappear in the quote removal step when Bash parses the statement. This tells read to stop reading at the nil byte. Some people write it as -d $'\0', but it is not really necessary. -d '' is better.
-a my_array tells read to populate the array my_array while reading the stream.
You must use the printf '\0' statement after my_command, so that read returns 0; it's actually not a big deal if you don't (you'll just get an return code 1, which is okay if you don't use set -e – which you shouldn't anyway), but just bear that in mind. It's cleaner and more semantically correct. Note that this is different from printf '', which doesn't output anything. printf '\0' prints a null byte, needed by read to happily stop reading there (remember the -d '' option?).
If you can, i.e., if you're sure your code will run on Bash≥4, use the first method. And you can see it's shorter too.
If you want to use read, the loop (method 2) might have an advantage over method 3 if you want to do some processing as the lines are read: you have direct access to it (via the $line variable in the example I gave), and you also have access to the lines already read (via the array ${my_array[#]} in the example I gave).
Note that mapfile provides a way to have a callback eval'd on each line read, and in fact you can even tell it to only call this callback every N lines read; have a look at help mapfile and the options -C and -c therein. (My opinion about this is that it's a little bit clunky, but can be used sometimes if you only have simple things to do — I don't really understand why this was even implemented in the first place!).
Now I'm going to tell you why the following method:
my_array=( $( my_command) )
is broken when there are spaces:
$ # I'm using this command to test:
$ echo "one two"; echo "three four"
one two
three four
$ # Now I'm going to use the broken method:
$ my_array=( $( echo "one two"; echo "three four" ) )
$ declare -p my_array
declare -a my_array='([0]="one" [1]="two" [2]="three" [3]="four")'
$ # As you can see, the fields are not the lines
$
$ # Now look at the correct method:
$ mapfile -t my_array < <(echo "one two"; echo "three four")
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # Good!
Then some people will then recommend using IFS=$'\n' to fix it:
$ IFS=$'\n'
$ my_array=( $(echo "one two"; echo "three four") )
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # It works!
But now let's use another command, with globs:
$ echo "* one two"; echo "[three four]"
* one two
[three four]
$ IFS=$'\n'
$ my_array=( $(echo "* one two"; echo "[three four]") )
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="t")'
$ # What?
That's because I have a file called t in the current directory… and this filename is matched by the glob [three four]… at this point some people would recommend using set -f to disable globbing: but look at it: you have to change IFS and use set -f to be able to fix a broken technique (and you're not even fixing it really)! when doing that we're really fighting against the shell, not working with the shell.
$ mapfile -t my_array < <( echo "* one two"; echo "[three four]")
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="[three four]")'
here we're working with the shell!
You can use
my_array=( $(<command>) )
to store the output of command <command> into the array my_array.
You can access the length of that array using
my_array_length=${#my_array[#]}
Now the length is stored in my_array_length.
Here is a simple example. Imagine that you are going to put the files and directory names (under the current folder) to an array and count them. The script would be like;
my_array=( `ls` )
my_array_length=${#my_array[#]}
echo $my_array_length
Or, you can iterate over this array by adding the following script:
for element in "${my_array[#]}"
do
echo "${element}"
done
Please note that this is the core concept and the input must be sanitized before the processing, i.e. removing extra characters, handling empty Strings, and etc. (which is out of the topic of this thread).
It helps me all the time suppose you want to copy whole list of directories into current directory into an array
bucketlist=($(ls))
#then print them one by one
for bucket in "${bucketlist[#]}"; do
echo " here is bucket: ${bucket}"
done
If I type:
echo "1 the
dquote> 2 quick
dquote> 3 brown" | while read -a D; do echo "${D[1]}--${D[0]}"; done
in bash it says:
the--1
quick--2
brown--3
but in zsh it says:
zsh: bad option: -a
Why? And what should I do instead?
In both shells read is a builtin. It shares the same purpose, but the implementation and options differ.
In order to read in an array in zsh, read requires the option -A (instead of -a):
echo "1 the
2 quick
3 brown" | while read -A D; do echo $D[2]--$D[1]; done
Note: There are many more differences between zsh and bash:
In zsh arrays are numbered from one by default, in bash they start from zero.
echo $ARRAY prints outputs all elements in zsh but only the first element in bash
To print the third element of an array in sh you can use echo $ARRAY[3]. In bash braces are needed to delimit the subscript, also the subscript for the third element is 2: echo ${ARRAY[2]}.
In zsh you usually do not need to quote parameter expansions in order to handle values with white spaces correctly. For example
FILENAME="no such file"
cat $FILENAME
will print only one error message in zsh:
cat: 'no such file': No such file or directory
but three error messages in bash:
cat: no: No such file or directory
cat: such: No such file or directory
cat: file: No such file or directory
In zsh the builtin echo evaluates escape codes by default. In bash you need to pass the -e argument for that.
echo 'foo\tbar'
zsh:
foo bar
bash:
foo\tbar
…
Generally, it is important to keep in mind that, while zsh and bash are similar, they are far from being the same.
I need to read the output of a command in my script into an array. The command is, for example:
ps aux | grep | grep | x
and it gives the output line by line like this:
10
20
30
I need to read the values from the command output into an array, and then I will do some work if the size of the array is less than three.
The other answers will break if output of command contains spaces (which is rather frequent) or glob characters like *, ?, [...].
To get the output of a command in an array, with one line per element, there are essentially 3 ways:
With Bash≥4 use mapfile—it's the most efficient:
mapfile -t my_array < <( my_command )
Otherwise, a loop reading the output (slower, but safe):
my_array=()
while IFS= read -r line; do
my_array+=( "$line" )
done < <( my_command )
As suggested by Charles Duffy in the comments (thanks!), the following might perform better than the loop method in number 2:
IFS=$'\n' read -r -d '' -a my_array < <( my_command && printf '\0' )
Please make sure you use exactly this form, i.e., make sure you have the following:
IFS=$'\n' on the same line as the read statement: this will only set the environment variable IFS for the read statement only. So it won't affect the rest of your script at all. The purpose of this variable is to tell read to break the stream at the EOL character \n.
-r: this is important. It tells read to not interpret the backslashes as escape sequences.
-d '': please note the space between the -d option and its argument ''. If you don't leave a space here, the '' will never be seen, as it will disappear in the quote removal step when Bash parses the statement. This tells read to stop reading at the nil byte. Some people write it as -d $'\0', but it is not really necessary. -d '' is better.
-a my_array tells read to populate the array my_array while reading the stream.
You must use the printf '\0' statement after my_command, so that read returns 0; it's actually not a big deal if you don't (you'll just get an return code 1, which is okay if you don't use set -e – which you shouldn't anyway), but just bear that in mind. It's cleaner and more semantically correct. Note that this is different from printf '', which doesn't output anything. printf '\0' prints a null byte, needed by read to happily stop reading there (remember the -d '' option?).
If you can, i.e., if you're sure your code will run on Bash≥4, use the first method. And you can see it's shorter too.
If you want to use read, the loop (method 2) might have an advantage over method 3 if you want to do some processing as the lines are read: you have direct access to it (via the $line variable in the example I gave), and you also have access to the lines already read (via the array ${my_array[#]} in the example I gave).
Note that mapfile provides a way to have a callback eval'd on each line read, and in fact you can even tell it to only call this callback every N lines read; have a look at help mapfile and the options -C and -c therein. (My opinion about this is that it's a little bit clunky, but can be used sometimes if you only have simple things to do — I don't really understand why this was even implemented in the first place!).
Now I'm going to tell you why the following method:
my_array=( $( my_command) )
is broken when there are spaces:
$ # I'm using this command to test:
$ echo "one two"; echo "three four"
one two
three four
$ # Now I'm going to use the broken method:
$ my_array=( $( echo "one two"; echo "three four" ) )
$ declare -p my_array
declare -a my_array='([0]="one" [1]="two" [2]="three" [3]="four")'
$ # As you can see, the fields are not the lines
$
$ # Now look at the correct method:
$ mapfile -t my_array < <(echo "one two"; echo "three four")
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # Good!
Then some people will then recommend using IFS=$'\n' to fix it:
$ IFS=$'\n'
$ my_array=( $(echo "one two"; echo "three four") )
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # It works!
But now let's use another command, with globs:
$ echo "* one two"; echo "[three four]"
* one two
[three four]
$ IFS=$'\n'
$ my_array=( $(echo "* one two"; echo "[three four]") )
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="t")'
$ # What?
That's because I have a file called t in the current directory… and this filename is matched by the glob [three four]… at this point some people would recommend using set -f to disable globbing: but look at it: you have to change IFS and use set -f to be able to fix a broken technique (and you're not even fixing it really)! when doing that we're really fighting against the shell, not working with the shell.
$ mapfile -t my_array < <( echo "* one two"; echo "[three four]")
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="[three four]")'
here we're working with the shell!
You can use
my_array=( $(<command>) )
to store the output of command <command> into the array my_array.
You can access the length of that array using
my_array_length=${#my_array[#]}
Now the length is stored in my_array_length.
Here is a simple example. Imagine that you are going to put the files and directory names (under the current folder) to an array and count them. The script would be like;
my_array=( `ls` )
my_array_length=${#my_array[#]}
echo $my_array_length
Or, you can iterate over this array by adding the following script:
for element in "${my_array[#]}"
do
echo "${element}"
done
Please note that this is the core concept and the input must be sanitized before the processing, i.e. removing extra characters, handling empty Strings, and etc. (which is out of the topic of this thread).
It helps me all the time suppose you want to copy whole list of directories into current directory into an array
bucketlist=($(ls))
#then print them one by one
for bucket in "${bucketlist[#]}"; do
echo " here is bucket: ${bucket}"
done