Say I have an array in zsh
a=(1 2 3)
I want to append .txt to each element
echo ${a}.txt # this doesn't work
So the output is
1.txt 2.txt 3.txt
UPDATE:
I guess I can do this, but I think there's a more idiomatic way:
for i in $a; do
echo $i.txt
done
You need to set RC_EXPAND_PARAM option:
$ setopt RC_EXPAND_PARAM
$ echo ${a}.txt
1.txt 2.txt 3.txt
From zsh manual:
RC_EXPAND_PARAM (-P)
Array expansions of the form `foo${xx}bar', where the parameter xx is set to
(a b c), are substituted with `fooabar foobbar foocbar' instead of the
default `fooa b cbar'. Note that an empty array will therefore cause all
arguments to be removed.
You can also set this option just for for one array expansion using ^ flag:
$ echo ${^a}.txt
1.txt 2.txt 3.txt
$ echo ${^^a}.txt
1 2 3.txt
Again citing zsh manual:
${^spec}
Turn on the RC_EXPAND_PARAM option for the evaluation of spec; if the `^' is
doubled, turn it off. When this option is set, array expansions of the form
foo${xx}bar, where the parameter xx is set to (a b c), are substituted with
`fooabar foobbar foocbar' instead of the default `fooa b cbar'. Note that an
empty array will therefore cause all arguments to be removed.
Related
What would be a bash equivalent of the following R function?
vec=4:9
out=paste0("foo_",vec,"_bar")
out
"foo_4_bar" "foo_5_bar" "foo_6_bar" "foo_7_bar" "foo_8_bar" "foo_9_bar"
You can use declare an array with suffix and prefix and then use brace expansion to populate incrementing numbers:
arr=("foo_" "_bar") # array with suffix and prefix
echo "${arr[0]}"{4..9}"${arr[1]}" # brace expansion
foo_4_bar foo_5_bar foo_6_bar foo_7_bar foo_8_bar foo_9_bar
You can use brace expansion:
$ echo foo_{4..9}_bar
foo_4_bar foo_5_bar foo_6_bar foo_7_bar foo_8_bar foo_9_bar
$ out=( foo_{4..9}_bar )
$ echo "${out[1]}"
foo_5_bar
This works even if your vec is not generated via a brace expansion:
vec=( {4..9} ) # would work even with vec=( *.txt ) or readarray -t vec <file, etc.
out=( "${vec[#]/#/foo_}" ) # add foo_ prefix
out=( "${out[#]/%/_bar}" ) # add _bar suffix
declare -p out # print resulting array definition
See the Parameter Expansion page on the bash-hackers wiki, particularly the "Anchoring" section under "Search and Replace".
I'm struggling with a project. I am supposed to write a bash script which will work like tr command. At the beginning I would like to save all commands arguments into separated arrays. And in case if an argument is a word I would like to have each char in separated array field,eg.
tr_mine AB DC
I would like to have two arrays: a[0] = A, a[1] = B and b[0]=C b[1]=D.
I found a way, but it's not working:
IFS="" read -r -a array <<< "$a"
No sed, no awk, all bash internals.
Assuming that words are always separated with blanks (space and/or tabs),
also assuming that words are given as arguments, and writing for bash only:
#!/bin/bash
blank=$'[ \t]'
varname='A'
n=1
while IFS='' read -r -d '' -N 1 c ; do
if [[ $c =~ $blank ]]; then n=$((n+1)); continue; fi
eval ${varname}${n}'+=("'"$c"'")'
done <<<"$#"
last=$(eval echo \${#${varname}${n}[#]}) ### Find last character index.
unset "${varname}${n}[$last-1]" ### Remove last (trailing) newline.
for ((j=1;j<=$n;j++)); do
k="A$j[#]"
printf '<%s> ' "${!k}"; echo
done
That will set each array A1, A2, A3, etc. ... to the letters of each word.
The value at the end of the first loop of $n is the count of words processed.
Printing may be a little tricky, that is why the code to access each letter is given above.
Applied to your sample text:
$ script.sh AB DC
<A> <B>
<D> <C>
The script is setting two (array) vars A1 and A2.
And each letter is one array element: A1[0] = A, A1[1] = B and A2[0]=C, A2[1]=D.
You need to set a variable ($k) to the array element to access.
For example, to echo fourth letter (0 based) of second word (1 based) you need to do (that may be changed if needed):
k="A2[3]"; echo "${!k}" ### Indirect addressing.
The script will work as this:
$ script.sh ABCD efghi
<A> <B> <C> <D>
<e> <f> <g> <h> <i>
Caveat: Characters will be split even if quoted. However, quoted arguments is the correct way to use this script to avoid the effect of shell metacharacters ( |,&,;,(,),<,>,space,tab ). Of course, spaces (even if repeated) will split words as defined by the variable $blank:
$ script.sh $'qwer;rttt fgf\ngfg'
<q> <w> <e> <r> <;> <r> <t> <t> <t>
<>
<>
<>
<f> <g> <f> <
> <g> <f> <g>
As the script will accept and correctly process embebed newlines we need to use: unset "${varname}${n}[$last-1]" to remove the last trailing "newline". If that is not desired, quote the line.
Security Note: The eval is not much of a problem here as it is only processing one character at a time. It would be difficult to create an attack based on just one character. Anyway, the usual warning is valid: Always sanitize your input before using this script. Also, most (not quoted) metacharacters of bash will break this script.
$ script.sh qwer(rttt fgfgfg
bash: syntax error near unexpected token `('
I would strongly suggest to do this in another language if possible, it will be a lot easier.
Now, the closest I come up with is:
#!/bin/bash
sentence="AC DC"
words=`echo "$sentence" | tr " " "\n"`
# final array
declare -A result
# word count
wc=0
for i in $words; do
# letter count in the word
lc=0
for l in `echo "$i" | grep -o .`; do
result["w$wc-l$lc"]=$l
lc=$(($lc+1))
done
wc=$(($wc+1))
done
rLen=${#result[#]}
echo "Result Length $rLen"
for i in "${!result[#]}"
do
echo "$i => ${result[$i]}"
done
The above prints:
Result Length 4
w1-l1 => C
w1-l0 => D
w0-l0 => A
w0-l1 => C
Explanation:
Dynamic variables are not supported in bash (ie create variables using variables) so I am using an associative array instead (result)
Arrays in bash are single dimension. To fake a 2D array I use the indexes: w for words and l for letters. This will make further processing a pain...
Associative arrays are not ordered thus results appear in random order when printing
${!result[#]} is used instead of ${result[#]}. The first iterates keys while the second iterates values
I know this is not exactly what you ask for, but I hope it will point you to the right direction
Try this :
sentence="$#"
read -r -a words <<< "$sentence"
for word in ${words[#]}; do
inc=$(( i++ ))
read -r -a l${inc} <<< $(sed 's/./& /g' <<< $word)
done
echo ${words[1]} # print "CD"
echo ${l1[1]} # print "D"
The first read reads all words, the internal one is for letters.
The sed command add a space after each letters to make the string splittable by read -a. You can also use this sed command to remove unwanted characters from words (eg commas) before splitting.
If special characters are allowed in words, you can use a simple grep instead of the sed command (as suggested in http://www.unixcl.com/2009/07/split-string-to-characters-in-bash.html) :
read -r -a l${inc} <<< $(grep -o . <<< $word)
The word array is ${w}.
The letters arrays are named l# where # is an increment added for each word read.
According to this reference sheet on hyperpolyglot.org, the following syntax can be used to set an array.
i=(1 2 3)
But I get an error with dash which is the default for /bin/sh on Ubuntu and should be POSIX compliant.
# Trying the syntax with dash in my terminal
> dash -i
$ i=(1 2 3)
dash: 1: Syntax error: "(" unexpected
$ exit
# Working fine with bash
> bash -i
$ i=(1 2 3)
$ echo ${i[#]}
1 2 3
$ exit
Is the reference sheet misleading or erroneous?
If yes, what would be the correct way to define an array or a list and be POSIX compliant?
Posix does not specify arrays, so if you are restricted to Posix shell features, you cannot use arrays.
I'm afraid your reference is mistaken. Sadly, not everything you find on the internet is correct.
As said by rici, dash doesn't have array support. However, there are workarounds if what you're looking to do is write a loop.
For loop won't do arrays, but you can do the splitting using a while loop + the read builtin. Since the dash read builtin also doesn't support delimiters, you would have to work around that too.
Here's a sample script:
myArray="a b c d"
echo "$myArray" | tr ' ' '\n' | while read item; do
# use '$item'
echo $item
done
Some deeper explanation on that:
The tr ' ' '\n' will let you do a single-character replace where
you remove the spaces & add newlines - which are the default delim
for the read builtin.
read will exit with a failing exit code when it detects that stdin
has been closed - which would be when your input has been fully
processed.
Since echo prints an extra newline after its input, that will let you
process the last "element" in your array.
This would be equivalent to the bash code:
myArray=(a b c d)
for item in ${myArray[#]}; do
echo $item
done
If you want to retrieve the n-th element (let's say 2-th for the purpose of the example):
myArray="a b c d"
echo $myArray | cut -d\ -f2 # change -f2 to -fn
It is true that the POSIX sh shell does not have named arrays in the same sense that bash and other shells have, but there is a list that sh shells (as well as bash and others) could use, and that's the list of positional parameters.
This list usually contains the arguments passed to the current script or shell function, but you can set its values with the set built-in command:
#!/bin/sh
set -- this is "a list" of "several strings"
In the above script, the positional parameters $1, $2, ..., are set to the five string shown. The -- is used to make sure that you don't unexpectedly set a shell option (which the set command is also able to do). This is only ever an issue if the first argument starts with a - though.
To e.g. loop over these strings, you can use
for string in "$#"; do
printf 'Got the string "%s"\n' "$string"
done
or the shorter
for string do
printf 'Got the string "%s"\n' "$string"
done
or just
printf 'Got the string "%s"\n' "$#"
set is also useful for expanding globs into lists of pathnames:
#!/bin/sh
set -- "$HOME"/*/
# "visible directory" below really means "visible directory, or visible
# symbolic link to a directory".
if [ ! -d "$1" ]; then
echo 'You do not have any visible directories in your home directory'
else
printf 'There are %d visible directories in your home directory\n' "$#"
echo 'These are:'
printf '\t%s\n' "$#"
fi
The shift built-in command can be used to shift off the first positional parameter from the list.
#!/bin/sh
# pathnames
set -- path/name/1 path/name/2 some/other/pathname
# insert "--exclude=" in front of each
for pathname do
shift
set -- "$#" --exclude="$pathname"
done
# call some command with our list of command line options
some_command "$#"
You can use the argument list $# as an array in POSIX shells
It's trivial to initialize, shift, unshift, and push:
# initialize $# containing a string, a variable's value, and a glob's matches
set -- "item 1" "$variable" *.wav
# shift (remove first item, accepts a numeric argument to remove more)
shift
# unshift (prepend new first item)
set -- "new item" "$#"
# push (append new last item)
set -- "$#" "new item"
Here's a pop implementation:
# pop (remove last item, store it in $last)
i=0
for last in "$#"; do
if [ $((i+=1)) = 1 ]; then set --; fi # increment $i. first run: empty $#
if [ $i = $# ]; then break; fi # stop before processing the last item
set -- "$#" "$last" # add $a back to $#
done
echo "$last has been removed from ($*)"
($* puts the contents of $# into a single space-delimited string, ideal for quoting within another string.)
Iterate through the $# array and modify some of its contents:
i=0
for a in "$#"; do
if [ $((i+=1)) = 1 ]; then set --; fi # increment $i. first run: empty $#
a="${a%.*}.mp3" # example tweak to $a: change extension to .mp3
set -- "$#" "$a" # add $a back to $#
done
Refer to items in the $# array:
echo "$1 is the first item"
echo "$# is the length of the array"
echo "all items in the array (properly quoted): $#"
echo "all items in the array (in a string): $*"
[ "$n" -ge 0 ] && eval "echo \"the ${n}th item in the array is \$$n\""
(eval is dangerous, so I've ensured $n is a number before running it)
There are two ways to set $last to the final item of a list without popping it:
with an eval (safe since $# is always a number):
eval last="\$$#"
... or with a loop:
for last in "$#"; do true; done
⚠️ Warning: Functions have their own $# arrays. You'll have to pass it to the function, like my_function "$#" if read-only or else set -- $(my_function "$#") if you want to manipulate $# and don't expect spaces in item values.
If you need to handle spaces in item values, it becomes much more cumbersome:
# ensure my_function() returns each list item on its own line
i=1
my_function "$#" |while IFS= read line; do
if [ $i = 1 ]; then unset i; set --; fi
set -- "$#" "$line"
done
This still won't work with newlines in your items. You'd have to escape them to another character (but not null) and then escape them back later.
Assigning arrays to variables in Bash script seems rather complicated:
a=("a" "b" "c")
b=$a
echo ${a[0]}
echo ${a[1]}
echo ${b[0]}
echo ${b[1]}
leads to
a
b
a
instead of
a
b
a
b
Why? How can I fix it?
If you want to copy a variable that holds an array to another name, you do it like this:
a=('a' 'b' 'c')
b=( "${a[#]}" )
Why?
If a is an array, $a expands to the first element in the array. That is why b in your example only has one value. In bash, variables that refer to arrays aren't assignable like pointers would be in C++ or Java. Instead variables expand (as in Parameter Expansion) into strings and those strings are copied and associated with the variable being assigned.
How can I fix it?
To copy a sparse array that contains values with spaces, the array must be copied one element at a time by the indices - which can be obtained with ${!a[#]}.
declare -a b=()
for i in ${!a[#]}; do
b[$i]="${a[$i]}"
done
From the bash man page:
It is possible to obtain the keys (indices) of an array as well as the values.
${!name[#]} and ${!name[*]} expand to the indices assigned in array variable name.
The treatment when in double quotes is similar to the expansion of the special
parameters # and * within double quotes.
Here's a script you can test on your own:
#!/bin/bash
declare -a a=();
a[1]='red hat'
a[3]='fedora core'
declare -a b=();
# Copy method that works for sparse arrays with spaces in the values.
for i in ${!a[#]}; do
b[$i]="${a[$i]}"
done
# does not work, but as LeVar Burton says ...
#b=("${a[#]}")
echo a indicies: ${!a[#]}
echo b indicies: ${!b[#]}
echo "values in b:"
for u in "${b[#]}"; do
echo $u
done
Prints:
a indicies: 1 3
b indicies: 1 3 # or 0 1 with line uncommented
values in b:
red hat
fedora core
This also works for associative arrays in bash 4, if you use declare -A (with capital A instead of lower case) when declaring the arrays.
I'm trying to construct an array in bash of the filenames from my camera:
FILES=(2011-09-04 21.43.02.jpg
2011-09-05 10.23.14.jpg
2011-09-09 12.31.16.jpg
2011-09-11 08.43.12.jpg)
As you can see, there is a space in the middle of each filename.
I've tried wrapping each name in quotes, and escaping the space with a backslash, neither of which works.
When I try to access the array elements, it continues to treat the space as the elementdelimiter.
How can I properly capture the filenames with a space inside the name?
I think the issue might be partly with how you're accessing the elements. If I do a simple for elem in $FILES, I experience the same issue as you. However, if I access the array through its indices, like so, it works if I add the elements either numerically or with escapes:
for ((i = 0; i < ${#FILES[#]}; i++))
do
echo "${FILES[$i]}"
done
Any of these declarations of $FILES should work:
FILES=(2011-09-04\ 21.43.02.jpg
2011-09-05\ 10.23.14.jpg
2011-09-09\ 12.31.16.jpg
2011-09-11\ 08.43.12.jpg)
or
FILES=("2011-09-04 21.43.02.jpg"
"2011-09-05 10.23.14.jpg"
"2011-09-09 12.31.16.jpg"
"2011-09-11 08.43.12.jpg")
or
FILES[0]="2011-09-04 21.43.02.jpg"
FILES[1]="2011-09-05 10.23.14.jpg"
FILES[2]="2011-09-09 12.31.16.jpg"
FILES[3]="2011-09-11 08.43.12.jpg"
There must be something wrong with the way you access the array's items. Here's how it's done:
for elem in "${files[#]}"
...
From the bash manpage:
Any element of an array may be referenced using ${name[subscript]}. ... If subscript is # or *, the word expands to all members of name. These subscripts differ only when the word appears within double quotes. If the word is double-quoted, ${name[*]} expands to a single word with the value of each array member separated by the first character of the IFS special variable, and ${name[#]} expands each element of name to a separate word.
Of course, you should also use double quotes when accessing a single member
cp "${files[0]}" /tmp
You need to use IFS to stop space as element delimiter.
FILES=("2011-09-04 21.43.02.jpg"
"2011-09-05 10.23.14.jpg"
"2011-09-09 12.31.16.jpg"
"2011-09-11 08.43.12.jpg")
IFS=""
for jpg in ${FILES[*]}
do
echo "${jpg}"
done
If you want to separate on basis of . then just do IFS="."
Hope it helps you:)
I agree with others that it's likely how you're accessing the elements that is the problem. Quoting the file names in the array assignment is correct:
FILES=(
"2011-09-04 21.43.02.jpg"
"2011-09-05 10.23.14.jpg"
"2011-09-09 12.31.16.jpg"
"2011-09-11 08.43.12.jpg"
)
for f in "${FILES[#]}"
do
echo "$f"
done
Using double quotes around any array of the form "${FILES[#]}" splits the array into one word per array element. It doesn't do any word-splitting beyond that.
Using "${FILES[*]}" also has a special meaning, but it joins the array elements with the first character of $IFS, resulting in one word, which is probably not what you want.
Using a bare ${array[#]} or ${array[*]} subjects the result of that expansion to further word-splitting, so you'll end up with words split on spaces (and anything else in $IFS) instead of one word per array element.
Using a C-style for loop is also fine and avoids worrying about word-splitting if you're not clear on it:
for (( i = 0; i < ${#FILES[#]}; i++ ))
do
echo "${FILES[$i]}"
done
If you had your array like this:
#!/bin/bash
Unix[0]='Debian'
Unix[1]="Red Hat"
Unix[2]='Ubuntu'
Unix[3]='Suse'
for i in $(echo ${Unix[#]});
do echo $i;
done
You would get:
Debian
Red
Hat
Ubuntu
Suse
I don't know why but the loop breaks down the spaces and puts them as an individual item, even you surround it with quotes.
To get around this, instead of calling the elements in the array, you call the indexes, which takes the full string thats wrapped in quotes.
It must be wrapped in quotes!
#!/bin/bash
Unix[0]='Debian'
Unix[1]='Red Hat'
Unix[2]='Ubuntu'
Unix[3]='Suse'
for i in $(echo ${!Unix[#]});
do echo ${Unix[$i]};
done
Then you'll get:
Debian
Red Hat
Ubuntu
Suse
This was already answered above, but that answer was a bit terse and the man page excerpt is a bit cryptic. I wanted to provide a fully worked example to demonstrate how this works in practice.
If not quoted, an array just expands to strings separated by spaces, so that
for file in ${FILES[#]}; do
expands to
for file in 2011-09-04 21.43.02.jpg 2011-09-05 10.23.14.jpg 2011-09-09 12.31.16.jpg 2011-09-11 08.43.12.jpg ; do
But if you quote the expansion, bash adds double quotes around each term, so that:
for file in "${FILES[#]}"; do
expands to
for file in "2011-09-04 21.43.02.jpg" "2011-09-05 10.23.14.jpg" "2011-09-09 12.31.16.jpg" "2011-09-11 08.43.12.jpg" ; do
The simple rule of thumb is to always use [#] instead of [*] and quote array expansions if you want spaces preserved.
To elaborate on this a little further, the man page in the other answer is explaining that if unquoted, $* an $# behave the same way, but they are different when quoted. So, given
array=(a b c)
Then $* and $# both expand to
a b c
and "$*" expands to
"a b c"
and "$#" expands to
"a" "b" "c"
Not exactly an answer to the quoting/escaping problem of the original question but probably something that would actually have been more useful for the op:
unset FILES
for f in 2011-*.jpg; do FILES+=("$f"); done
echo "${FILES[#]}"
Where of course the expression would have to be adopted to the specific requirement (e.g. *.jpg for all or 2001-09-11*.jpg for only the pictures of a certain day).
For those who prefer set array in oneline mode, instead of using for loop
Changing IFS temporarily to new line could save you from escaping.
OLD_IFS="$IFS"
IFS=$'\n'
array=( $(ls *.jpg) ) #save the hassle to construct filename
IFS="$OLD_IFS"
Escaping works.
#!/bin/bash
FILES=(2011-09-04\ 21.43.02.jpg
2011-09-05\ 10.23.14.jpg
2011-09-09\ 12.31.16.jpg
2011-09-11\ 08.43.12.jpg)
echo ${FILES[0]}
echo ${FILES[1]}
echo ${FILES[2]}
echo ${FILES[3]}
Output:
$ ./test.sh
2011-09-04 21.43.02.jpg
2011-09-05 10.23.14.jpg
2011-09-09 12.31.16.jpg
2011-09-11 08.43.12.jpg
Quoting the strings also produces the same output.
#! /bin/bash
renditions=(
"640x360 80k 60k"
"1280x720 320k 128k"
"1280x720 320k 128k"
)
for z in "${renditions[#]}"; do
echo "$z"
done
OUTPUT
640x360 80k 60k
1280x720 320k 128k
1280x720 320k 128k
`
Another solution is using a "while" loop instead a "for" loop:
index=0
while [ ${index} -lt ${#Array[#]} ]
do
echo ${Array[${index}]}
index=$(( $index + 1 ))
done
If you aren't stuck on using bash, different handling of spaces in file names is one of the benefits of the fish shell. Consider a directory which contains two files: "a b.txt" and "b c.txt". Here's a reasonable guess at processing a list of files generated from another command with bash, but it fails due to spaces in file names you experienced:
# bash
$ for f in $(ls *.txt); { echo $f; }
a
b.txt
b
c.txt
With fish, the syntax is nearly identical, but the result is what you'd expect:
# fish
for f in (ls *.txt); echo $f; end
a b.txt
b c.txt
It works differently because fish splits the output of commands on newlines, not spaces.
If you have a case where you do want to split on spaces instead of newlines, fish has a very readable syntax for that:
for f in (ls *.txt | string split " "); echo $f; end
If the elements of FILES come from another file whose file names are line-separated like this:
2011-09-04 21.43.02.jpg
2011-09-05 10.23.14.jpg
2011-09-09 12.31.16.jpg
2011-09-11 08.43.12.jpg
then try this so that the whitespaces in the file names aren't regarded as delimiters:
while read -r line; do
FILES+=("$line")
done < ./files.txt
If they come from another command, you need to rewrite the last line like this:
while read -r line; do
FILES+=("$line")
done < <(./output-files.sh)
I used to reset the IFS value and rollback when done.
# backup IFS value
O_IFS=$IFS
# reset IFS value
IFS=""
FILES=(
"2011-09-04 21.43.02.jpg"
"2011-09-05 10.23.14.jpg"
"2011-09-09 12.31.16.jpg"
"2011-09-11 08.43.12.jpg"
)
for file in ${FILES[#]}; do
echo ${file}
done
# rollback IFS value
IFS=${O_IFS}
Possible output from the loop:
2011-09-04 21.43.02.jpg
2011-09-05 10.23.14.jpg
2011-09-09 12.31.16.jpg
2011-09-11 08.43.12.jpg