In bash shell for variables:
#!/bin/bash
set -o nounset
my_var=aaa
unset var
echo "$var"
Because set command is defined to return error if variable is not set, last line returns error:
line 6: var: unbound variable
OK, that is what I want.
Now the same thing with arrays:
#!/bin/bash
set -o nounset
my_array=(aaa bbb)
unset my_array
echo "${my_array[#]}"
But to my surprise last line does not return error. I would like bash script to return error when array is not defined.
${my_array[#]} is similar to $# which is documented to be ignored by nounset:
-u Treat unset variables and parameters other than the special parameters "#" and "*" as an error when performing parameter expansion. If expansion is attempted on an unset variable or parameter, the shell prints an error message, and, if not interactive, exits with a non-zero status.
Returning the array size is not ignored, though. Prepend the following line to make sure the array is not unset:
: ${#my_array[#]}
Related
In Bash scripts, I frequently find this pattern useful, where I first print the command I'm about to execute, then I execute the command:
echo 'Running this cmd: ls -1 "$HOME/temp/some folder with spaces'
ls -1 "$HOME/temp/some folder with spaces"
echo 'Running this cmd: df -h'
df -h
# etc.
Notice the single quotes in the echo command to prevent variable expansion there! The idea is that I want to print the cmd I'm running, exactly as I will type and run the command, then run it!
How do I wrap this up into a function?
Wrapping the command up into a standard bash array, and then printing and calling it, like this, sort-of works:
# Print and run the passed-in command
# USAGE:
# cmd_array=(ls -a -l -F /)
# print_and_run_cmd cmd_array
# See:
# 1. My answer on how to pass regular "indexed" and associative arrays by reference:
# https://stackoverflow.com/a/71060036/4561887 and
# 1. My answer on how to pass associative arrays: https://stackoverflow.com/a/71060913/4561887
print_and_run_cmd() {
local -n array_reference="$1"
echo "Running cmd: ${cmd_array[#]}"
# run the command by calling all elements of the command array at once
${cmd_array[#]}
}
For simple commands like this it works fine:
Usage:
cmd_array=(ls -a -l -F /)
print_and_run_cmd cmd_array
Output:
Running cmd: ls -a -l -F /
(all output of that cmd is here)
But for more-complicated commands it is broken!:
Usage:
cmd_array=(ls -1 "$HOME/temp/some folder with spaces")
print_and_run_cmd cmd_array
Desired output:
Running cmd: ls -1 "$HOME/temp/some folder with spaces"
(all output of that command should be here)
Actual Output:
Running cmd: ls -1 /home/gabriel/temp/some folder with spaces
ls: cannot access '/home/gabriel/temp/some': No such file or directory
ls: cannot access 'folder': No such file or directory
ls: cannot access 'with': No such file or directory
ls: cannot access 'spaces': No such file or directory
The first problem, as you can see, is that $HOME got expanded in the Running cmd: line, when it shouldn't have, and the double quotes around that path argument were removed, and the 2nd problem is that the command doesn't actually run.
How do I fix these 2 problems?
References:
my bash demo program where I have this print_and_run_cmd function: https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world/blob/master/bash/argument_parsing__3_advanced__gen_prog_template.sh
where I first documented how to pass bash arrays by reference, as I do in that function:
Passing arrays as parameters in bash
How to pass an associative array as argument to a function in Bash?
Follow-up question:
Bash: how to print and run a cmd array which has the pipe operator, |, in it
If you've got Bash version 4.4 or later, this function may do what you want:
function print_and_run_cmd
{
local PS4='Running cmd: '
local -
set -o xtrace
"$#"
}
For example, running
print_and_run_cmd echo 'Hello World!'
outputs
Running cmd: echo 'Hello World!'
Hello World!
local PS4='Running cmd: ' sets a prefix for commands printed by the shell when the xtrace option is on. The default is + . Localizing it means that the previous value of PS4 is automatically restored when the function returns.
local - causes any changes to shell options to be reverted automatically when the function returns. In particular, it causes the set -o xtrace on the next line to be automatically undone when the function returns. Support for local - was added in Bash 4.4.
From man bash, under the local [option] [name[=value] ... | - ] section (emphasis added):
If name is -, the set of shell options is made local to the function in which local is invoked: shell options changed using the set builtin inside the function are restored to their original values when the function returns.
set -o xtrace (which is equivalent to set -x) causes the shell to print commands, preceded by the expanded value of PS4, before running them.
See help set.
Check your scripts with shellcheck:
Line 2:
local -n array_reference="$1"
^-- SC2034 (warning): array_reference appears unused. Verify use (or export if used externally).
Line 3:
echo "Running cmd: ${cmd_array[#]}"
^-- SC2145 (error): Argument mixes string and array. Use * or separate argument.
^-- SC2154 (warning): cmd_array is referenced but not assigned.
Line 5:
${cmd_array[#]}
^-- SC2068 (error): Double quote array expansions to avoid re-splitting elements.
You might want to research https://github.com/koalaman/shellcheck/wiki/SC2068 . We fix all errors and we get:
print_and_run_cmd() {
local -n array_reference="$1"
echo "Running cmd: ${array_reference[*]}"
# run the command by calling all elements of the command array at once
"${array_reference[#]}"
}
For me it's odd to pass an array by reference in this case. I would pass the actual values. I often do:
prun() {
# in the style of set -x
# print to stderr, so output can be captured
echo "+ $*" >&2
# or echo "+ ${*#Q}" >&2
# or echo "+$(printf " %q" "$#")" >&2
# or echo "+$(/bin/printf " %q" "$#")" >&2
"$#"
}
prun "${cmd_array[#]}"
How do I fix these 2 problems?
Incorporate into your workflow linters, formatters and static analysis tools, like shellcheck, and check the problems they point out.
And quote variable expansion. It's "${array[#]}".
You can achieve what you want with DEBUG trap :
#!/bin/bash
set -T
trap 'test "$FUNCNAME" = print_and_run_cmd || trap_saved_command="${BASH_COMMAND}"' DEBUG
print_and_run_cmd(){
echo "Running this cmd: ${trap_saved_command#* }"
"$#"
}
outer(){
print_and_run_cmd ls -1 "$HOME/temp/some folder with spaces"
}
outer
# output ->
# Running this cmd: ls -1 "$HOME/temp/some folder with spaces"
# ...
I really like #pjh's answer, so I've marked it as correct. It doesn't fully answer my original question though, so if another answer comes along that does, I may have to change that. Anyway, see #pjh's answer or a full explanation of how the below code works, and what all those lines mean. I've helped edit that answer with some of the sources from man bash and help set.
I'd like to change the formatting and provide some more examples, however, to show that variable expansion does take place within the command. I'd also like to provide one version which passes by reference, and one which does not, so you can choose the call style which you like best.
Here are my examples, showing both call styles (print_and_run1 cmd_array and print_and_run2 "${cmd_array[#]}"):
#!/usr/bin/env bash
# Print and run the passed-in command, which is passed in as an
# array **by reference**.
# See here for a full explanation: https://stackoverflow.com/a/71151669/4561887
# USAGE:
# cmd_array=(ls -a -l -F /)
# print_and_run1 cmd_array
print_and_run1() {
local -n array_reference="$1"
local PS4='Running cmd: '
local -
set -o xtrace
# Call the cmd
"${array_reference[#]}"
}
# Print and run the passed-in command, which is passed in as members
# of an array **by value**.
# See here for a full explanation: https://stackoverflow.com/a/71151669/4561887
# USAGE:
# cmd_array=(ls -a -l -F /)
# print_and_run2 "${cmd_array[#]}"
print_and_run2() {
local PS4='Running cmd: '
local -
set -o xtrace
# Call the cmd
"$#"
}
cmd_array=(ls -1 "$HOME/temp/some folder with spaces")
print_and_run1 cmd_array
echo ""
print_and_run2 "${cmd_array[#]}"
echo ""
Sample run and output:
eRCaGuy_hello_world/bash$ ./print_and_run.sh
Running cmd: ls -1 '/home/gabriel/temp/some folder with spaces'
file1.txt
file2.txt
Running cmd: ls -1 '/home/gabriel/temp/some folder with spaces'
file1.txt
file2.txt
This seems to work too:
print_and_run_cmd() {
echo "Running cmd: $1"
eval "$cmd"
}
cmd='ls -1 "$HOME/temp/some folder with spaces"'
print_and_run_cmd "$cmd"
Output:
Running cmd: ls -1 "$HOME/temp/some folder with spaces"
(result of running the cmd is here)
But now the problem is, if I want to print an expanded version of the cmd too, to verify that part worked properly, I can't, or at least, don't know how.
I have encountered a very curious problem, while trying to learn bash.
Usually trying to print an echo by simply parsing the variable name like this only outputs the first member Hello.
#!/bin/bash
declare -a test
test[0]="Hello"
test[1]="World"
echo $test # Only prints "Hello"
BUT, for some reason this piece of code prints out ALL members of the given array.
#!/bin/bash
declare -a files
counter=0
for file in "./*"
do
files[$counter]=$file
let $((counter++))
done
echo $files # prints "./file1 ./file2 ./file3" and so on
And I can't seem to wrap my head around it on why it outputs the whole array instead of only the first member. I think it has something to do with my usage of the foreach-loop, but I was unable to find any concrete answer. It's driving me crazy!
Please send help!
When you quoted the pattern, you only created a single entry in your array:
$ declare -p files
declare -a files=([0]="./*")
If you had quoted the parameter expansion, you would see
$ echo "$files"
./*
Without the quotes, the expansion is subject to pathname generation, so echo receives multiple arguments, each of which is printed.
To build the array you expected, drop the quotes around the pattern. The results of pathname generation are not subject to further word-splitting (or recursive pathname generation), so no quotes would be needed.
for file in ./*
do
...
done
The following script throws an error:
declare -a service_ports=(22 80 443 445)
ssh root#host 'bash -s' << EOF
export x=0
while [ \$x -le "${#service_ports[#]}" ]
do
echo Port ${service_ports[\$x]} # ERROR HERE
x=\$(( \$x + 1 ))
done
EOF
When I run this bash script I get:
./q.sh: line 6: $x: syntax error: operand expected (error token is "$x")
I need to escape the $x variable because I use a "bash -s" remote shell. When I remove the backslash I only access my local variable and not the one on the server where the script is executed.
Anyone know the solution to access the content of the array?
You can quote the here document terminator declaration to avoid having to escape anything in the input string (of course that also means you can't inject the value of a local variable in the remote script):
my_command << 'EOF'
foo $bar
[...]
EOF
If you need to inject local variable contents, you could instead go for mixed quotes:
my_input='some $literal$ strings'
my_input="${my_input}${my_variable}"
[...]
Also, you can simplify your command in several ways:
You shouldn't have to specify bash -s when running a remote command, unless that is different from your default shell.
There's no point in exporting variables which are not used in subsequent script calls
You don't need to declare -a when you assign an array literal immediately
I am trying to tidy up some bash code. I have a bunch of lines like the example below that set variables, unless set already, using the command-line arguments given to the script.
[[ -z "${myvar1}" && -n $1 ]] && myvar1="$1" && shift
[[ -z "${myvar2}" && -n $1 ]] && myvar2="$1" && shift
...repeated many times...
I thought I'd write a function to do that and call it like this
positional_arg myvar1
positional_arg myvar2
However, this would require the function to access and manipulate the argument list. I am not sure that is possible, so this is what I came up with...
args=("$#")
positional_arg() {
local value=$args
[[ -z "${!1}" && -n "$value" ]] && eval "$1='$value'" && args=(${args[#]:1})
}
The problems that I am aware of with this is that
it depends on a global args array being set before use
the construct args=(${args[#]:1}) which shifts the array isn't whitespace-friendly so this will not work with arguments that contain whitespace.
So, I'd like to discover if it is possible for a function to access and manipulate its script's positional arguments.
Also any alternative suggestions for implementing this fuctionality in bash would be welcome, especially if they overcome the above problems!
(GNU bash, version 4.3.18(1)-release)
First, you might want to consider using simply
: ${myvar1:=$1}
: ${myvar2:=$2}
# etc
which simply sets myvar1 to the value (empty or not) of $1 only if myvar1 doesn't already have a non-empty value.
local args=(${args[#]:1}) does not preserve whitespace, but local args=( "${args[#]:1}") does.
Since $# is used for both shell and function positional arguments, you'll sadly have to copy the shell arguments into a known global before calling your function. Of course, if you actually want to modify the arguments, you'll also have to copy them back after the function returns (something like args=("$#"); my_function; set -- "${args[#]}").
An alternative is to pass the shell arguments as additional function arguments:
my-function () {
local_args=()
while [[ $1 != "--" ]]; do
local_args+=("$1")
shift
done
shift
# $# is now a copy of the shell arguments,
# but you'll still have to copy them to a global
# if you want to make any changes visible post-call
}
my-function arg1 args2 -- "$#"
You can also set the variables indirectly (without potentionally dangerous eval) in a loop:
#!/bin/bash
#set variables in order from "$#"
for var in myvar1 myvar2 v3 x #names of the variables...
do
#use one of the following to assgin to the variable
printf -v "$var" "%s" "$1" && shift #print into variable
#read -r "$var" <<< "$1" && shift #read into variable
#declare "$var"="$1" && shift #declare a variable
done
#print the content of the variables
echo "=$myvar1=$myvar2=$v3=$x="
#or indirectly:
for var in myvar1 myvar2 v3 x
do
echo "$var:>>${!var}<<"
done
echo "Unused/remaining args:"
printf ">>%s<<\n" "$#"
what for the invocation like:
script '1 1' 2 3 '4 4' 5 6
prints
=1 1=2=3=4 4=
myvar1:>>1 1<<
myvar2:>>2<<
v3:>>3<<
x:>>4 4<<
Unused/remaining args:
>>5<<
>>6<<
Anyway, using much position dependent variables is a source of the future hard-debugging... IMHO, in cases when the script must take many different variables, is better to use getopt and use the script as
script -i vali -o valo .... -z valz # ;)
The easiest way would be to call positional_arg and pass it the command line variables with $#
positional_arg $#
Then you will be access the command line arguments within the function, using $1, $2 etc.
$0 will be the name of the script.
If you need to send further variables, you can simple add them after the command-line arguments.
positional_arg $# $test
This will just increase the size of the array.
if this is an issue for you, simple add your other variables before $#
positional_arg $test $#
In this case, you know how many variables you are sending the function, and therefore you can make the allowances in your variable calls within your function.
$0 in this case, still remains as the name of the script, and all other reference are increased by the number of variables you add.
In the example above $1 before the function call, becomes $2 within the function.
I'm trying to declare an empty array in Shell Script but I'm experiencing an error.
#!/bin/bash
list=$#
newlist=()
for l in $list; do
newlist+=($l)
done
echo "new"
echo $newlist
When I execute it, I get test.sh: 5: test.sh: Syntax error: "(" unexpected
Run it with bash:
bash test.sh
And seeing the error, it seems you're actually running it with dash:
> dash test.sh
test.sh: 5: test.sh: Syntax error: "(" unexpected
Only this time you probably used the link to it (/bin/sh -> /bin/dash).
I find following syntax more readable.
declare -a <name of array>
For more details see Bash Guide for Beginners: 10.2. Array variables.
In BASH 4+ you can use the following for declaring an empty Array:
declare -a ARRAY_NAME=()
You can then append new items NEW_ITEM1 & NEW_ITEM2 by:
ARRAY_NAME+=(NEW_ITEM1)
ARRAY_NAME+=(NEW_ITEM2)
Please note that parentheses () is required while adding the new items. This is required so that new items are appended as an Array element. If you did miss the (), NEW_ITEM2 will become a String append to first Array Element ARRAY_NAME[0].
Above example will result into:
echo ${ARRAY_NAME[#]}
NEW_ITEM1 NEW_ITEM2
echo ${ARRAY_NAME[0]}
NEW_ITEM1
echo ${ARRAY_NAME[1]}
NEW_ITEM2
Next, if you performed (note the missing parenthesis):
ARRAY_NAME+=NEW_ITEM3
This will result into:
echo ${ARRAY_NAME[#]}
NEW_ITEM1NEW_ITEM3 NEW_ITEM2
echo ${ARRAY_NAME[0]}
NEW_ITEM1NEW_ITEM3
echo ${ARRAY_NAME[1]}
NEW_ITEM2
Thanks to #LenW for correcting me on append operation.
Try this to see if you are oriented to dash or bash
ls -al /bin/sh
If it says /bin/sh -> /bin/dash, then type this:
sudo rm /bin/sh
sudo ln -s /bin/bash /bin/sh
Then type again:
ls -al /bin/sh*
then must says something like this:
/bin/sh -> /bin/bash
It means that now sh is properly oriented to Bash and your arrays will work.
DOMAINS=(1); if [[ ${DOMAINS-} ]]; then # true
unset DOMAINS; if [[ ${DOMAINS-} ]]; then # false
If the array is empty just do this:
NEWLIST=
You can check it with:
if [ $NEWLIST ] ; then
# do something
fi
a non empty array declaration looks like this:
NEWLIST=('1' '2' '3')
To fill an array during process:
ARRAY=("$(find . -name '*.mp3')")
Hope this helps