In Bash scripts, I frequently find this pattern useful, where I first print the command I'm about to execute, then I execute the command:
echo 'Running this cmd: ls -1 "$HOME/temp/some folder with spaces'
ls -1 "$HOME/temp/some folder with spaces"
echo 'Running this cmd: df -h'
df -h
# etc.
Notice the single quotes in the echo command to prevent variable expansion there! The idea is that I want to print the cmd I'm running, exactly as I will type and run the command, then run it!
How do I wrap this up into a function?
Wrapping the command up into a standard bash array, and then printing and calling it, like this, sort-of works:
# Print and run the passed-in command
# USAGE:
# cmd_array=(ls -a -l -F /)
# print_and_run_cmd cmd_array
# See:
# 1. My answer on how to pass regular "indexed" and associative arrays by reference:
# https://stackoverflow.com/a/71060036/4561887 and
# 1. My answer on how to pass associative arrays: https://stackoverflow.com/a/71060913/4561887
print_and_run_cmd() {
local -n array_reference="$1"
echo "Running cmd: ${cmd_array[#]}"
# run the command by calling all elements of the command array at once
${cmd_array[#]}
}
For simple commands like this it works fine:
Usage:
cmd_array=(ls -a -l -F /)
print_and_run_cmd cmd_array
Output:
Running cmd: ls -a -l -F /
(all output of that cmd is here)
But for more-complicated commands it is broken!:
Usage:
cmd_array=(ls -1 "$HOME/temp/some folder with spaces")
print_and_run_cmd cmd_array
Desired output:
Running cmd: ls -1 "$HOME/temp/some folder with spaces"
(all output of that command should be here)
Actual Output:
Running cmd: ls -1 /home/gabriel/temp/some folder with spaces
ls: cannot access '/home/gabriel/temp/some': No such file or directory
ls: cannot access 'folder': No such file or directory
ls: cannot access 'with': No such file or directory
ls: cannot access 'spaces': No such file or directory
The first problem, as you can see, is that $HOME got expanded in the Running cmd: line, when it shouldn't have, and the double quotes around that path argument were removed, and the 2nd problem is that the command doesn't actually run.
How do I fix these 2 problems?
References:
my bash demo program where I have this print_and_run_cmd function: https://github.com/ElectricRCAircraftGuy/eRCaGuy_hello_world/blob/master/bash/argument_parsing__3_advanced__gen_prog_template.sh
where I first documented how to pass bash arrays by reference, as I do in that function:
Passing arrays as parameters in bash
How to pass an associative array as argument to a function in Bash?
Follow-up question:
Bash: how to print and run a cmd array which has the pipe operator, |, in it
If you've got Bash version 4.4 or later, this function may do what you want:
function print_and_run_cmd
{
local PS4='Running cmd: '
local -
set -o xtrace
"$#"
}
For example, running
print_and_run_cmd echo 'Hello World!'
outputs
Running cmd: echo 'Hello World!'
Hello World!
local PS4='Running cmd: ' sets a prefix for commands printed by the shell when the xtrace option is on. The default is + . Localizing it means that the previous value of PS4 is automatically restored when the function returns.
local - causes any changes to shell options to be reverted automatically when the function returns. In particular, it causes the set -o xtrace on the next line to be automatically undone when the function returns. Support for local - was added in Bash 4.4.
From man bash, under the local [option] [name[=value] ... | - ] section (emphasis added):
If name is -, the set of shell options is made local to the function in which local is invoked: shell options changed using the set builtin inside the function are restored to their original values when the function returns.
set -o xtrace (which is equivalent to set -x) causes the shell to print commands, preceded by the expanded value of PS4, before running them.
See help set.
Check your scripts with shellcheck:
Line 2:
local -n array_reference="$1"
^-- SC2034 (warning): array_reference appears unused. Verify use (or export if used externally).
Line 3:
echo "Running cmd: ${cmd_array[#]}"
^-- SC2145 (error): Argument mixes string and array. Use * or separate argument.
^-- SC2154 (warning): cmd_array is referenced but not assigned.
Line 5:
${cmd_array[#]}
^-- SC2068 (error): Double quote array expansions to avoid re-splitting elements.
You might want to research https://github.com/koalaman/shellcheck/wiki/SC2068 . We fix all errors and we get:
print_and_run_cmd() {
local -n array_reference="$1"
echo "Running cmd: ${array_reference[*]}"
# run the command by calling all elements of the command array at once
"${array_reference[#]}"
}
For me it's odd to pass an array by reference in this case. I would pass the actual values. I often do:
prun() {
# in the style of set -x
# print to stderr, so output can be captured
echo "+ $*" >&2
# or echo "+ ${*#Q}" >&2
# or echo "+$(printf " %q" "$#")" >&2
# or echo "+$(/bin/printf " %q" "$#")" >&2
"$#"
}
prun "${cmd_array[#]}"
How do I fix these 2 problems?
Incorporate into your workflow linters, formatters and static analysis tools, like shellcheck, and check the problems they point out.
And quote variable expansion. It's "${array[#]}".
You can achieve what you want with DEBUG trap :
#!/bin/bash
set -T
trap 'test "$FUNCNAME" = print_and_run_cmd || trap_saved_command="${BASH_COMMAND}"' DEBUG
print_and_run_cmd(){
echo "Running this cmd: ${trap_saved_command#* }"
"$#"
}
outer(){
print_and_run_cmd ls -1 "$HOME/temp/some folder with spaces"
}
outer
# output ->
# Running this cmd: ls -1 "$HOME/temp/some folder with spaces"
# ...
I really like #pjh's answer, so I've marked it as correct. It doesn't fully answer my original question though, so if another answer comes along that does, I may have to change that. Anyway, see #pjh's answer or a full explanation of how the below code works, and what all those lines mean. I've helped edit that answer with some of the sources from man bash and help set.
I'd like to change the formatting and provide some more examples, however, to show that variable expansion does take place within the command. I'd also like to provide one version which passes by reference, and one which does not, so you can choose the call style which you like best.
Here are my examples, showing both call styles (print_and_run1 cmd_array and print_and_run2 "${cmd_array[#]}"):
#!/usr/bin/env bash
# Print and run the passed-in command, which is passed in as an
# array **by reference**.
# See here for a full explanation: https://stackoverflow.com/a/71151669/4561887
# USAGE:
# cmd_array=(ls -a -l -F /)
# print_and_run1 cmd_array
print_and_run1() {
local -n array_reference="$1"
local PS4='Running cmd: '
local -
set -o xtrace
# Call the cmd
"${array_reference[#]}"
}
# Print and run the passed-in command, which is passed in as members
# of an array **by value**.
# See here for a full explanation: https://stackoverflow.com/a/71151669/4561887
# USAGE:
# cmd_array=(ls -a -l -F /)
# print_and_run2 "${cmd_array[#]}"
print_and_run2() {
local PS4='Running cmd: '
local -
set -o xtrace
# Call the cmd
"$#"
}
cmd_array=(ls -1 "$HOME/temp/some folder with spaces")
print_and_run1 cmd_array
echo ""
print_and_run2 "${cmd_array[#]}"
echo ""
Sample run and output:
eRCaGuy_hello_world/bash$ ./print_and_run.sh
Running cmd: ls -1 '/home/gabriel/temp/some folder with spaces'
file1.txt
file2.txt
Running cmd: ls -1 '/home/gabriel/temp/some folder with spaces'
file1.txt
file2.txt
This seems to work too:
print_and_run_cmd() {
echo "Running cmd: $1"
eval "$cmd"
}
cmd='ls -1 "$HOME/temp/some folder with spaces"'
print_and_run_cmd "$cmd"
Output:
Running cmd: ls -1 "$HOME/temp/some folder with spaces"
(result of running the cmd is here)
But now the problem is, if I want to print an expanded version of the cmd too, to verify that part worked properly, I can't, or at least, don't know how.
I am trying to write a script that manages dotfiles for me, and this is what I have:
dots=(
~/.config/alacritty
~/.config/dunst
~/.config/flameshot
~/.config/i3
~/.config/i3status
~/.config/kitty
~/.config/nvim
~/.config/picom
~/.config/rofi
~/.config/gtk-3.0
~/.config/obs-studio
~/.config/sway
~/.config/waybar
~/.config/swappy
~/.config/mako
~/.config/swaylock
~/.config/Code/User/settings.json
~/.config/Code/User/keybindings.json
~/.config/fontconfig
~/.bash_logout
~/.bash_profile
~/.bashrc
~/.profile
~/.stignore
~/.gtkrc-2.0
)
for name in "${dots[#]}"; do
if [ ! -e "$name" ]; then
echo "$name does not exist."
ln -sv "$HOME/Dotfiles/dots/${name#~/.config/}" "$name"
else
echo "$name exists."
fi
done
The issue here is that it removes the ~/.config/ part from directories like ~/.config/sway, but it doesn't remove the ~/ part for things like ~/.profile.
My question is, how would I make it where it removed the ~/.config/ part and the ~/ part for files that have them?
Edit: Replaced echo with ln -sv in the symlink line. I replaced it with echo so that I can test if it works or not.
My question is, how would I make it where it removed the ~/.config/ part and the ~/ part for files that have them?
Personally, I'd do it in two steps, storing the intermediate result in a separate variable:
local_part=${name#~/}
ln -sv "$HOME/Dotfiles/dots/${local_part#.config/}" "$name"
Given a text file with paths (on each line a new path).
Need to create directories from paths. This is done easily like mkdir -p is/isapp/ip/ipapp. Then chgrp group1 is/isapp/ip/ipapp. But the problem is that access only changes for the final ipapp directory. And you need to change access for all newly created directories, while not changing access for directories that already existed before the mkdir -p command. Therefore, you need to check which directories already exist and change permissions only for newly created directories. Below I tried to split the path from file and gradually increase it until the moment when the search does not find the directory. And then chgrp -R with the path to the directory that was not found. Below are my code sketches. I would be grateful for any help.
#!/bin/bash
FILE=$1 /(file with paths(in each line new path))
while read LINE; do
IFS='/' read -ra my_array <<< "$my_string"
if ! [ -d "${my_array[0]}" ]; then
mkdir -p "${my_array[0]}"
else -d "${my_array[0]}"/"${my_array[#]}"
done
fi
Something like this would work: (basically for each directory level try to cd up, and if you can't create the directory with the proper permissions).
#!/bin/bash
MODE=u+rw
ROOT=$(pwd)
while read PATH; do
IFS='/' read -r -a dirs <<< "${PATH}"
for dir in "${dirs[#]}"; do
[ -d "${dir}" ] || mkdir "${dir}" -m ${MODE} || exit 1
cd "${dir}"
done
cd "${ROOT}"
done
Note: this is reading from stdin (so you would have to pipe your file into the script), or alternatively add < ${FILE} right after the done to pipe it in manually). The quotes around the "${dir}" and "${dirs[#]}" are required in case there are any whitespace characters in the filenames.
The exit 1 saves you in case the mkdir fails (say there's a file with the name of the directory you want to create).
Just some background, I have a file with 1000 servers in it new line delimted. I have to read them to an array the run about 5 commands over SSH. I have been using heredoc notation but that seems to fail. Currently I get an error saying the host isn't recognized.
IFS='\n' read -d '' -r -a my_arr < file
my_arr=()
for i in "${my_arr[#]}"; do
ssh "$1" bash -s << "EOF"
echo "making back up of some file"
cp /path/to/file /path/to/file.bak
exit
EOF
done
I get output that lists the first server but then all the ones in the array as well. I know that I am missing a redirect for STDIN that causes this.
Thanks for the help.
Do you need an array? What is wrong with:
while read -r host
do
ssh "$host" bash -s << "EOF"
echo "making back up of some file"
cp /path/to/file /path/to/file.bak
EOF
done < file
To be clear -- the problem here, and the only problem present in the code actually included in your question, is that you're using $1 inside your loop, whereas you specified $i as the variable that contains the entry being iterated over on each invocation of the loop.
That is to say: ssh "$1" needs to instead by ssh "$i".
I'm working on a project where I have to process the contents of a directory passed in as an argument, and I need to include invisible files (ones that start with .) as well. This is how I'm approaching it
#!/bin/bash
cd $1
for file in `dir -a -d * `;
do
#more code blah blah
even though I use the -a tag on the dir command, it still ignores invisible files. Any ideas why?
Just do:
#!/bin/bash
shopt -s dotglob
cd "$1"
for file in *; do
# more code blah blah
done
From the bash manpage
When a pattern is used for filename expansion, the character ‘.’ at
the start of a filename or immediately following a slash must be
matched explicitly, unless the shell option dotglob is set.