Using declare for referencing variables from an array in bash - arrays

I am trying to loop through an array of directories using a bash script so I can list directories with their timestamp, ownership etc using ls -arlt. I am reviewing bash so would like some feedback.
It works with declare -a for those indirect references but for each directory it outputs and extra directory from the /home/user.
I tried to use declare -n and declare -r for each directory and doesn't work.
#!/bin/bash
# Bash variables
acpi=/etc/acpi
apm=/etc/apm
xml=/etc/xml
array=( acpi apm xml )
# Function to display timestamp, ownership ...
displayInfo()
{
for i in "${array[#]}"; do
declare -n curArray=$i
if [[ -d ${curArray} ]]; then
declare -a _acpi=${curArray[0]} _apm=${curArray[1]} _xml=${curArray[2]}
echo "Displaying folder apci: "
cd $_acpi
ls -alrt
read -p "Press enter to continue"
echo "Displaying folder apm: "
cd $_apm
ls -alrt
read -p "Press enter to continue"
echo "Displaying folder xml: "
cd $_xml
ls -alrt
read -p "Press enter to continue"
else
echo "Displayed Failed" >&2
exit 1
fi
done
}
displayInfo
exit 0
It outputs an extra directory listing the /home/user and don't want that output.

There are a lot of complex and powerful shell features being used here, but in ways that don't fit together or make sense. I'll go over the mistakes in a minute, first let me just give how I'd do it. One thing I will use that you might not be familiar with is indirect variable references with ${!var} -- this is like using a nameref variable, but IMO it's clearer what's going on.
acpi=/etc/acpi
apm=/etc/apm
xml=/etc/xml
array=( acpi apm xml )
displayInfo()
{
for curDirectory in "${array[#]}"; do
if [[ -d ${!curDirectory} ]]; then
echo "Displaying folder $curDirectory:"
ls -alrt "${!curDirectory}"
read -p "Press enter to continue"
else
echo "Error: ${!curDirectory} does not exist or is not a directory" >&2
exit 1
fi
done
}
displayInfo
(One problem with this is that it does the "Press enter to continue" thing after each directory, rather than just between them. This can be fixed, but it's a little more work.)
Ok, now for what went wrong with the original. My main recommendation for you would be to try mentally stepping through your code to see what it's doing. It can help to put set -x before it, so the shell will print its interpretation of what it's doing as it runs, and see how it compares to what you expected. Let's do a short walkthrough of the displayInfo function:
for i in "${array[#]}"; do
This will loop over the contents of array, so on the first pass through the loop i will be set to "acpi". Good so far.
declare -n curArray=$i
This creates a nameref variable pointing to the other variable acpi -- this is similar to what I did with ${! }, and basically reasonable so far. Well, with one exception: the name suggests it's an array, but acpi is a plain variable, not an array.
if [[ -d ${curArray} ]]; then
This checks whether the contents of the acpi variable, "/etc/acpi" is the path of an existing directory (which it is). Still doing good.
declare -a _acpi=${curArray[0]} _apm=${curArray[1]} _xml=${curArray[2]}
Here's where things go completely off the rails. curArray points to the variable acpi, so ${curArray[0]} etc are equivalent to ${acpi[0]} etc. But acpi isn't an array, it's a plain variable, so ${acpi[0]} gets its value, and ${acpi[1]} and ${acpi[2]} get nothing. Furthermore, you're using declare -a (declare arrays), but you're just assigning single values to _acpi, _apm, and _xml. They're declared as arrays, but you're just using them as plain variables (basically the reverse of how you're using curArray -> acpi).
There's a deeper confusion here as well. The for loop above is iterating over "acpi", "apm", and "xml", and we're currently working on "acpi". During this pass through the loop, you should only be working on acpi, not also trying to work on apm and xml. That's the point of having a for loop there.
Ok, that's the main problem here, but let me just point out a couple of other things I'd consider bad practice:
cd $_apm
ls -alrt
Using a variable reference without double-quotes around it like this invites parsing confusion; you should almost always put double-quotes, like cd "$_apm". Also, using cd in a script is dangerous because if it fails the rest of the script will execute in the wrong place. In this case, _apm is empty, so without double-quotes it's equivalent to just cd, which moves to your home directory. This is why you're getting that result. If you used cd "$_apm" it would get an error instead... but since you don't check for that it'll go ahead and still list an irrelevant location.
It's almost always better to avoid cd and its complications entirely, and just use explicit paths, like ls -alrt "$_apm".
echo "Displayed Failed" >&2
exit 1
Do you actually want to exit the entire script if one of the directories doesn't exist? It'd make more sense to me to just return 1 (which exits just the function, not the entire script), or better yet continue (which just goes on to the next iteration of the loop -- i.e. the next directory on the list). I left the exit in my version, but I'd recommend changing it.
One more similar thing:
acpi=/etc/acpi
apm=/etc/apm
xml=/etc/xml
array=( acpi apm xml )
Is there any actual reason to use this array -> variable name -> actual directory path system (and resulting indirect expansion or nameref complications), rather than just having an array of directory paths, like this?
array=( /etc/acpi /etc/apm /etc/xml )
I left the indirection in my version above, but really if there's no reason for it I'd remove the complication.

Related

Why does "echo $array" print all members of the array in this specific case instead of only the first member like in any other case?

I have encountered a very curious problem, while trying to learn bash.
Usually trying to print an echo by simply parsing the variable name like this only outputs the first member Hello.
#!/bin/bash
declare -a test
test[0]="Hello"
test[1]="World"
echo $test # Only prints "Hello"
BUT, for some reason this piece of code prints out ALL members of the given array.
#!/bin/bash
declare -a files
counter=0
for file in "./*"
do
files[$counter]=$file
let $((counter++))
done
echo $files # prints "./file1 ./file2 ./file3" and so on
And I can't seem to wrap my head around it on why it outputs the whole array instead of only the first member. I think it has something to do with my usage of the foreach-loop, but I was unable to find any concrete answer. It's driving me crazy!
Please send help!
When you quoted the pattern, you only created a single entry in your array:
$ declare -p files
declare -a files=([0]="./*")
If you had quoted the parameter expansion, you would see
$ echo "$files"
./*
Without the quotes, the expansion is subject to pathname generation, so echo receives multiple arguments, each of which is printed.
To build the array you expected, drop the quotes around the pattern. The results of pathname generation are not subject to further word-splitting (or recursive pathname generation), so no quotes would be needed.
for file in ./*
do
...
done

How to set arrays with variables with loop in bash [duplicate]

I am confused about a bash script.
I have the following code:
function grep_search() {
magic_way_to_define_magic_variable_$1=`ls | tail -1`
echo $magic_variable_$1
}
I want to be able to create a variable name containing the first argument of the command and bearing the value of e.g. the last line of ls.
So to illustrate what I want:
$ ls | tail -1
stack-overflow.txt
$ grep_search() open_box
stack-overflow.txt
So, how should I define/declare $magic_way_to_define_magic_variable_$1 and how should I call it within the script?
I have tried eval, ${...}, \$${...}, but I am still confused.
I've been looking for better way of doing it recently. Associative array sounded like overkill for me. Look what I found:
suffix=bzz
declare prefix_$suffix=mystr
...and then...
varname=prefix_$suffix
echo ${!varname}
From the docs:
The ‘$’ character introduces parameter expansion, command substitution, or arithmetic expansion. ...
The basic form of parameter expansion is ${parameter}. The value of parameter is substituted. ...
If the first character of parameter is an exclamation point (!), and parameter is not a nameref, it introduces a level of indirection. Bash uses the value formed by expanding the rest of parameter as the new parameter; this is then expanded and that value is used in the rest of the expansion, rather than the expansion of the original parameter. This is known as indirect expansion. The value is subject to tilde expansion, parameter expansion, command substitution, and arithmetic expansion. ...
Use an associative array, with command names as keys.
# Requires bash 4, though
declare -A magic_variable=()
function grep_search() {
magic_variable[$1]=$( ls | tail -1 )
echo ${magic_variable[$1]}
}
If you can't use associative arrays (e.g., you must support bash 3), you can use declare to create dynamic variable names:
declare "magic_variable_$1=$(ls | tail -1)"
and use indirect parameter expansion to access the value.
var="magic_variable_$1"
echo "${!var}"
See BashFAQ: Indirection - Evaluating indirect/reference variables.
Beyond associative arrays, there are several ways of achieving dynamic variables in Bash. Note that all these techniques present risks, which are discussed at the end of this answer.
In the following examples I will assume that i=37 and that you want to alias the variable named var_37 whose initial value is lolilol.
Method 1. Using a “pointer” variable
You can simply store the name of the variable in an indirection variable, not unlike a C pointer. Bash then has a syntax for reading the aliased variable: ${!name} expands to the value of the variable whose name is the value of the variable name. You can think of it as a two-stage expansion: ${!name} expands to $var_37, which expands to lolilol.
name="var_$i"
echo "$name" # outputs “var_37”
echo "${!name}" # outputs “lolilol”
echo "${!name%lol}" # outputs “loli”
# etc.
Unfortunately, there is no counterpart syntax for modifying the aliased variable. Instead, you can achieve assignment with one of the following tricks.
1a. Assigning with eval
eval is evil, but is also the simplest and most portable way of achieving our goal. You have to carefully escape the right-hand side of the assignment, as it will be evaluated twice. An easy and systematic way of doing this is to evaluate the right-hand side beforehand (or to use printf %q).
And you should check manually that the left-hand side is a valid variable name, or a name with index (what if it was evil_code # ?). By contrast, all other methods below enforce it automatically.
# check that name is a valid variable name:
# note: this code does not support variable_name[index]
shopt -s globasciiranges
[[ "$name" == [a-zA-Z_]*([a-zA-Z_0-9]) ]] || exit
value='babibab'
eval "$name"='$value' # carefully escape the right-hand side!
echo "$var_37" # outputs “babibab”
Downsides:
does not check the validity of the variable name.
eval is evil.
eval is evil.
eval is evil.
1b. Assigning with read
The read builtin lets you assign values to a variable of which you give the name, a fact which can be exploited in conjunction with here-strings:
IFS= read -r -d '' "$name" <<< 'babibab'
echo "$var_37" # outputs “babibab\n”
The IFS part and the option -r make sure that the value is assigned as-is, while the option -d '' allows to assign multi-line values. Because of this last option, the command returns with an non-zero exit code.
Note that, since we are using a here-string, a newline character is appended to the value.
Downsides:
somewhat obscure;
returns with a non-zero exit code;
appends a newline to the value.
1c. Assigning with printf
Since Bash 3.1 (released 2005), the printf builtin can also assign its result to a variable whose name is given. By contrast with the previous solutions, it just works, no extra effort is needed to escape things, to prevent splitting and so on.
printf -v "$name" '%s' 'babibab'
echo "$var_37" # outputs “babibab”
Downsides:
Less portable (but, well).
Method 2. Using a “reference” variable
Since Bash 4.3 (released 2014), the declare builtin has an option -n for creating a variable which is a “name reference” to another variable, much like C++ references. Just as in Method 1, the reference stores the name of the aliased variable, but each time the reference is accessed (either for reading or assigning), Bash automatically resolves the indirection.
In addition, Bash has a special and very confusing syntax for getting the value of the reference itself, judge by yourself: ${!ref}.
declare -n ref="var_$i"
echo "${!ref}" # outputs “var_37”
echo "$ref" # outputs “lolilol”
ref='babibab'
echo "$var_37" # outputs “babibab”
This does not avoid the pitfalls explained below, but at least it makes the syntax straightforward.
Downsides:
Not portable.
Risks
All these aliasing techniques present several risks. The first one is executing arbitrary code each time you resolve the indirection (either for reading or for assigning). Indeed, instead of a scalar variable name, like var_37, you may as well alias an array subscript, like arr[42]. But Bash evaluates the contents of the square brackets each time it is needed, so aliasing arr[$(do_evil)] will have unexpected effects… As a consequence, only use these techniques when you control the provenance of the alias.
function guillemots {
declare -n var="$1"
var="«${var}»"
}
arr=( aaa bbb ccc )
guillemots 'arr[1]' # modifies the second cell of the array, as expected
guillemots 'arr[$(date>>date.out)1]' # writes twice into date.out
# (once when expanding var, once when assigning to it)
The second risk is creating a cyclic alias. As Bash variables are identified by their name and not by their scope, you may inadvertently create an alias to itself (while thinking it would alias a variable from an enclosing scope). This may happen in particular when using common variable names (like var). As a consequence, only use these techniques when you control the name of the aliased variable.
function guillemots {
# var is intended to be local to the function,
# aliasing a variable which comes from outside
declare -n var="$1"
var="«${var}»"
}
var='lolilol'
guillemots var # Bash warnings: “var: circular name reference”
echo "$var" # outputs anything!
Source:
BashFaq/006: How can I use variable variables (indirect variables, pointers, references) or associative arrays?
BashFAQ/048: eval command and security issues
Example below returns value of $name_of_var
var=name_of_var
echo $(eval echo "\$$var")
Use declare
There is no need on using prefixes like on other answers, neither arrays. Use just declare, double quotes, and parameter expansion.
I often use the following trick to parse argument lists contanining one to n arguments formatted as key=value otherkey=othervalue etc=etc, Like:
# brace expansion just to exemplify
for variable in {one=foo,two=bar,ninja=tip}
do
declare "${variable%=*}=${variable#*=}"
done
echo $one $two $ninja
# foo bar tip
But expanding the argv list like
for v in "$#"; do declare "${v%=*}=${v#*=}"; done
Extra tips
# parse argv's leading key=value parameters
for v in "$#"; do
case "$v" in ?*=?*) declare "${v%=*}=${v#*=}";; *) break;; esac
done
# consume argv's leading key=value parameters
while test $# -gt 0; do
case "$1" in ?*=?*) declare "${1%=*}=${1#*=}";; *) break;; esac
shift
done
Combining two highly rated answers here into a complete example that is hopefully useful and self-explanatory:
#!/bin/bash
intro="You know what,"
pet1="cat"
pet2="chicken"
pet3="cow"
pet4="dog"
pet5="pig"
# Setting and reading dynamic variables
for i in {1..5}; do
pet="pet$i"
declare "sentence$i=$intro I have a pet ${!pet} at home"
done
# Just reading dynamic variables
for i in {1..5}; do
sentence="sentence$i"
echo "${!sentence}"
done
echo
echo "Again, but reading regular variables:"
echo $sentence1
echo $sentence2
echo $sentence3
echo $sentence4
echo $sentence5
Output:
You know what, I have a pet cat at home
You know what, I have a pet chicken at home
You know what, I have a pet cow at home
You know what, I have a pet dog at home
You know what, I have a pet pig at home
Again, but reading regular variables:
You know what, I have a pet cat at home
You know what, I have a pet chicken at home
You know what, I have a pet cow at home
You know what, I have a pet dog at home
You know what, I have a pet pig at home
This will work too
my_country_code="green"
x="country"
eval z='$'my_"$x"_code
echo $z ## o/p: green
In your case
eval final_val='$'magic_way_to_define_magic_variable_"$1"
echo $final_val
This should work:
function grep_search() {
declare magic_variable_$1="$(ls | tail -1)"
echo "$(tmpvar=magic_variable_$1 && echo ${!tmpvar})"
}
grep_search var # calling grep_search with argument "var"
An extra method that doesn't rely on which shell/bash version you have is by using envsubst. For example:
newvar=$(echo '$magic_variable_'"${dynamic_part}" | envsubst)
For zsh (newers mac os versions), you should use
real_var="holaaaa"
aux_var="real_var"
echo ${(P)aux_var}
holaaaa
Instead of "!"
As per BashFAQ/006, you can use read with here string syntax for assigning indirect variables:
function grep_search() {
read "$1" <<<$(ls | tail -1);
}
Usage:
$ grep_search open_box
$ echo $open_box
stack-overflow.txt
Even though it's an old question, I still had some hard time with fetching dynamic variables names, while avoiding the eval (evil) command.
Solved it with declare -n which creates a reference to a dynamic value, this is especially useful in CI/CD processes, where the required secret names of the CI/CD service are not known until runtime. Here's how:
# Bash v4.3+
# -----------------------------------------------------------
# Secerts in CI/CD service, injected as environment variables
# AWS_ACCESS_KEY_ID_DEV, AWS_SECRET_ACCESS_KEY_DEV
# AWS_ACCESS_KEY_ID_STG, AWS_SECRET_ACCESS_KEY_STG
# -----------------------------------------------------------
# Environment variables injected by CI/CD service
# BRANCH_NAME="DEV"
# -----------------------------------------------------------
declare -n _AWS_ACCESS_KEY_ID_REF=AWS_ACCESS_KEY_ID_${BRANCH_NAME}
declare -n _AWS_SECRET_ACCESS_KEY_REF=AWS_SECRET_ACCESS_KEY_${BRANCH_NAME}
export AWS_ACCESS_KEY_ID=${_AWS_ACCESS_KEY_ID_REF}
export AWS_SECRET_ACCESS_KEY=${_AWS_SECRET_ACCESS_KEY_REF}
echo $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY
aws s3 ls
Wow, most of the syntax is horrible! Here is one solution with some simpler syntax if you need to indirectly reference arrays:
#!/bin/bash
foo_1=(fff ddd) ;
foo_2=(ggg ccc) ;
for i in 1 2 ;
do
eval mine=( \${foo_$i[#]} ) ;
echo ${mine[#]}" " ;
done ;
For simpler use cases I recommend the syntax described in the Advanced Bash-Scripting Guide.
KISS approach:
a=1
c="bam"
let "$c$a"=4
echo $bam1
results in 4
I want to be able to create a variable name containing the first argument of the command
script.sh file:
#!/usr/bin/env bash
function grep_search() {
eval $1=$(ls | tail -1)
}
Test:
$ source script.sh
$ grep_search open_box
$ echo $open_box
script.sh
As per help eval:
Execute arguments as a shell command.
You may also use Bash ${!var} indirect expansion, as already mentioned, however it doesn't support retrieving of array indices.
For further read or examples, check BashFAQ/006 about Indirection.
We are not aware of any trick that can duplicate that functionality in POSIX or Bourne shells without eval, which can be difficult to do securely. So, consider this a use at your own risk hack.
However, you should re-consider using indirection as per the following notes.
Normally, in bash scripting, you won't need indirect references at all. Generally, people look at this for a solution when they don't understand or know about Bash Arrays or haven't fully considered other Bash features such as functions.
Putting variable names or any other bash syntax inside parameters is frequently done incorrectly and in inappropriate situations to solve problems that have better solutions. It violates the separation between code and data, and as such puts you on a slippery slope toward bugs and security issues. Indirection can make your code less transparent and harder to follow.
For indexed arrays, you can reference them like so:
foo=(a b c)
bar=(d e f)
for arr_var in 'foo' 'bar'; do
declare -a 'arr=("${'"$arr_var"'[#]}")'
# do something with $arr
echo "\$$arr_var contains:"
for char in "${arr[#]}"; do
echo "$char"
done
done
Associative arrays can be referenced similarly but need the -A switch on declare instead of -a.
POSIX compliant answer
For this solution you'll need to have r/w permissions to the /tmp folder.
We create a temporary file holding our variables and leverage the -a flag of the set built-in:
$ man set
...
-a Each variable or function that is created or modified is given the export attribute and marked for export to the environment of subsequent commands.
Therefore, if we create a file holding our dynamic variables, we can use set to bring them to life inside our script.
The implementation
#!/bin/sh
# Give the temp file a unique name so you don't mess with any other files in there
ENV_FILE="/tmp/$(date +%s)"
MY_KEY=foo
MY_VALUE=bar
echo "$MY_KEY=$MY_VALUE" >> "$ENV_FILE"
# Now that our env file is created and populated, we can use "set"
set -a; . "$ENV_FILE"; set +a
rm "$ENV_FILE"
echo "$foo"
# Output is "bar" (without quotes)
Explaining the steps above:
# Enables the -a behavior
set -a
# Sources the env file
. "$ENV_FILE"
# Disables the -a behavior
set +a
While I think declare -n is still the best way to do it there is another way nobody mentioned it, very useful in CI/CD
function dynamic(){
export a_$1="bla"
}
dynamic 2
echo $a_2
This function will not support spaces so dynamic "2 3" will return an error.
for varname=$prefix_suffix format, just use:
varname=${prefix}_suffix

bash script with double for loop over multiple arrays with access to array names

I'm using rsync to copy specific files from a source directory (and subdirectories) to a destination directory (and subdirectories). The mapping of the subdirectories is not identical, so I'm defining arrays of subdirectories of the destination directory that contain the source file paths.
I've been unable to successfully loop over the destination arrays with access to the names of the arrays. Here's a MWE (filenames and directories must be edited, obviously).
#!/bin/bash
sourcedir=~/Dropbox/230/
destdir=~/Dropbox/230/website/
# Destination arrays
organization=(
syllabus/syllabus.pdf
)
notes=(
notes/ME230_2014S_Part0_Lec00.pdf
notes/ME230_2014S_Part1_Lec01_partial.pdf
)
echo "synchronizing: rsyncing from $sourcedir to $destdir"
for destsubdir in $organization $notes
do
for sourcesubdir in "${destsubdir[#]}"
do
echo "from $sourcedir$sourcesubdir to $destdir$destsubdir/"
rsync -avz "$sourcedir$sourcesubdir" "$destdir$destsubdir/"
done
done
This is obviously wrong because I don't want the destination to be the contents of $destsubdir but the name of the variable itself. I've been unable to successfully achieve this.
Note that a proper solution would allow any number of arrays to included without restructuring (e.g. no extra for-loops).
If multi-indexed arrays were possible in bash, this would be much easier, I think. Thanks for any help!
Solution
#jeanrjc has a good solution. Here's what it looks like in terms of my example above.
#!/bin/bash
sourcedir=~/Dropbox/230/
destdir=~/Dropbox/230/website/
# Destination arrays
organization=(
syllabus/syllabus.pdf
)
notes=(
notes/ME230_2014S_Part0_Lec00.pdf
notes/ME230_2014S_Part1_Lec01_partial.pdf
)
destsubdirArray=(
organization
notes
)
echo "synchronizing: rsyncing from $sourcedir to $destdir"
for destsubdir in ${destsubdirArray[*]}
do
destsubdirContents=$destsubdir[#];
for sourcesubdir in ${!destsubdirContents}
do
echo "from $sourcedir$sourcesubdir to $destdir$destsubdir/"
rsync -avz "$sourcedir$sourcesubdir" "$destdir$destsubdir/"
done
done
I'm not sure I get what you meant, but maybe this can help you :
one=( "one" 5 6 7 8 )
two=( "two" 11 22 33 )
three=( "tree" 777 888 999 )
all=( one two three )
for i in ${all[*]}; do
ref=$i[#];
for j in ${!ref}; do
echo $j
done
done
one
5
6
7
8
two
11
22
33
tree
777
888
999
Bash 4.3 supports namerefs -- the expansion of a nameref is the expansion of the nameref's contents rather than the nameref itself (it may help to think of namerefs as being similar to symlinks).
declare -n destsubdir
for destsubdir in organization notes ; do
for sourcesubdir in "${destsubdir[#]}" ; do
echo "$sourcedir$sourcesubdir $destdir$sourcesubdir"
# rsync -avz "$sourcedir$sourcesubdir" "$destdir$sourcesubdir"
done
done
unset -n destsubdir
For earlier bash versions the nearest equivalent is, as kisp states, to eval the entire inner loop. The quoting requirements can get out of hand really quickly.
for destsubdir in organization notes ; do
eval 'for sourcesubdir in "${'"$destsubdir"'[#]}" ; do
echo "$sourcedir$sourcesubdir" "$destdir$sourcesubdir"
# rsync -avz "$sourcedir$sourcesubdir" "$destdir$sourcesubdir"
done'
done
Note: those rsync commands have not been tested.
Solution to pass the variable names instead of the variables :
You can pass the variable names itself to the first for, then evaluate them in the second if needed.
like
for var_name in variable1 variable2 ; do
eval "myvar=\$$var_name"
Anyway, the single for loop can be used with the elements only if you use ${array[#]} everywhere.

Check whether files in a file list exist in a certain directory

The runtime arguments are as follows: $1 is the path to the file containing the list of files
$2 is the path to the directory containing the files
What I want to do is check that each file listed in $1 exists in the $2 directory
I'm thinking something like:
for f in 'cat $1'
do
if (FILEEXISTSIN$2DIRECTORY)
then echo '$f exists in $2'
else echo '$f is missing in $2' sleep 5 exit
fi
done
As you can see, I want it so that if any of the files listed in $1 don't exist in $2 directory, the script states this then closes. The only part I can't get my head around is the (FILEEXISTSIN$2DIRECTORY) part. I know that you can do [ -e $f ], but I don't know how you can make sure its checking that it exists in the $2 directory.
Edit: Thinking further upon this, perhaps I could use nested for loops?
If your specified input file contains a newline-separated list of files to check, then the following solution (using a while read loop) is robust enough to handle file names with spaces properly.
Generally, you should never make use of a loop of the form for i in $(command), and instead opt for a while loop. See http://mywiki.wooledge.org/DontReadLinesWithFor for more details.
while read -r file; do
if [[ -e "$2/$file" ]]; then
echo "$f exists in $2"
else
echo "$f does not exist in $2"
sleep 5
exit 1
fi
done < "$1"
Since you're dealing with a list of file names without spaces in the names (because the $(cat $1) notation will split things up like that), it is relatively straight forward:
for file in $(cat $1)
do
if [ -e "$2/$file" ]
then echo "$file exists in $2"
else echo "$file is missing in $2"; sleep 5; exit 1
fi
done
Basically, use the built-in string concatenation facilities to build the full path to the file, and use the test or [ operator to check the files existence.
The complexities arise if you have to deal with arbitrary file names, especially if one of the arbitrary characters in an arbitrary file name can be the newline character. Suffice to say, they complicate the issue sufficiently that I won't deal with it unless you say you need it dealt with, and even then, I'll negotiate on whether newlines in names need to be handled. The double-quoted variable expansion is a key part of the strategy for dealing with it. The other part of the problem is how to get the file names accurately into a variable.

save and restore shell variables

I have two shell scripts that I'd like to invoke from a C program. I would like shell variables set in the first script to be visible in the second. Here's what it would look like:
a.sh:
var=blah
<save vars>
b.sh:
<restore vars>
echo $var
The best I've come up with so far is a variant on "set > /tmp/vars" to save the variables and "eval $(cat /tmp/vars)" to restore them. The "eval" chokes when it tries to restore a read-only variable, so I need to grep those out. A list of these variables is available via "declare -r". But there are some vars which don't show up in this list, yet still can't be set in eval, e.g. BASH_ARGC. So I need to grep those out, too.
At this point, my solution feels very brittle and error-prone, and I'm not sure how portable it is. Is there a better way to do this?
One way to avoid setting problematic variables is by storing only those which have changed during the execution of each script. For example,
a.sh:
set > /tmp/pre
foo=bar
set > /tmp/post
grep -v -F -f/tmp/pre /tmp/post > /tmp/vars
b.sh:
eval $(cat /tmp/vars)
echo $foo
/tmp/vars contains this:
PIPESTATUS=([0]="0")
_=
foo=bar
Evidently evaling the first two lines has no adverse effect.
If you can use a common prefix on your variable names, here is one way to do it:
# save the variables
yourprefix_width=1200
yourprefix_height=2150
yourprefix_length=1975
yourprefix_material=gravel
yourprefix_customer_array=("Acme Plumbing" "123 Main" "Anytown")
declare -p $(echo ${!yourprefix#}) > varfile
# load the variables
while read -r line
do
if [[ $line == declare\ * ]]
then
eval "$line"
fi
done < varfile
Of course, your prefix will be shorter. You could do further validation upon loading the variables to make sure that the variable names conform to your naming scheme.
The advantage of using declare is that it is more secure than just using eval by itself.
If you need to, you can filter out variables that are marked as readonly or select variables that are marked for export.
Other commands of interest (some may vary by Bash version):
export - without arguments, lists all exported variables using a declare format
declare -px - same as the previous command
declare -pr - lists readonly variables
If it's possible for a.sh to call b.sh, it will carry over if they're exported. Or having a parent set all the values necessary and then call both. That's the most secure and sure method I can think of.
Not sure if it's accepted dogma, but:
bash -c 'export foo=bar; env > xxxx'
env `cat xxxx` otherscript.sh
The otherscript will have the env printed to xxxx ...
Update:
Also note:
man execle
On how to set environment variables for another system call from within C, if you need to do that. And:
man getenv
and http://www.crasseux.com/books/ctutorial/Environment-variables.html
An alternative to saving and restoring shell state would be to make the C program and the shell program work in parallel: the C program starts the shell program, which runs a.sh, then notifies the C program (perhaps passing some information it's learned from executing a.sh), and when the C program is ready for more it tells the shell program to run b.sh. The shell program would look like this:
. a.sh
echo "information gleaned from a"
arguments_for_b=$(read -r)
. b.sh
And the general structure of the C program would be:
set up two pairs of pipes, one for C->shell and one for shell->C
fork, exec the shell wrapper
read information gleaned from a on the shell->C pipe
more processing
write arguments for b on the C->shell pipe
wait for child process to end
I went looking for something similar and couldn't find it either, so I made the two scripts below. To start, just say shellstate, then probably at least set -i and set -o emacs which this reset_shellstate doesn't do for you. I don't know a way to ask bash which variables it thinks are special.
~/bin/reset_shellstate:
#!/bin/bash
__="$PWD/shellstate_${1#_}"
trap '
declare -p >"'"$__"'"
trap >>"'"$__"'"
echo cd \""$PWD"\" >>"'"$__"'" # setting PWD did this already, but...
echo set +abefhikmnptuvxBCEHPT >>"'"$__"'"
echo set -$- >>"'"$__"'" # must be last before sed, see $s/s//2 below
sed -ri '\''
$s/s//2
s,^trap --,trap,
/^declare -[^ ]*r/d
/^declare -[^ ]* [A-Za-z0-9_]*[^A-Za-z0-9_=]/d
/^declare -[^ ]* [^= ]*_SESSION_/d
/^declare -[^ ]* BASH[=_]/d
/^declare -[^ ]* (DISPLAY|GROUPS|SHLVL|XAUTHORITY)=/d
/^declare -[^ ]* WINDOW(ID|PATH)=/d
'\'' "'"$__"'"
shopt -op >>"'"$__"'"
shopt -p >>"'"$__"'"
declare -f >>"'"$__"'"
echo "Shell state saved in '"$__"'"
' 0
unset __
~/bin/shellstate:
#!/bin/bash
shellstate=shellstate_${1#_}
test -s $shellstate || reset_shellstate $1
shift
bash --noprofile --init-file shellstate_${1#_} -is "$#"
exit $?

Resources