How to set arrays with variables with loop in bash [duplicate] - arrays
I am confused about a bash script.
I have the following code:
function grep_search() {
magic_way_to_define_magic_variable_$1=`ls | tail -1`
echo $magic_variable_$1
}
I want to be able to create a variable name containing the first argument of the command and bearing the value of e.g. the last line of ls.
So to illustrate what I want:
$ ls | tail -1
stack-overflow.txt
$ grep_search() open_box
stack-overflow.txt
So, how should I define/declare $magic_way_to_define_magic_variable_$1 and how should I call it within the script?
I have tried eval, ${...}, \$${...}, but I am still confused.
I've been looking for better way of doing it recently. Associative array sounded like overkill for me. Look what I found:
suffix=bzz
declare prefix_$suffix=mystr
...and then...
varname=prefix_$suffix
echo ${!varname}
From the docs:
The ‘$’ character introduces parameter expansion, command substitution, or arithmetic expansion. ...
The basic form of parameter expansion is ${parameter}. The value of parameter is substituted. ...
If the first character of parameter is an exclamation point (!), and parameter is not a nameref, it introduces a level of indirection. Bash uses the value formed by expanding the rest of parameter as the new parameter; this is then expanded and that value is used in the rest of the expansion, rather than the expansion of the original parameter. This is known as indirect expansion. The value is subject to tilde expansion, parameter expansion, command substitution, and arithmetic expansion. ...
Use an associative array, with command names as keys.
# Requires bash 4, though
declare -A magic_variable=()
function grep_search() {
magic_variable[$1]=$( ls | tail -1 )
echo ${magic_variable[$1]}
}
If you can't use associative arrays (e.g., you must support bash 3), you can use declare to create dynamic variable names:
declare "magic_variable_$1=$(ls | tail -1)"
and use indirect parameter expansion to access the value.
var="magic_variable_$1"
echo "${!var}"
See BashFAQ: Indirection - Evaluating indirect/reference variables.
Beyond associative arrays, there are several ways of achieving dynamic variables in Bash. Note that all these techniques present risks, which are discussed at the end of this answer.
In the following examples I will assume that i=37 and that you want to alias the variable named var_37 whose initial value is lolilol.
Method 1. Using a “pointer” variable
You can simply store the name of the variable in an indirection variable, not unlike a C pointer. Bash then has a syntax for reading the aliased variable: ${!name} expands to the value of the variable whose name is the value of the variable name. You can think of it as a two-stage expansion: ${!name} expands to $var_37, which expands to lolilol.
name="var_$i"
echo "$name" # outputs “var_37”
echo "${!name}" # outputs “lolilol”
echo "${!name%lol}" # outputs “loli”
# etc.
Unfortunately, there is no counterpart syntax for modifying the aliased variable. Instead, you can achieve assignment with one of the following tricks.
1a. Assigning with eval
eval is evil, but is also the simplest and most portable way of achieving our goal. You have to carefully escape the right-hand side of the assignment, as it will be evaluated twice. An easy and systematic way of doing this is to evaluate the right-hand side beforehand (or to use printf %q).
And you should check manually that the left-hand side is a valid variable name, or a name with index (what if it was evil_code # ?). By contrast, all other methods below enforce it automatically.
# check that name is a valid variable name:
# note: this code does not support variable_name[index]
shopt -s globasciiranges
[[ "$name" == [a-zA-Z_]*([a-zA-Z_0-9]) ]] || exit
value='babibab'
eval "$name"='$value' # carefully escape the right-hand side!
echo "$var_37" # outputs “babibab”
Downsides:
does not check the validity of the variable name.
eval is evil.
eval is evil.
eval is evil.
1b. Assigning with read
The read builtin lets you assign values to a variable of which you give the name, a fact which can be exploited in conjunction with here-strings:
IFS= read -r -d '' "$name" <<< 'babibab'
echo "$var_37" # outputs “babibab\n”
The IFS part and the option -r make sure that the value is assigned as-is, while the option -d '' allows to assign multi-line values. Because of this last option, the command returns with an non-zero exit code.
Note that, since we are using a here-string, a newline character is appended to the value.
Downsides:
somewhat obscure;
returns with a non-zero exit code;
appends a newline to the value.
1c. Assigning with printf
Since Bash 3.1 (released 2005), the printf builtin can also assign its result to a variable whose name is given. By contrast with the previous solutions, it just works, no extra effort is needed to escape things, to prevent splitting and so on.
printf -v "$name" '%s' 'babibab'
echo "$var_37" # outputs “babibab”
Downsides:
Less portable (but, well).
Method 2. Using a “reference” variable
Since Bash 4.3 (released 2014), the declare builtin has an option -n for creating a variable which is a “name reference” to another variable, much like C++ references. Just as in Method 1, the reference stores the name of the aliased variable, but each time the reference is accessed (either for reading or assigning), Bash automatically resolves the indirection.
In addition, Bash has a special and very confusing syntax for getting the value of the reference itself, judge by yourself: ${!ref}.
declare -n ref="var_$i"
echo "${!ref}" # outputs “var_37”
echo "$ref" # outputs “lolilol”
ref='babibab'
echo "$var_37" # outputs “babibab”
This does not avoid the pitfalls explained below, but at least it makes the syntax straightforward.
Downsides:
Not portable.
Risks
All these aliasing techniques present several risks. The first one is executing arbitrary code each time you resolve the indirection (either for reading or for assigning). Indeed, instead of a scalar variable name, like var_37, you may as well alias an array subscript, like arr[42]. But Bash evaluates the contents of the square brackets each time it is needed, so aliasing arr[$(do_evil)] will have unexpected effects… As a consequence, only use these techniques when you control the provenance of the alias.
function guillemots {
declare -n var="$1"
var="«${var}»"
}
arr=( aaa bbb ccc )
guillemots 'arr[1]' # modifies the second cell of the array, as expected
guillemots 'arr[$(date>>date.out)1]' # writes twice into date.out
# (once when expanding var, once when assigning to it)
The second risk is creating a cyclic alias. As Bash variables are identified by their name and not by their scope, you may inadvertently create an alias to itself (while thinking it would alias a variable from an enclosing scope). This may happen in particular when using common variable names (like var). As a consequence, only use these techniques when you control the name of the aliased variable.
function guillemots {
# var is intended to be local to the function,
# aliasing a variable which comes from outside
declare -n var="$1"
var="«${var}»"
}
var='lolilol'
guillemots var # Bash warnings: “var: circular name reference”
echo "$var" # outputs anything!
Source:
BashFaq/006: How can I use variable variables (indirect variables, pointers, references) or associative arrays?
BashFAQ/048: eval command and security issues
Example below returns value of $name_of_var
var=name_of_var
echo $(eval echo "\$$var")
Use declare
There is no need on using prefixes like on other answers, neither arrays. Use just declare, double quotes, and parameter expansion.
I often use the following trick to parse argument lists contanining one to n arguments formatted as key=value otherkey=othervalue etc=etc, Like:
# brace expansion just to exemplify
for variable in {one=foo,two=bar,ninja=tip}
do
declare "${variable%=*}=${variable#*=}"
done
echo $one $two $ninja
# foo bar tip
But expanding the argv list like
for v in "$#"; do declare "${v%=*}=${v#*=}"; done
Extra tips
# parse argv's leading key=value parameters
for v in "$#"; do
case "$v" in ?*=?*) declare "${v%=*}=${v#*=}";; *) break;; esac
done
# consume argv's leading key=value parameters
while test $# -gt 0; do
case "$1" in ?*=?*) declare "${1%=*}=${1#*=}";; *) break;; esac
shift
done
Combining two highly rated answers here into a complete example that is hopefully useful and self-explanatory:
#!/bin/bash
intro="You know what,"
pet1="cat"
pet2="chicken"
pet3="cow"
pet4="dog"
pet5="pig"
# Setting and reading dynamic variables
for i in {1..5}; do
pet="pet$i"
declare "sentence$i=$intro I have a pet ${!pet} at home"
done
# Just reading dynamic variables
for i in {1..5}; do
sentence="sentence$i"
echo "${!sentence}"
done
echo
echo "Again, but reading regular variables:"
echo $sentence1
echo $sentence2
echo $sentence3
echo $sentence4
echo $sentence5
Output:
You know what, I have a pet cat at home
You know what, I have a pet chicken at home
You know what, I have a pet cow at home
You know what, I have a pet dog at home
You know what, I have a pet pig at home
Again, but reading regular variables:
You know what, I have a pet cat at home
You know what, I have a pet chicken at home
You know what, I have a pet cow at home
You know what, I have a pet dog at home
You know what, I have a pet pig at home
This will work too
my_country_code="green"
x="country"
eval z='$'my_"$x"_code
echo $z ## o/p: green
In your case
eval final_val='$'magic_way_to_define_magic_variable_"$1"
echo $final_val
This should work:
function grep_search() {
declare magic_variable_$1="$(ls | tail -1)"
echo "$(tmpvar=magic_variable_$1 && echo ${!tmpvar})"
}
grep_search var # calling grep_search with argument "var"
An extra method that doesn't rely on which shell/bash version you have is by using envsubst. For example:
newvar=$(echo '$magic_variable_'"${dynamic_part}" | envsubst)
For zsh (newers mac os versions), you should use
real_var="holaaaa"
aux_var="real_var"
echo ${(P)aux_var}
holaaaa
Instead of "!"
As per BashFAQ/006, you can use read with here string syntax for assigning indirect variables:
function grep_search() {
read "$1" <<<$(ls | tail -1);
}
Usage:
$ grep_search open_box
$ echo $open_box
stack-overflow.txt
Even though it's an old question, I still had some hard time with fetching dynamic variables names, while avoiding the eval (evil) command.
Solved it with declare -n which creates a reference to a dynamic value, this is especially useful in CI/CD processes, where the required secret names of the CI/CD service are not known until runtime. Here's how:
# Bash v4.3+
# -----------------------------------------------------------
# Secerts in CI/CD service, injected as environment variables
# AWS_ACCESS_KEY_ID_DEV, AWS_SECRET_ACCESS_KEY_DEV
# AWS_ACCESS_KEY_ID_STG, AWS_SECRET_ACCESS_KEY_STG
# -----------------------------------------------------------
# Environment variables injected by CI/CD service
# BRANCH_NAME="DEV"
# -----------------------------------------------------------
declare -n _AWS_ACCESS_KEY_ID_REF=AWS_ACCESS_KEY_ID_${BRANCH_NAME}
declare -n _AWS_SECRET_ACCESS_KEY_REF=AWS_SECRET_ACCESS_KEY_${BRANCH_NAME}
export AWS_ACCESS_KEY_ID=${_AWS_ACCESS_KEY_ID_REF}
export AWS_SECRET_ACCESS_KEY=${_AWS_SECRET_ACCESS_KEY_REF}
echo $AWS_ACCESS_KEY_ID $AWS_SECRET_ACCESS_KEY
aws s3 ls
Wow, most of the syntax is horrible! Here is one solution with some simpler syntax if you need to indirectly reference arrays:
#!/bin/bash
foo_1=(fff ddd) ;
foo_2=(ggg ccc) ;
for i in 1 2 ;
do
eval mine=( \${foo_$i[#]} ) ;
echo ${mine[#]}" " ;
done ;
For simpler use cases I recommend the syntax described in the Advanced Bash-Scripting Guide.
KISS approach:
a=1
c="bam"
let "$c$a"=4
echo $bam1
results in 4
I want to be able to create a variable name containing the first argument of the command
script.sh file:
#!/usr/bin/env bash
function grep_search() {
eval $1=$(ls | tail -1)
}
Test:
$ source script.sh
$ grep_search open_box
$ echo $open_box
script.sh
As per help eval:
Execute arguments as a shell command.
You may also use Bash ${!var} indirect expansion, as already mentioned, however it doesn't support retrieving of array indices.
For further read or examples, check BashFAQ/006 about Indirection.
We are not aware of any trick that can duplicate that functionality in POSIX or Bourne shells without eval, which can be difficult to do securely. So, consider this a use at your own risk hack.
However, you should re-consider using indirection as per the following notes.
Normally, in bash scripting, you won't need indirect references at all. Generally, people look at this for a solution when they don't understand or know about Bash Arrays or haven't fully considered other Bash features such as functions.
Putting variable names or any other bash syntax inside parameters is frequently done incorrectly and in inappropriate situations to solve problems that have better solutions. It violates the separation between code and data, and as such puts you on a slippery slope toward bugs and security issues. Indirection can make your code less transparent and harder to follow.
For indexed arrays, you can reference them like so:
foo=(a b c)
bar=(d e f)
for arr_var in 'foo' 'bar'; do
declare -a 'arr=("${'"$arr_var"'[#]}")'
# do something with $arr
echo "\$$arr_var contains:"
for char in "${arr[#]}"; do
echo "$char"
done
done
Associative arrays can be referenced similarly but need the -A switch on declare instead of -a.
POSIX compliant answer
For this solution you'll need to have r/w permissions to the /tmp folder.
We create a temporary file holding our variables and leverage the -a flag of the set built-in:
$ man set
...
-a Each variable or function that is created or modified is given the export attribute and marked for export to the environment of subsequent commands.
Therefore, if we create a file holding our dynamic variables, we can use set to bring them to life inside our script.
The implementation
#!/bin/sh
# Give the temp file a unique name so you don't mess with any other files in there
ENV_FILE="/tmp/$(date +%s)"
MY_KEY=foo
MY_VALUE=bar
echo "$MY_KEY=$MY_VALUE" >> "$ENV_FILE"
# Now that our env file is created and populated, we can use "set"
set -a; . "$ENV_FILE"; set +a
rm "$ENV_FILE"
echo "$foo"
# Output is "bar" (without quotes)
Explaining the steps above:
# Enables the -a behavior
set -a
# Sources the env file
. "$ENV_FILE"
# Disables the -a behavior
set +a
While I think declare -n is still the best way to do it there is another way nobody mentioned it, very useful in CI/CD
function dynamic(){
export a_$1="bla"
}
dynamic 2
echo $a_2
This function will not support spaces so dynamic "2 3" will return an error.
for varname=$prefix_suffix format, just use:
varname=${prefix}_suffix
Related
Why does "echo $array" print all members of the array in this specific case instead of only the first member like in any other case?
I have encountered a very curious problem, while trying to learn bash. Usually trying to print an echo by simply parsing the variable name like this only outputs the first member Hello. #!/bin/bash declare -a test test[0]="Hello" test[1]="World" echo $test # Only prints "Hello" BUT, for some reason this piece of code prints out ALL members of the given array. #!/bin/bash declare -a files counter=0 for file in "./*" do files[$counter]=$file let $((counter++)) done echo $files # prints "./file1 ./file2 ./file3" and so on And I can't seem to wrap my head around it on why it outputs the whole array instead of only the first member. I think it has something to do with my usage of the foreach-loop, but I was unable to find any concrete answer. It's driving me crazy! Please send help!
When you quoted the pattern, you only created a single entry in your array: $ declare -p files declare -a files=([0]="./*") If you had quoted the parameter expansion, you would see $ echo "$files" ./* Without the quotes, the expansion is subject to pathname generation, so echo receives multiple arguments, each of which is printed. To build the array you expected, drop the quotes around the pattern. The results of pathname generation are not subject to further word-splitting (or recursive pathname generation), so no quotes would be needed. for file in ./* do ... done
How to prepare variable with echo output in bash and then print it [duplicate]
This question already has answers here: Dynamic variable names in Bash (19 answers) Closed 6 years ago. I want to declare a variable, the name of which comes from the value of another variable, and I wrote the following piece of code: a="bbb" $a="ccc" but it didn't work. What's the right way to get this job done?
eval is used for this, but if you do it naively, there are going to be nasty escaping issues. This sort of thing is generally safe: name_of_variable=abc eval $name_of_variable="simpleword" # abc set to simpleword This breaks: eval $name_of_variable="word splitting occurs" The fix: eval $name_of_variable="\"word splitting occurs\"" # not anymore The ultimate fix: put the text you want to assign into a variable. Let's call it safevariable. Then you can do this: eval $name_of_variable=\$safevariable # note escaped dollar sign Escaping the dollar sign solves all escape issues. The dollar sign survives verbatim into the eval function, which will effectively perform this: eval 'abc=$safevariable' # dollar sign now comes to life inside eval! And of course this assignment is immune to everything. safevariable can contain *, spaces, $, etc. (The caveat being that we're assuming name_of_variable contains nothing but a valid variable name, and one we are free to use: not something special.)
You can use declare and !, like this: John="nice guy" programmer=John echo ${!programmer} # echos nice guy Second example: programmer=Ines declare $programmer="nice gal" echo $Ines # echos nice gal
This might work for you: foo=bar declare $foo=baz echo $bar baz or this: foo=bar read $foo <<<"baz" echo $bar baz
You could make use of eval for this. Example: $ a="bbb" $ eval $a="ccc" $ echo $bbb ccc Hope this helps!
If you want to get the value of the variable instead of setting it you can do this var_name1="var_name2" var_name2=value_you_want eval temp_var=\$$var_name1 echo "$temp_var" You can read about it here indirect references.
You can assign a value to a variable using simple assignment using a value from another variable like so: #!/usr/bin/bash #variable one a="one" echo "Variable a is $a" #variable two with a's variable b="$a" echo "Variable b is $b" #change a a="two" echo "Variable a is $a" echo "Variable b is $b" The output of that is this: Variable a is one Variable b is one Variable a is two Variable b is one So just be sure to assign it like this b="$a" and you should be good.
Create array in bash with variables as array name
I'm not sure if this has been answered, I've looked and haven't found anything that looks like what I'm trying to do. I also posted this to stackexchange (https://unix.stackexchange.com/questions/189293/create-array-in-bash-with-variables-as-array-name) I have a number of shell scripts that are capable of running against a ksh or bash shell, and they make use of arrays. I created a function named "setArray" that interrogates the running shell and determines what builtin to use to create the array - for ksh, set -A, for bash, typeset -a. However, I'm having some issues with the bash portion. The function takes two arguments, the name of the array and the value to add. This then becomes ${ARRAY_NAME} and ${VARIABLE_VALUE}. Doing the following: set -A $(eval echo \${ARRAY_NAME}) $(eval echo \${${ARRAY_NAME}[*]}) "${VARIABLE_VALUE}" works perfectly in ksh. However, typeset -a $(eval echo \${ARRAY_NAME})=( $(eval echo \${${ARRAY_NAME}[*]}) "${VARIABLE_VALUE}" ) does not. This provides bash: syntax error near unexpected token '(' I know I can just make it a list of strings (e.g. MYARRAY="one two three") and just loop through it using the IFS, but I don't want to lose the ability to use an array either. Any thoughts ?
Given the assertion that the ksh portion of this function is working only the bash portion needs to be created. For which the following should work and, I believe, be safe and robust (though evidence to the contrary is welcome). eval $ARRAY_NAME+=\(\"\$VARIABLE_VALUE\"\) First expansion only expands $ARRAY_NAME to get eval array+=("$VARIABLE_VALUE") which eval then causes to be evaluated again normally.
Bash - expanding variable nested in variable
Noble StackOverflow readers, I have a comma seperated file, each line of which I am putting into an array. Data looks as so... 25455410,GROU,AJAXa,GROU1435804437 25455410,AING,EXS3d,AING4746464646 25455413,TRAD,DLGl,TRAD7176202067 There are 103 lines and I am able to generate the 103 arrays without issue. n=1; while read -r OrdLine; do IFS=',' read -a OrdLineArr${n} <<< "$OrdLine" let n++ done < $WkOrdsFile HOWEVER, I can only access the arrays as so... echo "${OrdLineArr3[0]} <---Gives 25455413 I cannot access it with the number 1-103 as a variable - for example the following doesn't work... i=3 echo "${OrdLineArr${i}[0]} That results in... ./script2.sh: line 24: ${OrdLineArr${i}[0]}: bad substitution I think that the answer might involve 'eval' but I cannot seem to find a fitting example to borrow. If somebody can fix this then the above code makes for a very easy to handle 2d array replacement in bash! Thanks so much for you help in advance! Dan
You can use indirect expansion. For example, if $key is OrdLineArr4[7], then ${!key} (with an exclamation point) means ${OrdLineArr4[7]}. (See §3.5.3 "Shell Parameter Expansion" in the Bash Reference Manual, though admittedly that passage doesn't really explain how indirect expansion interacts with arrays.) I'd recommend wrapping this in a function: function OrdLineArr () { local -i i="$1" # line number (1-103) local -i j="$2" # field number (0-3) local key="OrdLineArr$i[$j]" echo "${!key}" } Then you can write: echo "$(OrdLineArr 3 0)" # prints 25455413 i=3 echo "$(OrdLineArr $i 0)" # prints 25455413 This obviously isn't a total replacement for two-dimensional arrays, but it will accomplish what you need. Without using eval.
eval is usually a bad idea, but you can do it with: eval echo "\${OrdLineArr$i[0]}"
I would store each line in an array, but split it on demand: readarray OrdLineArr < $WkOrdsFile ... OrdLine=${OrdLineArr[i]} IFS=, read -a Ord <<< "$OrdLine" However, bash isn't really equipped for data processing; it's designed to facilitate process and file management. You should consider using a different language.
Exporting an array in bash script
I can not export an array from a bash script to another bash script like this: export myArray[0]="Hello" export myArray[1]="World" When I write like this there are no problem: export myArray=("Hello" "World") For several reasons I need to initialize my array into multiple lines. Do you have any solution?
Array variables may not (yet) be exported. From the manpage of bash version 4.1.5 under ubuntu 10.04. The following statement from Chet Ramey (current bash maintainer as of 2011) is probably the most official documentation about this "bug": There isn't really a good way to encode an array variable into the environment. http://www.mail-archive.com/bug-bash#gnu.org/msg01774.html
TL;DR: exportable arrays are not directly supported up to and including bash-5.1, but you can (effectively) export arrays in one of two ways: a simple modification to the way the child scripts are invoked use an exported function to store the array initialisation, with a simple modification to the child scripts Or, you can wait until bash-4.3 is released (in development/RC state as of February 2014, see ARRAY_EXPORT in the Changelog). Update: This feature is not enabled in 4.3. If you define ARRAY_EXPORT when building, the build will fail. The author has stated it is not planned to complete this feature. The first thing to understand is that the bash environment (more properly command execution environment) is different to the POSIX concept of an environment. The POSIX environment is a collection of un-typed name=value pairs, and can be passed from a process to its children in various ways (effectively a limited form of IPC). The bash execution environment is effectively a superset of this, with typed variables, read-only and exportable flags, arrays, functions and more. This partly explains why the output of set (bash builtin) and env or printenv differ. When you invoke another bash shell you're starting a new process, you loose some bash state. However, if you dot-source a script, the script is run in the same environment; or if you run a subshell via ( ) the environment is also preserved (because bash forks, preserving its complete state, rather than reinitialising using the process environment). The limitation referenced in #lesmana's answer arises because the POSIX environment is simply name=value pairs with no extra meaning, so there's no agreed way to encode or format typed variables, see below for an interesting bash quirk regarding functions , and an upcoming change in bash-4.3(proposed array feature abandoned). There are a couple of simple ways to do this using declare -p (built-in) to output some of the bash environment as a set of one or more declare statements which can be used reconstruct the type and value of a "name". This is basic serialisation, but with rather less of the complexity some of the other answers imply. declare -p preserves array indexes, sparse arrays and quoting of troublesome values. For simple serialisation of an array you could just dump the values line by line, and use read -a myarray to restore it (works with contiguous 0-indexed arrays, since read -a automatically assigns indexes). These methods do not require any modification of the script(s) you are passing the arrays to. declare -p array1 array2 > .bash_arrays # serialise to an intermediate file bash -c ". .bash_arrays; . otherscript.sh" # source both in the same environment Variations on the above bash -c "..." form are sometimes (mis-)used in crontabs to set variables. Alternatives include: declare -p array1 array2 > .bash_arrays # serialise to an intermediate file BASH_ENV=.bash_arrays otherscript.sh # non-interactive startup script Or, as a one-liner: BASH_ENV=<(declare -p array1 array2) otherscript.sh The last one uses process substitution to pass the output of the declare command as an rc script. (This method only works in bash-4.0 or later: earlier versions unconditionally fstat() rc files and use the size returned to read() the file in one go; a FIFO returns a size of 0, and so won't work as hoped.) In a non-interactive shell (i.e. shell script) the file pointed to by the BASH_ENV variable is automatically sourced. You must make sure bash is correctly invoked, possibly using a shebang to invoke "bash" explicitly, and not #!/bin/sh as bash will not honour BASH_ENV when in historical/POSIX mode. If all your array names happen to have a common prefix you can use declare -p ${!myprefix*} to expand a list of them, instead of enumerating them. You probably should not attempt to export and re-import the entire bash environment using this method, some special bash variables and arrays are read-only, and there can be other side-effects when modifying special variables. (You could also do something slightly disagreeable by serialising the array definition to an exportable variable, and using eval, but let's not encourage the use of eval ... $ array=([1]=a [10]="b c") $ export scalar_array=$(declare -p array) $ bash # start a new shell $ eval $scalar_array $ declare -p array declare -a array='([1]="a" [10]="b c")' ) As referenced above, there's an interesting quirk: special support for exporting functions through the environment: function myfoo() { echo foo } with export -f or set +a to enable this behaviour, will result in this in the (process) environment, visible with printenv: myfoo=() { echo foo } The variable is functionname (or functioname() for backward compatibility) and its value is () { functionbody }. When a subsequent bash process starts it will recreate a function from each such environment variable. If you peek into the bash-4.2 source file variables.c you'll see variables starting with () { are handled specially. (Though creating a function using this syntax with declare -f is forbidden.) Update: The "shellshock" security issue is related to this feature, contemporary systems may disable automatic function import from the environment as a mitigation. If you keep reading though, you'll see an #if 0 (or #if ARRAY_EXPORT) guarding code that checks variables starting with ([ and ending with ), and a comment stating "Array variables may not yet be exported". The good news is that in the current development version bash-4.3rc2 the ability to export indexed arrays (not associative) is enabled. This feature is not likely to be enabled, as noted above. We can use this to create a function which restores any array data required: % function sharearray() { array1=(a b c d) } % export -f sharearray % bash -c 'sharearray; echo ${array1[*]}' So, similar to the previous approach, invoke the child script with: bash -c "sharearray; . otherscript.sh" Or, you can conditionally invoke the sharearray function in the child script by adding at some appropriate point: declare -F sharearray >/dev/null && sharearray Note there is no declare -a in the sharearray function, if you do that the array is implicitly local to the function, not what is wanted. bash-4.2 supports declare -g that makes a variable declared in a function into a global, so declare -ga can then be used. (Since associative arrays require a declare -A you won't be able to use this method for global associative arrays prior to bash-4.2, from v4.2 declare -Ag will work as hoped.) The GNU parallel documentation has useful variation on this method, see the discussion of --env in the man page. Your question as phrased also indicates you may be having problems with export itself. You can export a name after you've created or modified it. "exportable" is a flag or property of a variable, for convenience you can also set and export in a single statement. Up to bash-4.2 export expects only a name, either a simple (scalar) variable or function name are supported. Even if you could (in future) export arrays, exporting selected indexes (a slice) may not be supported (though since arrays are sparse there's no reason it could not be allowed). Though bash also supports the syntax declare -a name[0], the subscript is ignored, and "name" is simply a normal indexed array.
Jeez. I don't know why the other answers made this so complicated. Bash has nearly built-in support for this. In the exporting script: myArray=( ' foo"bar ' $'\n''\nbaz)' ) # an array with two nasty elements myArray="${myArray[#]#Q}" ./importing_script.sh (Note, the double quotes are necessary for correct handling of whitespace within array elements.) Upon entry to importing_script.sh, the value of the myArray environment variable comprises these exact 26 bytes: ' foo"bar ' $'\n\\nbaz)' Then the following will reconstitute the array: eval "myArray=( ${myArray} )" CAUTION! Do not eval like this if you cannot trust the source of the myArray environment variable. This trick exhibits the "Little Bobby Tables" vulnerability. Imagine if someone were to set the value of myArray to ) ; rm -rf / #.
The environment is just a collection of key-value pairs, both of which are character strings. A proper solution that works for any kind of array could either Save each element in a different variable (e.g. MY_ARRAY_0=myArray[0]). Gets complicated because of the dynamic variable names. Save the array in the file system (declare -p myArray >file). Serialize all array elements into a single string. These are covered in the other posts. If you know that your values never contain a certain character (for example |) and your keys are consecutive integers, you can simply save the array as a delimited list: export MY_ARRAY=$(IFS='|'; echo "${myArray[*]}") And restore it in the child process: IFS='|'; myArray=($MY_ARRAY); unset IFS
Based on #mr.spuratic use of BASH_ENV, here I tunnel $# through script -f -c script -c <command> <logfile> can be used to run a command inside another pty (and process group) but it cannot pass any structured arguments to <command>. Instead <command> is a simple string to be an argument to the system library call. I need to tunnel $# of the outer bash into $# of the bash invoked by script. As declare -p cannot take #, here I use the magic bash variable _ (with a dummy first array value as that will get overwritten by bash). This saves me trampling on any important variables: Proof of concept: BASH_ENV=<( declare -a _=("" "$#") && declare -p _ ) bash -c 'set -- "${_[#]:1}" && echo "$#"' "But," you say, "you are passing arguments to bash -- and indeed I am, but these are a simple string of known character. Here is use by script SHELL=/bin/bash BASH_ENV=<( declare -a _=("" "$#") && declare -p _ && echo 'set -- "${_[#]:1}"') script -f -c 'echo "$#"' /tmp/logfile which gives me this wrapper function in_pty: in_pty() { SHELL=/bin/bash BASH_ENV=<( declare -a _=("" "$#") && declare -p _ && echo 'set -- "${_[#]:1}"') script -f -c 'echo "$#"' /tmp/logfile } or this function-less wrapper as a composable string for Makefiles: in_pty=bash -c 'SHELL=/bin/bash BASH_ENV=<( declare -a _=("" "$$#") && declare -p _ && echo '"'"'set -- "$${_[#]:1}"'"'"') script -qfc '"'"'"$$#"'"'"' /tmp/logfile' -- ... $(in_pty) test --verbose $# $^
I was editing a different post and made a mistake. Augh. Anyway, perhaps this might help? https://stackoverflow.com/a/11944320/1594168 Note that because the shell's array format is undocumented on bash or any other shell's side, it is very difficult to return a shell array in platform independent way. You would have to check the version, and also craft a simple script that concatinates all shell arrays into a file that other processes can resolve into. However, if you know the name of the array you want to take back home then there is a way, while a bit dirty. Lets say I have MyAry[42]="whatever-stuff"; MyAry[55]="foo"; MyAry[99]="bar"; So I want to take it home name_of_child=MyAry take_me_home="`declare -p ${name_of_child}`"; export take_me_home="${take_me_home/#declare -a ${name_of_child}=/}" We can see it being exported, by checking from a sub-process echo ""|awk '{print "from awk =["ENVIRON["take_me_home"]"]"; }' Result : from awk =['([42]="whatever-stuff" [55]="foo" [99]="bar")'] If we absolutely must, use the env var to dump it. env > some_tmp_file Then Before running the another script, # This is the magic that does it all source some_tmp_file
As lesmana reported, you cannot export arrays. So you have to serialize them before passing through the environment. This serialization useful other places too where only a string fits (su -c 'string', ssh host 'string'). The shortest code way to do this is to abuse 'getopt' # preserve_array(arguments). return in _RET a string that can be expanded # later to recreate positional arguments. They can be restored with: # eval set -- "$_RET" preserve_array() { _RET=$(getopt --shell sh --options "" -- -- "$#") && _RET=${_RET# --} } # restore_array(name, payload) restore_array() { local name="$1" payload="$2" eval set -- "$payload" eval "unset $name && $name=("\$#")" } Use it like this: foo=("1: &&& - *" "2: two" "3: %# abc" ) preserve_array "${foo[#]}" foo_stuffed=${_RET} restore_array newfoo "$foo_stuffed" for elem in "${newfoo[#]}"; do echo "$elem"; done ## output: # 1: &&& - * # 2: two # 3: %# abc This does not address unset/sparse arrays. You might be able to reduce the 2 'eval' calls in restore_array.
Although this question/answers are pretty old, this post seems to be the top hit when searching for "bash serialize array" And, although the original question wasn't quite related to serializing/deserializing arrays, it does seem that the answers have devolved in that direction. So with that ... I offer my solution: Pros All Core Bash Concepts No Evals No Sub-Commands Cons Functions take variable names as arguments (vs actual values) Serializing requires having at least one character that is not present in the array serialize_array.bash # shellcheck shell=bash ## # serialize_array # Serializes a bash array to a string, with a configurable seperator. # # $1 = source varname ( contains array to be serialized ) # $2 = target varname ( will contian the serialized string ) # $3 = seperator ( optional, defaults to $'\x01' ) # # example: # # my_arry=( one "two three" four ) # serialize_array my_array my_string '|' # declare -p my_string # # result: # # declare -- my_string="one|two three|four" # function serialize_array() { declare -n _array="${1}" _str="${2}" # _array, _str => local reference vars local IFS="${3:-$'\x01'}" # shellcheck disable=SC2034 # Reference vars assumed used by caller _str="${_array[*]}" # * => join on IFS } ## # deserialize_array # Deserializes a string into a bash array, with a configurable seperator. # # $1 = source varname ( contains string to be deserialized ) # $2 = target varname ( will contain the deserialized array ) # $3 = seperator ( optional, defaults to $'\x01' ) # # example: # # my_string="one|two three|four" # deserialize_array my_string my_array '|' # declare -p my_array # # result: # # declare -a my_array=([0]="one" [1]="two three" [2]="four") # function deserialize_array() { IFS="${3:-$'\x01'}" read -r -a "${2}" <<<"${!1}" # -a => split on IFS } NOTE: This is hosted as a gist here: https://gist.github.com/TekWizely/c0259f25e18f2368c4a577495cd566cd [edits] Logic simplified after running through shellcheck + shfmt. Added URL for hosted GIST
you (hi!) can use this, dont need writing a file, for ubuntu 12.04, bash 4.2.24 Also, your multiple lines array can be exported. cat >>exportArray.sh function FUNCarrayRestore() { local l_arrayName=$1 local l_exportedArrayName=${l_arrayName}_exportedArray # if set, recover its value to array if eval '[[ -n ${'$l_exportedArrayName'+dummy} ]]'; then eval $l_arrayName'='`eval 'echo $'$l_exportedArrayName` #do not put export here! fi } export -f FUNCarrayRestore function FUNCarrayFakeExport() { local l_arrayName=$1 local l_exportedArrayName=${l_arrayName}_exportedArray # prepare to be shown with export -p eval 'export '$l_arrayName # collect exportable array in string mode local l_export=`export -p \ |grep "^declare -ax $l_arrayName=" \ |sed 's"^declare -ax '$l_arrayName'"export '$l_exportedArrayName'"'` # creates exportable non array variable (at child shell) eval "$l_export" } export -f FUNCarrayFakeExport test this example on terminal bash (works with bash 4.2.24): source exportArray.sh list=(a b c) FUNCarrayFakeExport list bash echo ${list[#]} #empty :( FUNCarrayRestore list echo ${list[#]} #profit! :D I may improve it here PS.: if someone clears/improve/makeItRunFaster I would like to know/see, thx! :D
For arrays with values without spaces, I've been using a simple set of functions to iterate through each array element and concatenate the array: _arrayToStr(){ array=($#) arrayString="" for (( i=0; i<${#array[#]}; i++ )); do if [[ $i == 0 ]]; then arrayString="\"${array[i]}\"" else arrayString="${arrayString} \"${array[i]}\"" fi done export arrayString="(${arrayString})" } _strToArray(){ str=$1 array=${str//\"/} array=(${array//[()]/""}) export array=${array[#]} } The first function with turn the array into a string by adding the opening and closing parentheses and escaping all of the double quotation marks. The second function will strip the quotation marks and the parentheses and place them into a dummy array. In order export the array, you would pass in all the elements of the original array: array=(foo bar) _arrayToStr ${array[#]} At this point, the array has been exported into the value $arrayString. To import the array in the destination file, rename the array and do the opposite conversion: _strToArray "$arrayName" newArray=(${array[#]})
Much thanks to #stéphane-chazelas who pointed out all the problems with my previous attempts, this now seems to work to serialise an array to stdout or into a variable. This technique does not shell-parse the input (unlike declare -a/declare -p) and so is safe against malicious insertion of metacharacters in the serialised text. Note: newlines are not escaped, because read deletes the \<newlines> character pair, so -d ... must instead be passed to read, and then unescaped newlines are preserved. All this is managed in the unserialise function. Two magic characters are used, the field separator and the record separator (so that multiple arrays can be serialized to the same stream). These characters can be defined as FS and RS but neither can be defined as newline character because an escaped newline is deleted by read. The escape character must be \ the backslash, as that is what is used by read to avoid the character being recognized as an IFS character. serialise will serialise "$#" to stdout, serialise_to will serialise to the varable named in $1 serialise() { set -- "${#//\\/\\\\}" # \ set -- "${#//${FS:-;}/\\${FS:-;}}" # ; - our field separator set -- "${#//${RS:-:}/\\${RS:-:}}" # ; - our record separator local IFS="${FS:-;}" printf ${SERIALIZE_TARGET:+-v"$SERIALIZE_TARGET"} "%s" "$*${RS:-:}" } serialise_to() { SERIALIZE_TARGET="$1" serialise "${#:2}" } unserialise() { local IFS="${FS:-;}" if test -n "$2" then read -d "${RS:-:}" -a "$1" <<<"${*:2}" else read -d "${RS:-:}" -a "$1" fi } and unserialise with: unserialise data # read from stdin or unserialise data "$serialised_data" # from args e.g. $ serialise "Now is the time" "For all good men" "To drink \$drink" "At the \`party\`" $'Party\tParty\tParty' Now is the time;For all good men;To drink $drink;At the `party`;Party Party Party: (without a trailing newline) read it back: $ serialise_to s "Now is the time" "For all good men" "To drink \$drink" "At the \`party\`" $'Party\tParty\tParty' $ unserialise array "$s" $ echo "${array[#]/#/$'\n'}" Now is the time For all good men To drink $drink At the `party` Party Party Party or unserialise array # read from stdin Bash's read respects the escape character \ (unless you pass the -r flag) to remove special meaning of characters such as for input field separation or line delimiting. If you want to serialise an array instead of a mere argument list then just pass your array as the argument list: serialise_array "${my_array[#]}" You can use unserialise in a loop like you would read because it is just a wrapped read - but remember that the stream is not newline separated: while unserialise array do ... done
I've wrote my own functions for this and improved the method with the IFS: Features: Doesn't call to $(...) and so doesn't spawn another bash shell process Serializes ? and | characters into ?00 and ?01 sequences and back, so can be used over array with these characters Handles the line return characters between serialization/deserialization as other characters Tested in cygwin bash 3.2.48 and Linux bash 4.3.48 function tkl_declare_global() { eval "$1=\"\$2\"" # right argument does NOT evaluate } function tkl_declare_global_array() { local IFS=$' \t\r\n' # just in case, workaround for the bug in the "[#]:i" expression under the bash version lower than 4.1 eval "$1=(\"\${#:2}\")" } function tkl_serialize_array() { local __array_var="$1" local __out_var="$2" [[ -z "$__array_var" ]] && return 1 [[ -z "$__out_var" ]] && return 2 local __array_var_size eval declare "__array_var_size=\${#$__array_var[#]}" (( ! __array_var_size )) && { tkl_declare_global $__out_var ''; return 0; } local __escaped_array_str='' local __index local __value for (( __index=0; __index < __array_var_size; __index++ )); do eval declare "__value=\"\${$__array_var[__index]}\"" __value="${__value//\?/?00}" __value="${__value//|/?01}" __escaped_array_str="$__escaped_array_str${__escaped_array_str:+|}$__value" done tkl_declare_global $__out_var "$__escaped_array_str" return 0 } function tkl_deserialize_array() { local __serialized_array="$1" local __out_var="$2" [[ -z "$__out_var" ]] && return 1 (( ! ${#__serialized_array} )) && { tkl_declare_global $__out_var ''; return 0; } local IFS='|' local __deserialized_array=($__serialized_array) tkl_declare_global_array $__out_var local __index=0 local __value for __value in "${__deserialized_array[#]}"; do __value="${__value//\?01/|}" __value="${__value//\?00/?}" tkl_declare_global $__out_var[__index] "$__value" (( __index++ )) done return 0 } Example: a=($'1 \n 2' "3\"4'" 5 '|' '?') tkl_serialize_array a b tkl_deserialize_array "$b" c
I think you can try it this way (by sourcing your script after export): export myArray=(Hello World) . yourScript.sh