In TCSH :
I'm giving two lists containing different files in it. Then I have to pass those list as an array element and then the loop should execute twice as there are only 2 lists. But in this case loop is executing as much time as those files in both the lists.
set list_one = (one.s two.s three.s)
set list_two = (four.s five.s)
set arr=($list_one $list_two)
foreach i ($arr)
cat $i > $output.s
end
This is an example of my code, according to me loop should execute only twice(for list_one and list_two), but it's executing five times (for one.s two.s three.s four.s five.s)
If I use like this (mentioned below), loop executes for two times. But at cat command I get an error as, No such file or directory for each list that I passed in array.
set arr=(list_one list_two)
foreach i ($arr)
cat $i > $output.s
end
You were on the right track with your second approach, only a level of variable substitution was missing.
set arr=(list_one list_two)
foreach i ($arr)
eval cat \$$i >$output.s
end
Related
I have multiple arrays (~32). I want to remove all blank elements from them. How can it be done in a short way (may be via one foreach loop or 1-2 command lines)?
I tried the below, but it's not working:
my #refreshArrayList=("#list1", "#list2", "#list3","#list4", "#list5", "#list6" , "#list7");
foreach my $i (#refreshArrayList) {
$i = grep (!/^\s*$/, $i);
}
Let's say, #list1 = ("abc","def","","ghi"); #list2 = ("qwe","","rty","uy", "iop"), and similarly for other arrays. Now, I want to remove all blank elements from all the arrays.
Desired Output shall be: #list1 = ("abc","def","ghi"); #list2 = ("qwe","rty","uy", "iop") ### All blank elements are removed from all the arrays.
How can it be done?
You can create a list of list references and then iterator over these, like
for my $list (\#list1, \#list2, \#list3) {
#$list = grep (!/^\s*$/, #$list);
}
Of course, you could create this list of list references also dynamically, i.e.
my #list_of_lists;
push #list_of_lists, \#list1;
push #list_of_lists, \#list2;
...
for my $list (#list_of_lists) {
#$list = grep (!/^\s*$/, #$list);
}
#$_ = grep /\S/, #$_ for #AoA; # #AoA = (\#ary1, \#ary2, ...)
Explanation
First, this uses the statement modifier, "inverting" the usual for loop syntax into the form STMT for LIST
The for(each) modifier is an iterator: it executes the statement once for each item in the LIST (with $_ aliased to each item in turn).
It is mostly equivalent to a "normal" for loop, with the notable difference being that no scope is set and so there is no need to tear it down either, adding a small measure of efficiency.† Ae can have only one statement; but then again, that can be a do block. Having no scope means that we cannot declare lexical variables for the statement (unless a do block is used).
So the statement is #$_ = grep /\S/, #$_, executed for each element of the list.
In a for(each) loop, the variable that is set to each element in turn as the list is iterated over ("topicalizer") is an alias to those elements. So changing it changes elements. From perlsyn
If VAR is omitted, $_ is set to each value.
If any element of LIST is an lvalue, you can modify it by modifying VAR inside the loop.
In our case $_ is always an array reference, and then the underlying array is rewritten by dereferencing it (#$_) and assigning to that the output list of grep, which consists only of elements that have at least one non-space character (/\S/).
† I ran a three-way benchmark, of the statement-modifier loop against a "normal" loop with and without a topical variable.
For adding 100e6 numbers I get 8-11% speedup (on both a desktop and a server) and with a more involved calculation ($r = ($r + $_ ) / sqrt($_)) it's 4-5%.
A side observation: In both cases the full for loop without a variable (using default $_ for the topicalizer) is 1-2% faster than the one with a lexical topical variable set.
I can get this to work in ksh but not in bash which is really driving me nuts.
Hopefully it is something obvious that I'm overlooking.
I need to run an external command where each line of the output will be stored at an array index.
This simplified example looks like it is setting the array in the loop correctly however after the loop has completed those array assignments are gone? It's as though the loop is treated completely as an external shell?
junk.txt
this is a
test to see
if this works ok
testa.sh
#!/bin/bash
declare -i i=0
declare -a array
echo "Simple Test:"
array[0]="hello"
echo "array[0] = ${array[0]}"
echo -e "\nLoop through junk.txt:"
cat junk.txt | while read line
do
array[i]="$line"
echo "array[$i] = ${array[i]}"
let i++
done
echo -e "\nResults:"
echo " array[0] = ${array[0]}"
echo " Total in array = ${#array[*]}"
echo "The whole array:"
echo ${array[#]}
Output
Simple Test:
array[0] = hello
Loop through junk.txt:
array[0] = this is a
array[1] = test to see
array[2] = if this works ok
Results:
array[0] = hello
Total in array = 1
The whole array:
hello
So while in the loop, we assign array[i] and the echo verifies it.
But after the loop I'm back at array[0] containing "hello" with no other elements.
Same results across bash 3, 4 and different platforms.
Because your while loop is in a pipeline, all variable assignments in the loop body are local to the subshell in which the loop is executed. (I believe ksh does not run the command in a subshell, which is why you have the problem in bash.) Do this instead:
while read line
do
array[i]="$line"
echo "array[$i] = ${array[i]}"
let i++
done < junk.txt
Rarely, if ever, do you want to use cat to pipe a single file to another command; use input redirection instead.
UPDATE: since you need to run from a command and not a file, another option (if available) is process substitution:
while read line; do
...
done < <( command args ... )
If process substitution is not available, you'll need to output to a temporary file and redirect input from that file.
If you are using bash 4.2 or later, you can execute these two commands before your loop, and the original pipe-into-the-loop will work, since the while loop is the last command in the pipeline.
set +m # Turn off job control; it's probably already off in a non-interactive script
shopt -s lastpipe
cat junk.txt | while read line; do ...; done
UPDATE 2: Here is a loop-less solution based on user1596414's comment
array[0]=hello
IFS=$'\n' array+=( $(command) )
The output of your command is split into words based solely on newlines (so that each line is a separate word), and appends the resulting line-per-slot array to the original. This is very nice if you are only using the loop to build the array. It can also probably be modified to accomodate a small amount of per-line processing, vaguely similar to a Python list comprehension.
In shell scripting, I have a loop with an if
condition inside a for loop.
for i in val1 val2
do
if ((condition)) then
printf ...
fi
done
This loop returns me correct output which is a list of numbers. However, I want to store these numbers in
an array to use inside another loop. How do I do
this?
I want to know how to store the values returned by printf statement in an array.
Here's the solution:
#!/bin/bash
data=() #declare an array outside the scope of loop
idx=0 #initialize a counter to zero
for i in {53..99} #some random number range
do
data[idx]=`printf "number=%s\n" $i` #store data in array
idx=$((idx+1)) #increment the counter
done
echo ${data[*]} #your result
What code does
creates and empty array
creates an index-counter for array
stores result of output printf command in array at corresponding index (the backquote tells interpreter to do that)
I can get this to work in ksh but not in bash which is really driving me nuts.
Hopefully it is something obvious that I'm overlooking.
I need to run an external command where each line of the output will be stored at an array index.
This simplified example looks like it is setting the array in the loop correctly however after the loop has completed those array assignments are gone? It's as though the loop is treated completely as an external shell?
junk.txt
this is a
test to see
if this works ok
testa.sh
#!/bin/bash
declare -i i=0
declare -a array
echo "Simple Test:"
array[0]="hello"
echo "array[0] = ${array[0]}"
echo -e "\nLoop through junk.txt:"
cat junk.txt | while read line
do
array[i]="$line"
echo "array[$i] = ${array[i]}"
let i++
done
echo -e "\nResults:"
echo " array[0] = ${array[0]}"
echo " Total in array = ${#array[*]}"
echo "The whole array:"
echo ${array[#]}
Output
Simple Test:
array[0] = hello
Loop through junk.txt:
array[0] = this is a
array[1] = test to see
array[2] = if this works ok
Results:
array[0] = hello
Total in array = 1
The whole array:
hello
So while in the loop, we assign array[i] and the echo verifies it.
But after the loop I'm back at array[0] containing "hello" with no other elements.
Same results across bash 3, 4 and different platforms.
Because your while loop is in a pipeline, all variable assignments in the loop body are local to the subshell in which the loop is executed. (I believe ksh does not run the command in a subshell, which is why you have the problem in bash.) Do this instead:
while read line
do
array[i]="$line"
echo "array[$i] = ${array[i]}"
let i++
done < junk.txt
Rarely, if ever, do you want to use cat to pipe a single file to another command; use input redirection instead.
UPDATE: since you need to run from a command and not a file, another option (if available) is process substitution:
while read line; do
...
done < <( command args ... )
If process substitution is not available, you'll need to output to a temporary file and redirect input from that file.
If you are using bash 4.2 or later, you can execute these two commands before your loop, and the original pipe-into-the-loop will work, since the while loop is the last command in the pipeline.
set +m # Turn off job control; it's probably already off in a non-interactive script
shopt -s lastpipe
cat junk.txt | while read line; do ...; done
UPDATE 2: Here is a loop-less solution based on user1596414's comment
array[0]=hello
IFS=$'\n' array+=( $(command) )
The output of your command is split into words based solely on newlines (so that each line is a separate word), and appends the resulting line-per-slot array to the original. This is very nice if you are only using the loop to build the array. It can also probably be modified to accomodate a small amount of per-line processing, vaguely similar to a Python list comprehension.
I am trying to write code to break up a large array into many different small arrays. Eventually the array I would be passed is one of unknown size, this is just my test subject. I have gotten this far:
#!/bin/bash
num=(10 3 12 3 4 4)
inArray=${#num[#]}
numArrays=$(($inArray/2))
remain=$(($inArray%2))
echo $numArrays
echo $remain
nun=0
if test $remain -gt $nun; then
numArrays=$(($numArrays+1))
fi
array=(1 2)
j=0
for ((i=0;i<$numArrays;i++, j=j+2)); do
array=("${num[#]:$j:2}")
echo "The array says: ${array[#]}"
echo "The size? ${#array[#]}"
done
What I am really having a problem with is : I would like to make the variable 'array' be able to change names slightly every time, so each array is kept and has a unique name after the loop. I have tried making the name array_$i but that returns:
[Stephanie#~]$ ./tmp.sh
3
0
./tmp.sh: line 16: syntax error near unexpected token `"${num[#]:$j:2}"'
./tmp.sh: line 16: ` array_$i=("${num[#]:$j:2}")'
[Stephanie#RDT00069 ~]$ ./tmp.sh
3
0
./tmp.sh: line 16: syntax error near unexpected token `$i'
./tmp.sh: line 16: ` array($i)=("${num[#]:$j:2}")'
Does anyone have any advice?
Thanks
I don't think you can really avoid eval here, but you might be able to do it safely if you're careful. Here's my approach:
for name in "${!array_*}"; do # Get all names starting with array_
i="${name#array_*}" # Get the part after array_
if [[ $i != *[^0-9]* ]]; then # Check that it's a number.
printf '%s is not a valid subarray name\n' "$name"
else
# Create a variable named "statement" that contains code you want to eval.
printf -v statement 'cur_array=( "${%s[#]}" )' "$name"
eval "$statement"
# Do interesting things with $cur_array
fi
done
Before this, when you're just creating the array, you know what $name should be, so just use the printf -v part.
To make it even safer, you could save all the allowed array names in another array and check that $name is a member.
With simple variables, you can use the declare keyword to make indirect assignments:
v=foo
declare $v=5
echo $foo # Prints 5
This doesn't extend to arrays in the obvious (to me, anyway) sense:
i=2
# This produces a syntax error
declare -a array_$i=("${num[#]:$j:2}")
Instead, you can declare an empty array
declare -a array_$i
or assign items one at a time:
declare -a array_$i[0]=item1 array_$i[1]=item2
Here's an example of using a for-loop to copy, say, the 3rd
and 4th letters of a big array into a smaller one. We use
i as the dynamic part of the name of the smaller array, and
j as the index into that array.
letters=(a b c d e f)
i=1
j=0
for letter in "${letters[#]:2:2}"; do
# E.g., i=0 and j=1 would result in
# declare -a array_0[1]=c
declare -a array_$i[$j]=$letter
let j+=1
done
done
echo ${array_1[#]}; # c d
${foo[#]:x:y} gives us elements x, x+1, ..., x+y-1 from foo, and
You can wrap the whole thing inside another for-loop to accomplish the goal of splitting letters into 3 smaller arrays:
# We'll create array_0, array_1, and array_2
for i in 0 1 2; do
# Just like our subset above, but start at position i*2 instead of
# a constant.
for letter in "${letters[#]:$((i*2)):2}"; do
declare -a array_$i[$j]=$letter
done
done
Once you manage to populate your three arrays, how do you access them without eval? Bash has syntax for indirect access:
v=foo
foo=5
echo ${!v} # echoes 5!
The exclamation point says to use the word that follows as a variable whose value should be used as the name of the parameter to expand. Knowing that, you might think you could do the following, but you'd be wrong.
i=1
v=array_$i # array_1
echo ${!v[0]} # array_1[0] is c, so prints c, right? Wrong.
In the above, bash tries to find a variable called v[0] and expand it to get the name of a parameter to expand. We actually have to treat our array plus its index as a single name:
i=1
v=array_$i[0]
echo ${!v} # This does print c
This should work, but this is not a good solution, another language may be better bash does not support multi dimensional arrays
eval array_$i='('"${num[#]:$j:2}"')'
And then, for example
eval 'echo "${array_'$i'[0]}"'