I'm trying to create an alias that will get all "Modified" files and run the php syntax check on them...
function gitphpcheck () {
filearray=()
git diff --name-status | while read line; do
if [[ $line =~ ^M ]]
then
filename="`echo $line | awk '{ print $2 }'`"
echo "$filename" # correct output
filearray+=($filename)
fi
done
echo "--------------FILES"
echo ${filearray[#]}
# will do php check here, but echo of array is blank
}
As Wrikken says, the while body runs in a subshell, so all changes to the filearray array will disappear when the subshell ends. A couple of different solutions come to mind:
Process substitution (less readable but does not require a subshell)
while read line; do
:
done < <(git diff --name-status)
echo "${filearray[#]}"
Use the modified variable in the subshell using command grouping
git diff --name-status | {
while read line; do
:
done
echo "${filearray[#]}"
}
# filearray is empty here
You've piped | things to while, which is essentially another process, so the filearray variable is a different one (not the same scope).
Related
Bash allows to use: cat <(echo "$FILECONTENT")
Bash also allow to use: while read i; do echo $i; done </etc/passwd
to combine previous two this can be used: echo $FILECONTENT | while read i; do echo $i; done
The problem with last one is that it creates sub-shell and after the while loop ends variable i cannot be accessed any more.
My question is:
How to achieve something like this: while read i; do echo $i; done <(echo "$FILECONTENT") or in other words: How can I be sure that i survives while loop?
Please note that I am aware of enclosing while statement into {} but this does not solves the problem (imagine that you want use the while loop in function and return i variable)
The correct notation for Process Substitution is:
while read i; do echo $i; done < <(echo "$FILECONTENT")
The last value of i assigned in the loop is then available when the loop terminates.
An alternative is:
echo $FILECONTENT |
{
while read i; do echo $i; done
...do other things using $i here...
}
The braces are an I/O grouping operation and do not themselves create a subshell. In this context, they are part of a pipeline and are therefore run as a subshell, but it is because of the |, not the { ... }. You mention this in the question. AFAIK, you can do a return from within these inside a function.
Bash also provides the shopt builtin and one of its many options is:
lastpipe
If set, and job control is not active, the shell runs the last command of a pipeline not executed in the background in the current shell environment.
Thus, using something like this in a script makes the modfied sum available after the loop:
FILECONTENT="12 Name
13 Number
14 Information"
shopt -s lastpipe # Comment this out to see the alternative behaviour
sum=0
echo "$FILECONTENT" |
while read number name; do ((sum+=$number)); done
echo $sum
Doing this at the command line usually runs foul of 'job control is not active' (that is, at the command line, job control is active). Testing this without using a script failed.
Also, as noted by Gareth Rees in his answer, you can sometimes use a here string:
while read i; do echo $i; done <<< "$FILECONTENT"
This doesn't require shopt; you may be able to save a process using it.
Jonathan Leffler explains how to do what you want using process substitution, but another possibility is to use a here string:
while read i; do echo "$i"; done <<<"$FILECONTENT"
This saves a process.
This function makes duplicates $NUM times of jpg files (bash)
function makeDups() {
NUM=$1
echo "Making $1 duplicates for $(ls -1 *.jpg|wc -l) files"
ls -1 *.jpg|sort|while read f
do
COUNT=0
while [ "$COUNT" -le "$NUM" ]
do
cp $f ${f//sm/${COUNT}sm}
((COUNT++))
done
done
}
This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 3 years ago.
I have read some entries regarding this question like in here or here, but I haven't been able to make my code work. It should be very simple.
I need to append items from a while loop, after a small transformation, to an empty list. My code is as follows:
folder='/path/to/directories/'
ls $folder | while read dir ; do
if [[ $dir =~ ANF-* ]]; then
names=()
ls $folder/$dir/FASTQS | while read file ; do
name=$(echo $file | cut -d "_" -f 1-3 )
echo $name
names+=("$name")
done
echo ${names[*]} #Also tried echo ${names[#]}
fi
done
The first 'echo' works so it gets through the conditional and into the second loop.
I have also tried using 'declare -a' to create the empty array.
If I try to append $file to the list it does not work either.
I think the problem is the appending action because if I create an array that is not empty I got the array's items in the second 'echo'.
Thanks a lot in advance.
RX
Try to wrap your echo with double quotes:
folder='/path/to/directories/'
ls $folder | while read dir ; do
if [[ $dir =~ ANF-* ]]; then
names=()
local iteration=1
ls $folder/$dir/FASTQS | while read file ; do
name=$(echo $file | cut -d "_" -f 1-3 )
echo $name
names+=("$name")
echo "iteration $iteration"
iteration=$((iteration+1))
declare -p names
done
echo "${names[#]}"
# you can show the array like this:
declare -p names
fi
done
Your names array is being altered, but when the loop terminates, the alterations go away. Look at the answers #Benjamin W. kindly pointed.
I'm trying to read the information of a structured file into an associative array using Bash script. The file contains in each line the name of a person and its address, separated by a "|". For example:
person1|address of person1
person2|address of person2
...
personN|address of personN
I tried to do this using the script below. Within the WHILE loop, the information is being printed. However, in the FOR loop the information is not being printed. It seems that the information is not being stored in the associative array outside of the WHILE loop.
What am I doing wrong? Why this is not working? Is there more efficient ways to do that?
#!/bin/bash
declare -A address
cat adresses.txt | while read line
do
name=`echo $line | cut -d '|' -f 1`
add=`echo $line | cut -d '|' -f 2`
address[$name]=$add
echo "$name - ${address[$name]}"
done
for name in ${!address[*]}
do
echo "$name - ${address[$name]}"
done
Wrong and useless usage of cut
#!/bin/bash
declare -A address
while IFS=\| read name add
do
address[$name]=$add
done < adresses.txt
for name in ${!address[*]}
do
echo "$name - ${address[$name]}"
done
cat addresses.txt | while read line
do
...
done
Shell commands in a pipelines are executed in subshells. Variables set in subshells aren't visible the parent shell.
You can fix this by replacing the pipelines with a redirection.
while read line
do
...
done < addresses.txt
Extending the accepted answer to resolve the OP's comment:
#!/bin/bash
declare -A address
while IFS='|' read name add
do
address[$name]=$add
echo "$name - ${address[$name]}"
done < adresses.txt
for name in "${!address[#]}"
do
echo "$name - ${address[$name]}"
done
I am writing a bash function to get all git repositories, but I have met a problem when I want to store all the git repository pathnames to the array patharray. Here is the code:
gitrepo() {
local opt
declare -a patharray
locate -b '\.git' | \
while read pathname
do
pathname="$(dirname ${pathname})"
if [[ "${pathname}" != *.* ]]; then
# Note: how to add an element to an existing Bash Array
patharray=("${patharray[#]}" '\n' "${pathname}")
# echo -e ${patharray[#]}
fi
done
echo -e ${patharray[#]}
}
I want to save all the repository paths to the patharray array, but I can't get it outside the pipeline which is comprised of locate and while command.
But I can get the array in the pipeline command, the commented command # echo -e ${patharray[#]} works well if uncommented, so how can I solve the problem?
And I have tried the export command, however it seems that it can't pass the patharray to the pipeline.
Bash runs all commands of a pipeline in separate SubShells. When a subshell containing a while loop ends, all changes you made to the patharray variable are lost.
You can simply group the while loop and the echo statement together so they are both contained within the same subshell:
gitrepo() {
local pathname dir
local -a patharray
locate -b '\.git' | { # the grouping begins here
while read pathname; do
pathname=$(dirname "$pathname")
if [[ "$pathname" != *.* ]]; then
patharray+=( "$pathname" ) # add the element to the array
fi
done
printf "%s\n" "${patharray[#]}" # all those quotes are needed
} # the grouping ends here
}
Alternately, you can structure your code to not need a pipe: use ProcessSubstitution
( Also see the Bash manual for details - man bash | less +/Process\ Substitution):
gitrepo() {
local pathname dir
local -a patharray
while read pathname; do
pathname=$(dirname "$pathname")
if [[ "$pathname" != *.* ]]; then
patharray+=( "$pathname" ) # add the element to the array
fi
done < <(locate -b '\.git')
printf "%s\n" "${patharray[#]}" # all those quotes are needed
}
First of all, appending to an array variable is better done with array[${#array[*]}]="value" or array+=("value1" "value2" "etc") unless you wish to transform the entire array (which you don't).
Now, since pipeline commands are run in subprocesses, changes made to a variable inside a pipeline command will not propagate to outside it. There are a few options to get around this (most are listed in Greg's BashFAQ/024):
pass the result through stdout instead
the simplest; you'll need to do that anyway to get the value from the function (although there are ways to return a proper variable)
any special characters in paths can be handled reliably by using \0 as a separator (see Capturing output of find . -print0 into a bash array for reading \0-separated lists)
locate -b0 '\.git' | while read -r -d '' pathname; do dirname -z "$pathname"; done
or simply
locate -b0 '\.git' | xargs -0 dirname -z
avoid running the loop in a subprocess
avoid pipeline at all
temporary file/FIFO (bad: requires manual cleanup, accessible to others)
temporary variable (mediocre: unnecessary memory overhead)
process substitution (a special, syntax-supported case of FIFO, doesn't require manual cleanup; code adapted from Greg's BashFAQ/020):
i=0 #`unset i` will error on `i' usage if the `nounset` option is set
while IFS= read -r -d $'\0' file; do
patharray[i++]="$(dirname "$file")" # or however you want to process each file
done < <(locate -b0 '\.git')
use the lastpipe option (new in Bash 4.2) - doesn't run the last command of a pipeline in a subprocess (mediocre: has global effect)
This question already has an answer here:
How to use global arrays in bash?
(1 answer)
Closed 6 years ago.
Here is the code which I need to separate the files in array, but using the PIPE it is generating subshell so am not able to get access to arrays normal, executable and directory.and its not printing anything or don't know what is happening after #////////.Please help me regarding this.
i=0
j=0
k=0
normal[0]=
executable[0]=
directory[0]=
ls | while read line
do
if [ -f $line ];then
#echo "this is normal file>> $line"
normal[i]=$line
i=$((i+1))
fi
if [ -x $line ];then
#echo "this is executable file>> $line"
executable[j]=$line
j=$((j+1))
fi
if [ -d $line ];then
#echo "this is directory>> $line"
directory[k]=$line
k=$((k+1))
fi
done
#//////////////////////////////////////
echo "normal files are"
for k in "${normal[#]}"
do
echo "$k"
done
echo "executable files are"
for k in "${executable[#]}"
do
echo "$k"
done
echo "directories are"
for k in "${directory[#]}"
do
echo "$k"
done
There are several flaws to your script :
Your if tests should be written with [[, not [, which is for binary comparison (more info : here). If you want to keep [ or are not using bash, you will have to quote your line variable, i.e. write all your tests like this : if [ -f "$line" ];then
Don't use ls to list the current directory as it misbehaves in some cases. A glob would be more suited in your case (more info: here)
If you want to avoid using a pipe, use a for loop instead. Replace ls | while read line with for line in $(ls) or, to take my previous point in acount, for line in *
After doing that, I tested your script and it worked perfectly fine. You should note that some folders will be listed under both under "executable files" and "directories", due to them having +x rights (I don't know if this is the behaviour you wanted).
As a side note, you don't need to declare variables in bash before using them. Your first 6 lines are thus un-necessary. Variables i,j,k are not necessary as well as you can dynamicaly increment an array with the following syntax : normal+=("$line").
The simplest thing to do is to keep the subshell open until you no longer need the arrays. In other words:
ls | { while read line; do
...
echo "directories: ${directory[#]}" | tr ' ' \\n
}
In other words, add an open brace before the while and a closing brace at the end of the script.