CURRENTFILENAMES=( "$(ls $LOC -AFl | sed "1 d" | grep "[^/]$" | awk '{ print $9 }')" )
I have written the above code, however it is not behaving as I expect it to in a for-loop, which I wrote as so
for a in "$CURRENTFILENAMES"; do
CURRENTFILEPATHS=( "${LOC}/${a}" )
done
Which I expected to prepend the value in the variable LOC to all of the items in the CURRENTFILENAMES array, however it has just prepended it to the beginning of the array, how can I remedy this?
You need to use += operator for appending into an array:
CURRENTFILEPATHS+=( "${LOC}/${a}" )
However parsing ls output is not advisable, use find instead.
EDIT: Proper way to run this loop:
CURRENTFILEPATHS=()
while IFS= read -d '' -r f; do
CURRENTFILEPATHS+=( "$f" )
done < <(find "$LOC" -maxdepth 1 -type f -print0)
Related
Why isn't this bash array populating? I believe I've done them like this in the past. Echoing ${#XECOMMAND[#]} shows no data..
DIR=$1
TEMPFILE=/tmp/dir.tmp
ls -l $DIR | tail -n +2 | sed 's/\s\+/ /g' | cut -d" " -f5,9 > $TEMPFILE
i=0
cat $TEMPFILE | while read line ;do
if [[ $(echo $line | cut -d" " -f1) == 0 ]]; then
XECOMMAND[$i]="$(echo "$line" | cut -d" " -f2)"
(( i++ ))
fi
done
When you run the while loop like
somecommand | while read ...
then the while loop is executed in sub-shell, i.e. a different process than the main script. Thus, all variable assignments that happen in the loop, will not be reflected in the main process. The workaround is to use input redirection and/or command substitution, so that the loop executes in the current process. For example if you want to read from a file you do
while read ....
do
# do stuff
done < "$filename"
or if you wan't the output of a process you can do
while read ....
do
# do stuff
done < <(some command)
Finally, in bash 4.2 and above, you can set shopt -s lastpipe, which causes the last command in the pipeline to be executed in the current process.
I think you're trying to construct an array consisting of the names of all zero-length files and directories in $DIR. If so, you can do it like this:
mapfile -t ZERO_LENGTH < <(find "$DIR" -maxdepth 1 -size 0)
(Add -type f to the find command if you're only interested in regular files.)
This sort of solution is almost always better than trying to parse ls output.
The use of process substitution (< <(...)) rather than piping (... |) is important, because it means that the shell variable will be set in the current shell, not in an ephimeral subshell.
Can anyone explain why the following doesn't work please?
list
the letter is d
the number is 4
the number is 2
the letter is g
script.sh
#!/bin/bash
cat "$1" | grep letter | array=($(awk '{print $4}'))
for i in "${array[#]}"
do
:
echo $i
done
If I run this bash script.sh list I expect the array to print d and g, but it doesn't. I think its because of how I am trying to set the array.
I think its because of how I am trying to set the array.
Each command in a pipeline | is run in a subshell - as a separate process. The parent process does not "see" variable changes from a child process.
Just:
array=($(grep letter "$1" | awk '{print $4}'))
or
array=($(awk '/letter/{print $4}' "$1"))
Run variable assignment in the parent shell.
You should assign the complete row of piped commands to a variable.
array=($(cat "$1" | grep letter | awk '{print $4}'))
The cat and grep command can be combined with awk, but why do you want an array?
I think you want the process each element in one loop, so first remove the double quotes:
for i in ${array[#]}
do
:
echo $i
done
Next, try to do this without an array
while read -r i; do
:
echo $i
done < <(awk '/letter/ {print $4}' "$1")
I am trying to replace all the word "null" to elements in array. The problem is that after replacing one word of "null", I would like to replace the next "null" with next element in the array.
I am not very good with bash and I feel like this is quite a basic question.
Here is what I have so far:
for m in $(cat finalfile.csv)
do
if [ "$m" = "null" ]
then
m=cwearray[$counter]
let counter++
fi
done
This doesn't replace anything in the finalfile.csv.
For example if the file has:
"value1","value2","null","value3"\n
"value1","value2","null","value3"...
and the array has ["foo","bar"]
I would like it to be:
"value1","value2","foo","value3"\n
"value1","value2","bar","value3"...
can be done with bash, even with multiple nulls per line:
$ cat finalfile.csv
"value1","value2","null","null"
"value1","value2","null","value3"
$ cwearray=( foo bar baz )
$ idx=0
$ while read -r line; do
while [[ $line == *null* ]]; do
line=${line/null/${cwearray[idx++]}}
# ...............^^^^^^^^^^^^^^^^^^
# replace the _first_ "null" with the _next_ array element
done
echo "$line"
done < finalfile.csv > updatedfinalfile.csv
$ cat updatedfinalfile.csv
"value1","value2","foo","bar"
"value1","value2","baz","value3"
It's easier in Perl where you can increase the index directly in the replacement part of a substitution:
printf '%s\n' 1,2,3,null null,2,3,4 null,null,null,null \
| perl -pe 'BEGIN { #cwe = qw( A B C D E F ) }
s/(?:^|(?<=,))null(?=,|$)/$cwe[$i++]/g'
Update: It seems you've updated your question with a sample input. If nulls are double quoted, it gets even easier, as there's no need to check whether they're surrounded with commas or beginning/end of the line.
perl -pe 'BEGIN{ #cwe = qw( foo bar ) }
s/"null"/"$cwe[$i++]"/g'
An awk solution :
declare -a cwearray
cwearray=(foo bar)
awk -F, 'NR==FNR{repl[NR]=$0; next}{for(i=1;i<=NF;i++){if($i=="\"null\""){$i="\""repl[++counter]"\""}}}1' OFS="," <(for i in "${cwearray[#]}"; do echo "$i"; done) <file>
Read the file line by line. If a line contains null, then use sed to replace all occurrences of null with the corresponding value, retrieved via array index.
#!/bin/bash
file="finalfile.csv"
counter=0
array=(
"foo"
"bar"
)
while read -r line; do
item="${array[$counter]}"
echo "$line" | sed "s/null/$item/g"
((counter++))
done < "$file"
I have a log file I am trying to comment out lines that do not match my array. I did successfully learn how to create an array and I can echo out the array items but I am having trouble taking anything that doesn't match my array and adding something in front of it. Here is my code, if you have suggestions on another path or ways I can make it better:
for itsSaturday in $(find "$LOCATION" -mindepth 1 -maxdepth 1 -name "*.log" ); do
TEMPFILE="$itsSaturday.$$"
declare -a someArray=( "breakfast" "scrambled eggs" "Bloody Mary" )
theCall='some_additional_text_'
commentOn="## You_need_"
for arrayItem in "${someArray[#]}"; do
merged="$theCall$arrayItem"
if ! grep -q "$merged" "$itsSaturday"; then
sed -e '/$merged/! s:$commentOn$theCall::g' "$itsSaturday" > $TEMPFILE && mv $TEMPFILE "$itsSaturday"
fi
done
done
file:
some_additional_text_breakfast
some_additional_text_bacon
some_additional_text_scrambled eggs
some_additional_text_Bloody Mary
some_additional_text_orange juice
some_additional_text_breakfast
file into:
some_additional_text_breakfast
## You_need_some_additional_text_bacon
some_additional_text_scrambled eggs
some_additional_text_Bloody Mary
## You_need_some_additional_text_orange juice
some_additional_text_breakfast
How can I add a variable before items that do not match my array?
I don't like doing this using bash and sed, but I think the following might be enough:
#! /bin/bash
declare -a someArray=( "breakfast" "scrambled eggs" "Bloody Mary" )
theCall='some_additional_text_'
commentOn="## You_need_"
OIFS="$IFS"
IFS='|' mergedLines="${someArray[*]/#/$theCall}"
IFS="$OIFS"
for i in *.txt
do
TEMPFILE="$i.$$"
sed -r "/$mergedLines/!s/^/$commentOn/" "$i" >> "$TEMPFILE"
done
I shifted the array and other constants out of the loop.
"${someArray[*]/#/$theCall}" uses bash string substitution to append the contents of $theCall to every element in the array.
IFS='|' mergedLines="${someArray[*]} is a convenient trick to combine the elements of an array into a pipe-separated string.
Combined, (2) and (3) get me
some_additional_text_breakfast|some_additional_text_scrambled eggs|some_additional_text_Bloody Mary
in mergedLines.
Then it's just a matter of using extended regular expressions in sed (for |) and replacing non-matching lines.
Your sed pattern used single quotes, so the variables within were not expanded.
Try replacing the inner for-loop with:
PROG=$(printf '%s\n' "${COMMENT[#]}" | while read comment ; do
/bin/echo -n '$0 !~ /'"$comment"'$/ && '
done
echo '1 { printf commentOn } ; { print }')
awk -v commentOn="$commentOn" "$PROG" $itsSaturday > $TEMPFILE && mv $TEMPFILE $itsSaturday
On each file, this creates an awk program that does the work.
Problem:
Need to sort array before operating on them with function.
First, array is loaded with files:
unset a i
counter=1
while IFS= read -r -d $'\0' file; do
a[i++]="$file"
done < <(find $DIR -type f -print0)
Next, each member of array is sent to function
for f in "${a[#]}"
do
func_hash "$f"
[ $(expr $counter % 20) -eq 0 ] && printf "="
counter=$((counter + 1))
done
Somehow a sort needs to be thrown into the above for loop. Have looked
through the SO posts on sorting arrays but somehow my crazy file names
cause issues when I try to tack on a sort.
Ideas?
Thanks!
Bubnoff
UPDATE: Here's code with sort:
while IFS= read -r -d $'\0' file; do
func_hash "$file"
[ $(expr $counter % 20) -eq 0 ] && printf "="
counter=$((counter + 1))
done < <(find $DIR -type f -print0 | sort -z +1 -1)
It's sorting by full path rather than file name. Any ideas on how to
sort by file name given that the path is needed for the function?
UPDATE 2: Decided to compromise.
My main goal was to avoid temp files using sort. GNU sort can write back to the original
file with its '-o' option so now I can:
sort -o $OUT -t',' -k 1 $OUT
Anyone have a more 'elegant' solution ( whatever that means ).
SOLVED See jw013's answer below. Thanks man!
EDIT
while IFS= read -r -d/ && read -r -d '' file; do
a[i++]="$file"
done < <(find "$DIR" -type f -printf '%f/%p\0' | sort -z -t/ -k1 )
Rationale:
I make the assumption that / is never a legal character within a file name (which seems reasonable on most *nix filesystems since it is the path separator).
The -printf is used to print the file name without leading directories, then then full file name with path, separated by /. The sort takes place on the first field separated by / which should be the full file name without path.
The read is modified to first use / as a delimiter to throw out the pathless file name.
side note
Any POSIX shell should support the modulo operator as part of its arithmetic expansion. You can replace line with the call to external command expr in the second loop with
[ $(( counter % 20 )) -eq 0 ] ...