Why isn't this bash array populating? I believe I've done them like this in the past. Echoing ${#XECOMMAND[#]} shows no data..
DIR=$1
TEMPFILE=/tmp/dir.tmp
ls -l $DIR | tail -n +2 | sed 's/\s\+/ /g' | cut -d" " -f5,9 > $TEMPFILE
i=0
cat $TEMPFILE | while read line ;do
if [[ $(echo $line | cut -d" " -f1) == 0 ]]; then
XECOMMAND[$i]="$(echo "$line" | cut -d" " -f2)"
(( i++ ))
fi
done
When you run the while loop like
somecommand | while read ...
then the while loop is executed in sub-shell, i.e. a different process than the main script. Thus, all variable assignments that happen in the loop, will not be reflected in the main process. The workaround is to use input redirection and/or command substitution, so that the loop executes in the current process. For example if you want to read from a file you do
while read ....
do
# do stuff
done < "$filename"
or if you wan't the output of a process you can do
while read ....
do
# do stuff
done < <(some command)
Finally, in bash 4.2 and above, you can set shopt -s lastpipe, which causes the last command in the pipeline to be executed in the current process.
I think you're trying to construct an array consisting of the names of all zero-length files and directories in $DIR. If so, you can do it like this:
mapfile -t ZERO_LENGTH < <(find "$DIR" -maxdepth 1 -size 0)
(Add -type f to the find command if you're only interested in regular files.)
This sort of solution is almost always better than trying to parse ls output.
The use of process substitution (< <(...)) rather than piping (... |) is important, because it means that the shell variable will be set in the current shell, not in an ephimeral subshell.
Related
Can anyone explain why the following doesn't work please?
list
the letter is d
the number is 4
the number is 2
the letter is g
script.sh
#!/bin/bash
cat "$1" | grep letter | array=($(awk '{print $4}'))
for i in "${array[#]}"
do
:
echo $i
done
If I run this bash script.sh list I expect the array to print d and g, but it doesn't. I think its because of how I am trying to set the array.
I think its because of how I am trying to set the array.
Each command in a pipeline | is run in a subshell - as a separate process. The parent process does not "see" variable changes from a child process.
Just:
array=($(grep letter "$1" | awk '{print $4}'))
or
array=($(awk '/letter/{print $4}' "$1"))
Run variable assignment in the parent shell.
You should assign the complete row of piped commands to a variable.
array=($(cat "$1" | grep letter | awk '{print $4}'))
The cat and grep command can be combined with awk, but why do you want an array?
I think you want the process each element in one loop, so first remove the double quotes:
for i in ${array[#]}
do
:
echo $i
done
Next, try to do this without an array
while read -r i; do
:
echo $i
done < <(awk '/letter/ {print $4}' "$1")
I am very new to Unix shell script and trying to get some knowledge in shell scripting. Please check my requirement and my approach.
I have a input file having data
ABC = A:3 E:3 PS:6
PQR = B:5 S:5 AS:2 N:2
I am trying to parse the data and get the result as
ABC
A=3
E=3
PS=6
PQR
B=5
S=5
AS=2
N=2
The values can be added horizontally and vertically so I am trying to use an array. I am trying something like this:
myarr=(main.conf | awk -F"=" 'NR!=1 {print $1}'))
echo ${myarr[1]}
# Or loop through every element in the array
for i in "${myarr[#]}"
do
:
echo $i
done
or
awk -F"=" 'NR!=1 {
print $1"\n"
STR=$2
IFS=':' read -r -a array <<< "$STR"
for i in "${!array[#]}"
do
echo "$i=>${array[i]}"
done
}' main.conf
But when I add this code to a .sh file and try to run it, I get syntax errors as
$ awk -F"=" 'NR!=1 {
> print $1"\n"
> STR=$2
> FS= read -r -a array <<< "$STR"
> for i in "${!array[#]}"
> do
> echo "$i=>${array[i]}"
> done
>
> }' main.conf
awk: cmd. line:4: FS= read -r -a array <<< "$STR"
awk: cmd. line:4: ^ syntax error
awk: cmd. line:5: for i in "${!array[#]}"
awk: cmd. line:5: ^ syntax error
awk: cmd. line:8: done
awk: cmd. line:8: ^ syntax error
How can I complete the above expectations?
This is the awk code to do what you want:
$ cat tst.awk
BEGIN { FS="[ =:]+"; OFS="=" }
{
print $1
for (i=2;i<NF;i+=2) {
print $i, $(i+1)
}
print ""
}
and this is the shell script (yes, all a shell script does to manipulate text is call awk):
$ awk -f tst.awk file
ABC
A=3
E=3
PS=6
PQR
B=5
S=5
AS=2
N=2
A UNIX shell is an environment from which to call UNIX tools (find, sort, sed, grep, awk, tr, cut, etc.). It has its own language for manipulating (e.g. creating/destroying) files and processes and sequencing calls to tools but it is NOT intended to be used to manipulate text. The guys who invented shell also invented awk for shell to call to manipulate text.
Read https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice and the book Effective Awk Programming, 4th Edition, by Arnold Robbins.
First off, a command that does what you want:
$ sed 's/ = /\n/;y/: /=\n/' main.conf
ABC
A=3
E=3
PS=6
PQR
B=5
S=5
AS=2
N=2
This replaces, on each line, the first (and only) occurrence of = with a newline (the s command), then turns all : into = and all spaces into newlines (the y command). Notice that
this works only because there is a space at the end of the first line (otherwise it would be a bit more involved to get the empty line between the blocks) and
this works only with GNU sed because it substitutes newlines; see this fantastic answer for all the details and how to get it to work with BSD sed.
As for what you tried, there is almost too much wrong with it to try and fix it piece by piece: from the wild mixing of awk and Bash to syntax errors all over the place. I recommend you read good tutorials for both, for example:
The BashGuide
Effective AWK Programming
A Bash solution
Here is a way to solve the same in Bash; I didn't use any arrays.
#!/bin/bash
# Read line by line into the 'line' variable. Setting 'IFS' to the empty string
# preserves leading and trailing whitespace; '-r' prevents interpretation of
# backslash escapes
while IFS= read -r line; do
# Three parameter expansions:
# Replace ' = ' by newline (escape backslash)
line="${line/ = /\\n}"
# Replace ':' by '='
line="${line//:/=}"
# Replace spaces by newlines (escape backslash)
line="${line// /\\n}"
# Print the modified input line; '%b' expands backslash escapes
printf "%b" "$line"
done < "$1"
Output:
$ ./SO.sh main.conf
ABC
A=3
E=3
PS=6
PQR
B=5
S=5
AS=2
N=2
BACKGROUND
What I'm trying to do here is read from a file that has PID information and separate the columns out into arrays. I have completed this part already, though I think is a better way than what I currently have (catting the same file 4 times).
pid=( $(cat /tmp/deadpidlist.log | awk -F " " '{print $1}') )
cpu=( $(cat /tmp/deadpidlist.log | awk -F " " '{print $2}') )
mem=( $(cat /tmp/deadpidlist.log | awk -F " " '{print $3}') )
ctime=( $(cat /tmp/deadpidlist.log | awk -F " " '{print $4}') )
WHAT AM I DOING? oh god
After that, I need to loop through each PID and if the pid fits the criteria I am looking for, put the corresponding cpu usage, memory usage and cpu time beside it in a file and then email that file out.
for i in "${pid[#]}"
do
...
if grep -Fxq "$pattern" /tmp/or_report.log; then
echo "$i" >> /tmp/deadpidwalking.log
I have the rest of my code up on a gist here: https://gist.github.com/sithtoast/e1654adab3cceb137ba2
Thanks!
A simple loop in bash should be sufficient. Note the rarely seen use of subscripted arrays as arguments to read.
declare -a pid cpu mem ctime
i=-1
while ((i++)); read "pid[i]" "cpu[i]" "mem[i]" "ctime[i]" and_the_rest; do
:
done < /tmp/deadpidlist.log
A more straightforward loop would be
declare -a pid cpu mem ctime
while read a b c d and_the_rest; do
pid+=("$a")
cpu+=("$b")
mem+=("$c")
ctime+=("$d")
done < /tmp/deadpidlist.log
Use associative arrays for cpu, mem, and ctime. Also, read the input all in one pass via the read builtin.
declare -a pid
declare -A cpu
declare -A mem
declare -A ctime
while read this_pid this_cpu this_mem this_ctime tail; do
pid[${#pid[*]}]=$this_pid
cpu[$this_pid]=$this_cpu
mem[$this_pid]=$this_mem
ctime[$this_pid]=$this_ctime
done < /tmp/deadpidlist.log
for i in "${pid[#]}" do;
# ...
echo $i cpu[$i] mem[$i] ctime[$i]
done
For the 1st part:
while read -r ipid icpu imem ictime icmd
do
pid+=($ipid)
cpu+=($icpu)
mem+=($imem)
ctime+=($ictime)
cmd+=($icmd)
done < /tmp/deadpidlist.log
Some small comments for the gist:
Use functions. You can save much typing redirecting the output from a function - like:
some() {
echo some
cat /some/file
}
#later
some >> /some/outfile
also, you can save many echo with grouping them into one heredoc
some2() {
cat - <<EOF
some output what want
can use $variables
$(date) #command substitutions too
EOF
}
If you don't want variable expansions, use the heredoc as <<'EOF'
Also, you can use
let countertwo++ #instead of countertwo=$((countertwo + 1))
This question already has answers here:
How can I store the "find" command results as an array in Bash
(8 answers)
Closed 4 years ago.
How do I put the result of find $1 into an array?
In for loop:
for /f "delims=/" %%G in ('find $1') do %%G | cut -d\/ -f6-
I want to cry.
In bash:
file_list=()
while IFS= read -d $'\0' -r file ; do
file_list=("${file_list[#]}" "$file")
done < <(find "$1" -print0)
echo "${file_list[#]}"
file_list is now an array containing the results of find "$1
What's special about "field 6"? It's not clear what you were attempting to do with your cut command.
Do you want to cut each file after the 6th directory?
for file in "${file_list[#]}" ; do
echo "$file" | cut -d/ -f6-
done
But why "field 6"? Can I presume that you actually want to return just the last element of the path?
for file in "${file_list[#]}" ; do
echo "${file##*/}"
done
Or even
echo "${file_list[#]##*/}"
Which will give you the last path element for each path in the array. You could even do something with the result
for file in "${file_list[#]##*/}" ; do
echo "$file"
done
Explanation of the bash program elements:
(One should probably use the builtin readarray instead)
find "$1" -print0
Find stuff and 'print the full file name on the standard output, followed by a null character'. This is important as we will split that output by the null character later.
<(find "$1" -print0)
"Process Substitution" : The output of the find subprocess is read in via a FIFO (i.e. the output of the find subprocess behaves like a file here)
while ...
done < <(find "$1" -print0)
The output of the find subprocess is read by the while command via <
IFS= read -d $'\0' -r file
This is the while condition:
read
Read one line of input (from the find command). Returnvalue of read is 0 unless EOF is encountered, at which point while exits.
-d $'\0'
...taking as delimiter the null character (see QUOTING in bash manpage). Which is done because we used the null character using -print0 earlier.
-r
backslash is not considered an escape character as it may be part of the filename
file
Result (first word actually, which is unique here) is put into variable file
IFS=
The command is run with IFS, the special variable which contains the characters on which read splits input into words unset. Because we don't want to split.
And inside the loop:
file_list=("${file_list[#]}" "$file")
Inside the loop, the file_list array is just grown by $file, suitably quoted.
arrayname=( $(find $1) )
I don't understand your loop question? If you look how to work with that array then in bash you can loop through all array elements like this:
for element in $(seq 0 $((${#arrayname[#]} - 1)))
do
echo "${arrayname[$element]}"
done
This is probably not 100% foolproof, but it will probably work 99% of the time (I used the GNU utilities; the BSD utilities won't work without modifications; also, this was done using an ext4 filesystem):
declare -a BASH_ARRAY_VARIABLE=$(find <path> <other options> -print0 | sed -e 's/\x0$//' | awk -F'\0' 'BEGIN { printf "("; } { for (i = 1; i <= NF; i++) { printf "%c"gensub(/"/, "\\\\\"", "g", $i)"%c ", 34, 34; } } END { printf ")"; }')
Then you would iterate over it like so:
for FIND_PATH in "${BASH_ARRAY_VARIABLE[#]}"; do echo "$FIND_PATH"; done
Make sure to enclose $FIND_PATH inside double-quotes when working with the path.
Here's a simpler pipeless version, based on the version of user2618594
declare -a names=$(echo "("; find <path> <other options> -printf '"%p" '; echo ")")
for nm in "${names[#]}"
do
echo "$nm"
done
To loop through a find, you can simply use find:
for file in "`find "$1"`"; do
echo "$file" | cut -d/ -f6-
done
It was what I got from your question.
I need to know how many processes are running for a specific task (e.g. number of Apache tomcats) and if it's 1, then print the PID. Otherwise print out a message.
I need this in a BASH script, now when I perform something like:
result=`ps aux | grep tomcat | awk '{print $2}' | wc -l`
The number of items is assigned to result. Hurrah! But I don't have the PID(s). However when I attempt to perform this as an intermediary step (without the wc), I encounter problems. So if I do this:
result=`ps aux | grep tomcat | awk '{print $2}'`
Any attempts I make to modify the variable result just don't seem to work. I've tried set and tr (replace blanks with line-breaks), but I just cannot get the right result. Ideally I'd like the variable result to be an array with the PIDs as individual elements. Then I can see size, elements, easily.
Can anyone suggest what I am doing wrong?
Thanks,
Phil
Update:
I ended up using the following syntax:
pids=(`ps aux | grep "${searchStr}"| grep -v grep | awk '{print $2}'`)
number=${#pids[#]}
The key was putting the brackets around the back-ticked commands. Now the variable pids is an array and can be asked for length and elements.
Thanks to both choroba and Dimitre for their suggestions and help.
pids=($(
ps -eo pid,command |
sed -n '/[t]omcat/{s/^ *\([0-9]\+\).*/\1/;p}'
))
number=${#pids[#]}
pids=( ... ) creates an array.
$( ... ) returns its output as a string (similar to backquote).
Then, sed is called on the list of all the processes: for lines containing tomacat (the [t] prevents the sed itself from being included), only the pid is preserved and printed.
You may need to adjust the pgrep command (you may need or may not need the -f option).
_pids=(
$( pgrep -f tomcat )
)
(( ${#_pids[#]} == 1 )) &&
echo ${_pids[0]} ||
echo message
If you want to print the number of pids (with a message):
_pids=(
$( pgrep -f tomcat )
)
(( ${#_pids[#]} == 1 )) &&
echo ${_pids[0]} ||
echo "${#_pids[#]} running"
It should be noted that the pgrep utility and the syntax used are not standard.