Trouble with AWK'd command output and bash array - arrays

I am attempting to get a list of running VirtualBox VMs (the UUIDs) and put them into an array. The command below produces the output below:
$ VBoxManage list runningvms | awk -F '[{}]' '{print $(NF-1)}'
f93c17ca-ab1b-4ba2-95e5-a1b0c8d70d2a
46b285c3-cabd-4fbb-92fe-c7940e0c6a3f
83f4789a-b55b-4a50-a52f-dbd929bdfe12
4d1589ba-9153-489a-947a-df3cf4f81c69
I would like to take those UUIDs and put them into an array (possibly even an associative array for later use, but a simple array for now is sufficient)
If I do the following:
array1="( $(VBoxManage list runningvms | awk -F '[{}]' '{print $(NF-1)}') )"
The commands
array1_len=${#array1[#]}
echo $array1_len
Outputs "1" as in there's only 1 element. If I print out the elements:
echo ${array1[*]}
I get a single line of all the UUIDs
( f93c17ca-ab1b-4ba2-95e5-a1b0c8d70d2a 46b285c3-cabd-4fbb-92fe-c7940e0c6a3f 83f4789a-b55b-4a50-a52f-dbd929bdfe12 4d1589ba-9153-489a-947a-df3cf4f81c69 )
I did some research (Bash Guide/Arrays on how to tackle this and found this with command substitution and redirection, but it produces an empty array
while read -r -d '\0'; do
array2+=("$REPLY")
done < <(VBoxManage list runningvms | awk -F '[{}]' '{print $(NF-1)}')
I'm obviously missing something. I've looked at several simiar questions on this site such as:
Reading output of command into array in Bash
AWK output to bash Array
Creating an Array in Bash with Quoted Entries from Command Output
Unfortunately, none have helped. I would apprecaite any assistance in figuring out how to take the output and assign it to an array.
I am running this on macOS 10.11.6 (El Captain) and BASH version 3.2.57

Since you're on a Mac:
brew install bash
Then with this bash as your shell, pipe the output to:
readarray -t array1
Of the -t option, the man page says:
-t Remove a trailing delim (default newline) from each line read.
If the bash4 solution is admissible, then the advice given
e.g. by gniourf_gniourf at reading-output-of-command-into-array-in-bash
is still sound.

Related

Creating an array of Strings from Grep Command

I'm pretty new to Linux and I've been trying some learning recently. One thing I'm struggling is Within a log file I would like to grep for all the unique IDs that exist and store them in an array.
The format of the ids are like so id=12345678,
I'm struggling though to get these in to an array. So far I've tried a range of things, the below however
a=($ (grep -HR1 `id=^[0-9]' logfile))
echo ${#a[#]}
but the echo count is always returned as 0. So it is clear the populating of the array is not working. Have explored other pages online, but nothing seems to have a clear explanation of what I am looking for exactly.
a=($(grep -Eow 'id=[0-9]+' logfile))
a=("${a[#]#id=}")
printf '%s\n' "${a[#]}"
It's safe to split an unquoted command substitution here, as we aren't printing pathname expansion characters (*?[]), or whitespace (other than the new lines which delimit the list).
If this were not the case, mapfile -t a <(grep ...) is a good alternative.
-E is extended regex (for +)
-o prints only matching text
-w matches a whole word only
${a[#]#id=} strips the id suffix from each array element
Here is an example
my_array=()
while IFS= read -r line; do
my_array+=( "$line" )
done < <( ls )
echo ${#my_array[#]}
printf '%s\n' "${my_array[#]}"
It prints out 14 and then the names of the 14 files in the same folder. Just substitute your command instead of ls and you started.
Suggesting readarray command to make sure it array reads full lines.
readarray -t my_array < <(grep -HR1 'id=^[0-9]' logfile)
printf "%s\n" "${my_array[#]}"

exclude stdout based on an array of words

Question:
Is it possible to exclude some of a commands output its output based on an array of words?**
Why?
On Ubuntu/Debian there are two ways to list all available pkgs:
apt list (show all available pkgs, installed pkgs too)
apt-cache search . (show all available pkgs, installed pkgs too)
Difference is, the first command, you can exclude all installed pkgs using grep -v, problem is unlike the first, the second command you cannot exclude these as the word "installed" isnt present. Problem with the first command is it doesnt show the pkg description, so I want to use apt-cache search . but excluding all installed pkgs.
# List all of my installed pkgs,
# Get just the pkg's name,
# Swap newlines for spaces,
# Save this list as an array for exclusion.
INSTALLED=("$(apt list --installed 2> /dev/null | awk -F/ '{print $1}' | tr '\r\n' ' ')")
I then tried:
apt-cache search . | grep -v "${INSTALLED[#]}"
Unfortunately this doesnt work as I still see my installed pkgs too so I'm guessing its excluding the first pkg in the array and not the rest.
Again thank you in advance!
Would you please try the following:
#!/bin/bash
declare -A installed # an associative array to memorize the "installed" command names
while IFS=/ read -r cmd others; do # split the line on "/" and assign the variable "cmd" to the 1st field
(( installed[$cmd]++ )) # increment the array value associated with the "$cmd"
done < <(apt list --installed 2> /dev/null) # excecute the `apt` command and feed the output to the `while` loop
while IFS= read -r line; do # read the whole line "as is" because it includes a package description
read -r cmd others <<< "$line" # split the line on the whitespace and assign the variable "cmd" to the 1st field
[[ -z ${installed[$cmd]} ]] && echo "$line" # if the array value is not defined, the cmd is not installed, then print the line
done < <(apt-cache search .) # excecute the `apt-cache` command to feed the output to the `while` loop
The associative array installed is used to check if the command is
installed.
The 1st while loop scans over the installed list of the command and
store the command names in the associative array installed.
The 2nd while loop scans over the available command list and if the
command is not found in the associative array, then print it.
BTW your trial code starts with #!/bin/sh which is run with sh, not bash.
Please make sure it looks like #!/bin/bash.
Sorry if I'm misunderstanding, but you just want to get the list of packages which are not installed, right?
If so, you can just do this -
apt list --installed=false | grep -v '\[installed'

Issue storing awk output in bash array

I have below file:
Site is facebook.
Site is microsoft.
Site is google.
And below script:
#!/bin/bash
#tried arr=$(awk {'print'} test) which gives array length as 1
arr=($(awk {'print'} test))
echo "Length ::: ${#arr[#]}"
Here the expected output is 3. However, I am getting length of array as 9. Above is just an excerpt from a script and need to use awk here.
Please let me know where the issue is....
This is the correct way to build a shell array of one entry per line from output of awk (requires bash 4+):
readarray -t arr < <(awk '1' file)
declare -p arr
declare -a arr=([0]="Site is facebook." [1]="Site is microsoft." [2]="Site is google")
When you use this code:
arr=($(awk '1' file))
Then shell splits on default delimiter and assigns each word from awk output to a separate array entry.
Having said that please bear in mind that awk is capable of doing everything that shell can do and it is always better to process your data in awk itself.

How do I store the output from a find command in an array? + bash

I have the following find command with the following output:
$ find -name '*.jpg'
./public_html/github/screencasts-gh-pages/reactiveDataVis/presentation/images/telescope.jpg
./public_html/github/screencasts-gh-pages/introToBackbone/presentation/images/telescope.jpg
./public_html/github/StarCraft-master/img/Maps/(6)Thin Ice.jpg
./public_html/github/StarCraft-master/img/Maps/Snapshot.jpg
./public_html/github/StarCraft-master/img/Maps/Map_Grass.jpg
./public_html/github/StarCraft-master/img/Maps/(8)TheHunters.jpg
./public_html/github/StarCraft-master/img/Maps/(2)Volcanis.jpg
./public_html/github/StarCraft-master/img/Maps/(3)Trench wars.jpg
./public_html/github/StarCraft-master/img/Maps/(8)BigGameHunters.jpg
./public_html/github/StarCraft-master/img/Maps/(8)Turbo.jpg
./public_html/github/StarCraft-master/img/Maps/(4)Blood Bath.jpg
./public_html/github/StarCraft-master/img/Maps/(2)Switchback.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(6)Thin Ice.jpg
./public_html/github/StarCraft-master/img/Maps/Original/Map_Grass.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(8)TheHunters.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(2)Volcanis.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(3)Trench wars.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(8)BigGameHunters.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(8)Turbo.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(4)Blood Bath.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(2)Switchback.jpg
./public_html/github/StarCraft-master/img/Maps/Original/(4)Orbital Relay.jpg
./public_html/github/StarCraft-master/img/Maps/(4)Orbital Relay.jpg
./public_html/github/StarCraft-master/img/Bg/GameLose.jpg
./public_html/github/StarCraft-master/img/Bg/GameWin.jpg
./public_html/github/StarCraft-master/img/Bg/GameStart.jpg
./public_html/github/StarCraft-master/img/Bg/GamePlay.jpg
./public_html/github/StarCraft-master/img/Demo/Demo.jpg
./public_html/github/flot/examples/image/hs-2004-27-a-large-web.jpg
./public_html/github/minicourse-ajax-project/other/GameLose.jpg
How do I store this output in an array? I want it to handle filenames with spaces
I have tried this arrayname=($(find -name '*.jpg')) but this just stores the first element. # I am doing the following which seems to be just the first element?
$ arrayname=($(find -name '*.jpg'))
$ echo "$arrayname"
./public_html/github/screencasts-gh-pages/reactiveDataVis/presentation/images/telescope.jpg
$
I have tried here but again this just stores the 1st element
Other similar Qs
How do I capture the output from the ls or find command to store all file names in an array?
How do i store the output of a bash command in a variable?
If you know with certainty that your filenames will not contain newlines, then
mapfile -t arrayname < <(find ...)
If you want to be able to handle any file
arrayname=()
while IFS= read -d '' -r filename; do
arrayname+=("$filename")
done < <(find ... -print0)
echo "$arrayname" will only show the first element of the array. It is equivalent to echo "${arrayname[0]}". To dump an array:
printf "%s\n" "${arrayname[#]}"
# ............^^^^^^^^^^^^^^^^^ must use exactly this form, with the quotes.
arrayname=($(find ...)) is still wrong. It will store the file ./file with spaces.txt as 3 separate elements in the array.
If you have a sufficiently recent version of bash, you can save yourself a lot of trouble by just using a ** glob.
shopt -s globstar
files=(**/*.jpg)
The first line enables the feature. Once enabled, ** in a glob pattern will match any number (including 0) of directories in the path.
Using the glob in the array definition makes sure that whitespace is handled correctly.
To view an array in a form which could be used to define the array, use the -p (print) option to the declare builtin:
declare -p files

Having issues using IFS to cut a string into an array. BASH

I have tried everything I can think of to cut this into separate elements for my array but I am struggling..
Here is what I am trying to do..
(This command just rips out the IP addresses on the first element returned )
$ IFS=$"\n"
$ aaa=( $(netstat -nr | grep -v '^0.0.0.0' | grep -v 'eth' | grep "UGH" | sed 's/ .*//') )
$ echo "${#aaa[#]}"
1
$ echo "${aaa[0]}"
4.4.4.4
5.5.5.5
This shows more than one value when I am looking for the array to separate 4.4.4.4 into ${aaa[0]} and 5.5.5.5 into ${aaa[1]}
I have tried:
IFS="\n"
IFS=$"\n"
IFS=" "
Very confused as I have been working with arrays a lot recently and have never ran into this particular issue.
Can someone tell me what I am doing wrong?
There is a very good example on how to use IFS + read -a to split a string into an array on this other stackoverflow page
How does splitting string to array by 'read' with IFS word separator in bash generated extra space element?
netstat is deprecated, replaced by ss, so I'm not sure how to reproduce your exact problem

Resources