--As requested
GNU Bash: version 4.4.12(3)-release (x86_64-unknown-cygwin)
Cygwin Version: 2.8.2 (i think, or whatever is the most current)
Windows Server 2012R2
EDIT (requested to update with an actual example that can be verified)
testdir=$(mktemp -t -d testdir.XXXXXX)
cd "$testdir/" && touch file{1..99}
read -r -a ARRAY <<< $(ls -alh "$testdir" | grep -Eo "file[0-9]{2}")
# grep above grep should ignore 0-9 (single digits)
In proper Linux shell: echo "${ARRAY[#]}" returns file10 file11 file12 file13 file14 file15 ..... file99
In a proper Linux shell: echo "${ARRAY[5]}" returns: file15
In Cygwin shell: echo "${ARRAY[#]}" returns file10
In Cygwin shell: echo "${ARRAY[5]}" returns nothing (as nothing indexed)
for i in "${!ARRAY[#]}"; do echo "Key: $i"; echo "Value: ${ARRAY[$i]}"; done
In a proper Linux Shell:
Key: 0
Value: file10
...
Key: 89
Value: file99
In Cygwin:
Key: 0
Value: file10
file11
file12
file13
file14
...etc.
ORIGINAL QUESTION BELOW:
I'm using Cygwin for the first time and I'm having a hard time understanding why my indexed array does populate properly:
This has worked fine in a proper Linux shell, which I've double checked to make sure i'm not losing my mind:
Which gets a long list of results, all properly indexed with unique keys ... whether wrong or right, this is how I learned to do it, and its always worked for me.
When I do this exact same thing in Cygwin, I only get one result.
If I do declare -a ARRAY and then ARRAY=$(stuff to do, results populate the array) I get many results, but all stored under a single key.
I feel I must be missing something basic, because I don't think it should be this hard.
Related
Question:
Is it possible to exclude some of a commands output its output based on an array of words?**
Why?
On Ubuntu/Debian there are two ways to list all available pkgs:
apt list (show all available pkgs, installed pkgs too)
apt-cache search . (show all available pkgs, installed pkgs too)
Difference is, the first command, you can exclude all installed pkgs using grep -v, problem is unlike the first, the second command you cannot exclude these as the word "installed" isnt present. Problem with the first command is it doesnt show the pkg description, so I want to use apt-cache search . but excluding all installed pkgs.
# List all of my installed pkgs,
# Get just the pkg's name,
# Swap newlines for spaces,
# Save this list as an array for exclusion.
INSTALLED=("$(apt list --installed 2> /dev/null | awk -F/ '{print $1}' | tr '\r\n' ' ')")
I then tried:
apt-cache search . | grep -v "${INSTALLED[#]}"
Unfortunately this doesnt work as I still see my installed pkgs too so I'm guessing its excluding the first pkg in the array and not the rest.
Again thank you in advance!
Would you please try the following:
#!/bin/bash
declare -A installed # an associative array to memorize the "installed" command names
while IFS=/ read -r cmd others; do # split the line on "/" and assign the variable "cmd" to the 1st field
(( installed[$cmd]++ )) # increment the array value associated with the "$cmd"
done < <(apt list --installed 2> /dev/null) # excecute the `apt` command and feed the output to the `while` loop
while IFS= read -r line; do # read the whole line "as is" because it includes a package description
read -r cmd others <<< "$line" # split the line on the whitespace and assign the variable "cmd" to the 1st field
[[ -z ${installed[$cmd]} ]] && echo "$line" # if the array value is not defined, the cmd is not installed, then print the line
done < <(apt-cache search .) # excecute the `apt-cache` command to feed the output to the `while` loop
The associative array installed is used to check if the command is
installed.
The 1st while loop scans over the installed list of the command and
store the command names in the associative array installed.
The 2nd while loop scans over the available command list and if the
command is not found in the associative array, then print it.
BTW your trial code starts with #!/bin/sh which is run with sh, not bash.
Please make sure it looks like #!/bin/bash.
Sorry if I'm misunderstanding, but you just want to get the list of packages which are not installed, right?
If so, you can just do this -
apt list --installed=false | grep -v '\[installed'
I'm trying to write a backup script that takes a specific list of disk's UUIDs, mounts them to specified points, rsyncs the data to a specified end point, and then also does a bunch of other conditional checks when that's done. But since there will be a lot of checking after rsync, it would be nice to set each disk's UUID and associated/desired mount point as strings to save hardcoding them throughout the script, and also if say the UUIDs change in future (if a drive is updated or swapped out), this will be easier to maintain the script...
I've been looking at arrays in bash, to make the list of wanted disks, but have questions about how to make this possible, as my experience with arrays is non-existent!
The list of disks---in order of priority wanted to backup---are:
# sdc1 2.7T UUID 8C1CC0C19D012E29 /media/user/Documents/
# sdb1 1.8T UUID 39CD106C6FDA5907 /media/user/Photos/
# sdd1 3.7T UUID 5104D5B708E102C0 /media/user/Video/
... and notice I want to go sdc1, sdb1, sdd1 and so on, (i.e. custom order).
Is it possible to create this list, in order of priority, so it's something like this?
DisksToBackup
└─1
└─UUID => '8C1CC0C19D012E29'
└─MountPoint => '/media/user/Documents/'
└─2
└─UUID => '39CD106C6FDA5907'
└─MountPoint => '/media/user/Photos/'
└─3
└─UUID => '5104D5B708E102C0'
└─MountPoint => '/media/user/Video/'
OR some obviously better idea than this...
And then how to actually use this?
Let's say for example, how to go through our list and mount each disk (I know this is incorrect syntax, but again, I know nothing about arrays:
mount --uuid $DisksToBackup[*][UUID] $DisksToBackup[*][MountPoint]?
Update: Using Linux Mint 19.3
Output of bash --version gives: GNU bash, version 4.4.20(1)
Bash starting with version 4 provides associative arrays but only using a single dimension. Multiple dimensions you would have to simulate using keys such as 'sdc1-uuid' as shown in the following interactive bash examples (remove leading $ and > and bash output when putting into a script).
$ declare -A disks
$ disks=([0-uuid]=8C1CC0C19D012E29 [0-mount]=/media/user/Documents/
> [1-uuid]=39CD106C6FDA5907 [1-mount]=/media/user/Photos/)
$ echo ${disks[sdc1-uuid]}
8C1CC0C19D012E29
$ echo ${disks[*]}
/media/user/Documents/ 39CD106C6FDA5907 8C1CC0C19D012E29 /media/user/Photos/
$ echo ${!disks[*]}
0-mount 0-uuid 1-uuid 1-mount
However, there is no ordering for the keys (the order of the keys differs from the order in which we defined them). You may want to use a second array as in the following example which allows you to break down the multiple dimensions as well:
$ disks_order=(0 1)
$ for i in ${disks_order[*]}; do
> echo "${disks[$i-uuid]} ${disks[$i-mount]}"
> done
8C1CC0C19D012E29 /media/user/Documents/
39CD106C6FDA5907 /media/user/Photos/
In case you use bash version 3, you need to simulate the associative array using other means. See the question on associative arrays in bash 3 or simply represent your structure in a simple array such as which makes everything more readable anyway:
$ disks=(8C1CC0C19D012E29=/media/user/Documents/
> 39CD106C6FDA5907=/media/user/Photos/)
$ for disk in "${disks[#]}"; do
> uuid="${disk%=*}"
> path="${disk##*=}"
> echo "$uuid $path"
> done
8C1CC0C19D012E29 /media/user/Documents/
39CD106C6FDA5907 /media/user/Photos/
The %=* is a fancy way of saying remove everything after (and including) the = sign. And ##*= to remove everything before (and including) the = sign.
As an example of how one could read this data into a series of arrays:
#!/usr/bin/env bash
i=0
declare -g -A "disk$i"
declare -n currDisk="disk$i"
while IFS= read -r line || (( ${#currDisk[#]} )); do : "line=$line"
if [[ $line ]]; then
if [[ $line = *=* ]]; then
currDisk[${line%%=*}]=${line#*=}
else
printf 'WARNING: Ignoring unrecognized line: %q\n' "$line" >&2
fi
else
if [[ ${#currDisk[#]} ]]; then
declare -p "disk$i" >&2 # for debugging/demo: print out what we created
(( ++i ))
unset -n currDisk
declare -g -A "disk$i=( )"
declare -n currDisk="disk$i"
fi
fi
done < <(blkid -o export)
This gives you something like:
declare -g -A disk0=( [PARTLABEL]="primary" [UUID]="1111-2222-3333" [TYPE]=btrfs ...)
declare -g -A disk1=( [PARTLABEL]="esp" [LABEL]="boot" [TYPE]="vfat" ...)
...so you can write code iterating over them doing whatever search/comparison/etc you want. For example:
for _diskVar in "${!disk#}"; do # iterates over variable names starting with "disk"
declare -n _currDisk="$_diskVar" # refer to each such variable as _currDisk in turn
# replace the below with your actual application logic, whatever that is
if [[ ${_currDisk[LABEL]} = "something" ]] && [[ ${_currDisk[TYPE]} = "something_else" ]]; then
echo "Found ${_currDisk[DEVNAME]}"
fi
unset -n _currDisk # clear the nameref when done with it
done
Following my previous question which got closed— basically I have a script that check availability of packages on target server, the target server and the packages have been stored to an array.
declare -a prog=("gdebi" "firefox" "chromium-browser" "thunar")
declare -a snap=("beer2" "beer3")
# checkvar=$(
for f in "${prog[#]}"; do
for connect in "${snap[#]}"; do
ssh lg#"$connect" /bin/bash <<- EOF
if dpkg --get-selections | grep -qE "(^|\s)"$f"(\$|\s)"; then
status="[INSTALLED] [$connect]"
else
status=""
fi
printf '%s %s\n' "$f" "\$status"
EOF
done
done
With the help of fellow member here, I've made several fix to original script, script ran pretty well— except there's one problem, the output contain duplicate entries.
gdebi [INSTALLED] [beer2]
gdebi
firefox [INSTALLED] [beer2]
firefox [INSTALLED] [beer3]
chromium-browser [INSTALLED] [beer2]
chromium-browser [INSTALLED] [beer3]
thunar
thunar
I know it this is normal behavior, as for pass multiple server from snap array, making ssh travel to all the two server.
Considering that the script checks same package for two server, I want the output to be merged.
If beer2 have firefox packages, but beer3 doesn't.
firefox [INSTALLED] [beer2]
If beer3 have firefox packages, but beer2 doesn't.
firefox [INSTALLED] [beer3]
If both beer2 and beer3 have the packages.
firefox [INSTALLED] [beer2, beer3]
or
firefox [INSTALLED] [beer2] [beer3]
If both beer2 and beer3 doesn't have the package, it will return without extra parameter.
firefox
Sound like an easy task, but for the love of god I can't find how to achieve this, here's list of things I have tried.
Try to manipulate the for loops.
Try putting return value after one successful loops (exit code).
Try nested if.
All of the above doesn't seem to work, I haven't tried changing/manipulate the return string as I'm not really experienced with some text processing such as: awk, sed, tr and many others.
Can anyone shows how It's done ? Would really mean the world to me.
Pure Bash 4+ solution using associative array to store hosts the program is installed on:
#!/usr/bin/env bash
declare -A hosts_with_package=(["gdebi"]="" ["firefox"]="" ["chromium-browser"]="" ["thunar"]="")
declare -a hosts=("beer2" "beer3")
# Collect installed status
# Iterate all hosts
for host in "${hosts[#]}"; do
# Read the output of dpkg --get-selections with searched packages
while IFS=$' \t' read -r package status; do
# Test weather package is installed on host
if [ "$status" = "install" ]; then
# If no host listed for package, create first entry
if [ -z "${hosts_with_package[$package]}" ]; then
# Record the first host having the package installed
hosts_with_package["$package"]="$host"
else
# Additional hosts are concatenated as CSV
hosts_with_package["$package"]="${hosts_with_package[$package]}, $host"
fi
fi
# Feed the whole loop with the output of the dpkg --get-selections for packages names
# Packages names are the index of the hosts_with_package array
done < <(ssh "lg#$host" dpkg --get-selections "${!hosts_with_package[#]}")
done
# Output results
# Iterate the package name keys
for package in "${!hosts_with_package[#]}"; do
# Print package name without newline
printf '%s' "$package"
# If package is installed on some hosts
if [ -n "${hosts_with_package[$package]}" ]; then
# Continue the line with installed hosts
printf ' [INSTALLED] [%s]' "${hosts_with_package[$package]}"
fi
# End with a newline
echo
done
Instead of making several ssh connections in nested loops consider this change
prog=( mysql-server apache2 php ufw )
snap=( localhost )
for connect in ${snap[#]}; do
ssh $connect "
progs=( ${prog[#]} )
for prog in \${progs[#]}; do
dpkg -l | grep -q \$prog && echo \"\$prog [INSTALLED]\" || echo \"\$prog\"
done
"
done
Based on #Ivan answer
#!/bin/bash
prog=( "gdebi" "firefox" "chromium-browser" "thunar" )
snap=( "beer2" "beer3" )
# First, retrieve the list on installed program for each host
for connect in ${snap[#]}; do
ssh lg#"$connect" /bin/bash >/tmp/installed.${connect} <<- EOF
progs=( "${prog[#]}" )
for prog in \${progs[#]}; do
dpkg --get-selections | awk -v pkg=\$prog '\$1 == pkg && \$NF ~ /install/ {print \$1}'
done
EOF
done
# Filter the previous results to format the output as you need
awk '{
f = FILENAME;
gsub(/.*\./,"",f);
a[$1] = a[$1] "," f
}
END {
for (i in a)
print i ":[" substr(a[i],2) "]"
}' /tmp/installed.*
rm /tmp/installed.*
Example of output :
# With prog=( bash cat sed tail something firefox-esr )
firefox-esr:[localhost]
bash:[localhost,localhost2]
sed:[localhost,localhost2]
I am attempting to get a list of running VirtualBox VMs (the UUIDs) and put them into an array. The command below produces the output below:
$ VBoxManage list runningvms | awk -F '[{}]' '{print $(NF-1)}'
f93c17ca-ab1b-4ba2-95e5-a1b0c8d70d2a
46b285c3-cabd-4fbb-92fe-c7940e0c6a3f
83f4789a-b55b-4a50-a52f-dbd929bdfe12
4d1589ba-9153-489a-947a-df3cf4f81c69
I would like to take those UUIDs and put them into an array (possibly even an associative array for later use, but a simple array for now is sufficient)
If I do the following:
array1="( $(VBoxManage list runningvms | awk -F '[{}]' '{print $(NF-1)}') )"
The commands
array1_len=${#array1[#]}
echo $array1_len
Outputs "1" as in there's only 1 element. If I print out the elements:
echo ${array1[*]}
I get a single line of all the UUIDs
( f93c17ca-ab1b-4ba2-95e5-a1b0c8d70d2a 46b285c3-cabd-4fbb-92fe-c7940e0c6a3f 83f4789a-b55b-4a50-a52f-dbd929bdfe12 4d1589ba-9153-489a-947a-df3cf4f81c69 )
I did some research (Bash Guide/Arrays on how to tackle this and found this with command substitution and redirection, but it produces an empty array
while read -r -d '\0'; do
array2+=("$REPLY")
done < <(VBoxManage list runningvms | awk -F '[{}]' '{print $(NF-1)}')
I'm obviously missing something. I've looked at several simiar questions on this site such as:
Reading output of command into array in Bash
AWK output to bash Array
Creating an Array in Bash with Quoted Entries from Command Output
Unfortunately, none have helped. I would apprecaite any assistance in figuring out how to take the output and assign it to an array.
I am running this on macOS 10.11.6 (El Captain) and BASH version 3.2.57
Since you're on a Mac:
brew install bash
Then with this bash as your shell, pipe the output to:
readarray -t array1
Of the -t option, the man page says:
-t Remove a trailing delim (default newline) from each line read.
If the bash4 solution is admissible, then the advice given
e.g. by gniourf_gniourf at reading-output-of-command-into-array-in-bash
is still sound.
I have tried everything I can think of to cut this into separate elements for my array but I am struggling..
Here is what I am trying to do..
(This command just rips out the IP addresses on the first element returned )
$ IFS=$"\n"
$ aaa=( $(netstat -nr | grep -v '^0.0.0.0' | grep -v 'eth' | grep "UGH" | sed 's/ .*//') )
$ echo "${#aaa[#]}"
1
$ echo "${aaa[0]}"
4.4.4.4
5.5.5.5
This shows more than one value when I am looking for the array to separate 4.4.4.4 into ${aaa[0]} and 5.5.5.5 into ${aaa[1]}
I have tried:
IFS="\n"
IFS=$"\n"
IFS=" "
Very confused as I have been working with arrays a lot recently and have never ran into this particular issue.
Can someone tell me what I am doing wrong?
There is a very good example on how to use IFS + read -a to split a string into an array on this other stackoverflow page
How does splitting string to array by 'read' with IFS word separator in bash generated extra space element?
netstat is deprecated, replaced by ss, so I'm not sure how to reproduce your exact problem