Format output of array elements with variable lenght - arrays

I have this arrays, with output to a file.txt:
the first array contains all IPs in subnet (192.168.1.0, 192.168.1.1, ecc)
the second array contains an 1 if that host is up, a 0 if down, in order to obtain a matrix like this:
192.168.1.0 192.168.1.1 ...
0 1 ...
How can I allign this elements? I want that the bit status (0 or 1) is at the center of the IP address.
Consider that the IP address may vary, depending on the machine, for example it can be
71.5.0.0
I put all my script if you need more info:
#!/bin/bash
# written by Cosimo Colaci
# Declarations
ROOT_UID=0
E_NOROOT=65
declare -a iplist
echo "Reading hosts status..."
# Must run as root.
# if [ "$UID" -ne "$ROOT_UID" ]
# then
# echo "You need to run this script as root. Exiting..."
# exit $E_NOROOT
# fi
# Find Gateway IP and Netmask
ip_gateway=$(route | tail -1 | awk '{print $1}')
netmask=$(route | tail -1 | awk '{print $3}')
# echo "$netmask" FOR DEBUG
# Find subnet
OLDIFS="$IFS"
IFS=.
read -r i1 i2 i3 i4 <<< "$ip_gateway"
read -r m1 m2 m3 m4 <<< "$netmask"
subnet_ip=$(printf "%d.%d.%d.%d\n" "$((i1 & m1))" "$((i2 & m2))" "$((i3 & m3))" "$((ii4 & m4))")
# echo "$subnet_ip" FOR DEBUG
IFS="$OLDIFS"
# Express netmask in CIDR notation, to know how many hosts to scan
cidr_by_netmask() {
#function returns prefix for given netmask in arg1
bits=0
for octet in $(echo $1| sed 's/\./ /g'); do
binbits=$(echo "obase=2; ibase=10; ${octet}"| bc | sed 's/0//g')
let bits+=${#binbits}
done
echo "/${bits}"
}
suffix=$(cidr_by_netmask $netmask)
# echo "$suffix" FOR DEBUG
# First raw of matrix file: all IPs in subnet
iplist=( $(nmap -sL $subnet_ip$suffix | grep "Nmap scan report" | awk '{print $NF}') )
# echo "${iplist[#]}" FOR DEBUG
let i=1
let j=0
while [ $j -lt ${#iplist[#]} ]
do
raw1[$i]=${iplist[$j]}
((i++))
((j++))
done
raw1[0]="#IP/TIME"
for value in ${raw1[#]}
do
echo -en "$value\t"
done >ip-time_matrix.txt
echo -e "\n" >>ip-time_matrix.txt
# Other raws
let j=1
let k=0
export j
export k
echo "Scanning hosts every 10s..."
echo "Hit CTRL-C to stop and check file \"ip-time_matrix.txt\""
while true
do
nmap -sP $subnet_ip$suffix &>temp_esame95a.txt
raw2[0]="$(date +%T)"
for ip in ${iplist[#]}
do
if grep "$ip" temp_esame95a.txt &>/dev/null
then
raw2[$j]="---1---"
else
raw2[$j]="---0---"
fi
((j++))
done
for value in ${raw2[#]}
do
echo -en "$value\t"
done >>ip-time_matrix.txt
echo -e "\n" >>ip-time_matrix.txt
sleep 10
j=1
done

In a comment a suggested printf "%15.15s %s" "${ip}" ${bitstatus}. I should have added a \n, printf "%15.15s %s\n" "${ip}" ${bitstatus}.
OP reacted, that it solved his problem.

For the arrays:
ips=(192.168.1.0 127.001.0.1 192.168.1.1 192.169.1.1)
raw=(1 1 0 1)
you could use printf and cut commands,
printf "%15s" "${ips[#]}" | cut -c5-
printf "%15s" "${raw[#]}" | cut -c5-
to align the output:
192.168.1.0 127.001.0.1 192.168.1.1 192.169.1.1
1 1 0 1

Related

how to Iterate an Array List inside 2 bash loops

I have 2 arrays in bash.
Array number 1 is the vlan subnet without the last octet.
Array number 2 is a list of octets i want to ignore, while scanning the subnet with Nmap.
lets assume every subnet has 254 pingable ip's (class c subnet)
I want the script to scan each subnet, and exclude ip's that ends with 1,2,3,252,253,254 which are Usually routers / firewalls / switches.
I manages to run 2 iterations, but failed on the if [[ $host == $host."${ignore[#]" ]] to identify the relevant ip (sunbet + ignore string)
Would really appreciate your help.
#!/bin/bash
# lets assume each subnet has 254 ips and all ignore ip's like 10.6.114.1 10.6.115.1 and 10.5.120.1
declare -a vlans=(
10.6.114
10.6.115
10.5.120
)
declare -a ignore=(
1
2
3
252
253
254
)
for vlan in "${vlans[#]}"; do
nmap -sn "$vlan" | grep Nmap | awk "{print $5}" | sed -n '1!p' | sed -e "$d" | sort > /tmp/vlan_ips.txt
readarray -t hosts < /tmp/vlan_ips.txt
for host in "${hosts[#]}"; do
check=$(echo "$host" | cut -d"." -f1-3)
if [ $host == $check."${ignore[#]}" ]; then
echo 'skipping record'
fi
done
done
This might work for you:
for vlan in "${vlans[#]}"; do
for ign in "${ignore[#]}"; do
printf '%s.%s\n' "$vlan" "$ign"
done >/tmp/ignore
nmap -n -sn "$vlan.0/24" -oG - 2>/dev/null |
grep -vwFf /tmp/ignore |
awk '/Host:/{print $2}' |
while read -r host; do
echo "$host"
done
done

Create an array with device logical name based on list of serial numbers

I'm tying to identify disks based on a list of UUID I get out of an api call.
The list of devices on the linux VM is following:
NAME="sda" SERIAL="NUTANIX_NFS_5_0_70509_a07b6add_60c9_4758_8b27_0afe77820dbd"
NAME="sdb" SERIAL="NUTANIX_NFS_18_0_625_ae748138_9bc0_4499_8068_d00a84a800b6"
NAME="sdc" SERIAL="NUTANIX_NFS_18_0_626_8a082956_a2be_42ca_a14b_7d21ed3d5a2d"
NAME="sdd" SERIAL="NUTANIX_NFS_18_0_627_6185353f_5c13_47ab_af58_b72d4d07106b"
NAME="sde" SERIAL="NUTANIX_NFS_18_0_628_fbeed366_6956_4de1_a217_487d75fd003c"
NAME="sdf" SERIAL="NUTANIX_NFS_18_0_629_019dcefb_0e1b_4086_96a9_982ae52fd88a"
NAME="sdg" SERIAL="NUTANIX_NFS_18_0_630_5d0b55e8_d0bd_4ee4_a2be_0bef88d94732"
NAME="sdh" SERIAL="NUTANIX_NFS_18_0_631_823beaed_1c2e_4203_ab5f_be294382d0c5"
NAME="sdi" SERIAL="NUTANIX_NFS_18_0_632_e4ee8620_5b53_4187_8f8c_7123ee8d3486"
NAME="sdj" SERIAL="NUTANIX_NFS_18_0_633_41551a1b_7dd9_4fd4_95bc_1fcdebb9dc85"
NAME="sdk" SERIAL="NUTANIX_NFS_18_0_634_12d517fc_e726_4512_b176_5d4075d0ecfe"
NAME="sdl" SERIAL="NUTANIX_NFS_18_0_635_d682eee4_cec0_4809_9efe_97aec7bf15d0"
NAME="sdm" SERIAL="NUTANIX_NFS_18_0_636_59ea679e_cbd3_4cf5_a974_67409b719391"
NAME="sdn" SERIAL="NUTANIX_NFS_18_0_637_5523c88e_310a_4fb6_b76f_48f624d1ce1f"
NAME="sdo" SERIAL="NUTANIX_NFS_18_0_638_0e189f7a_785d_46f1_bb56_8f422d421ecb"
NAME="sdp" SERIAL="NUTANIX_NFS_18_0_639_9143ba87_d742_4130_bbe0_2084e8ebad3f"
NAME="sdq" SERIAL="NUTANIX_NFS_18_0_640_3b823817_c2b6_49af_9a55_fc4706216501"
The list of drives I want to identify is in this format:
'ae748138_9bc0_4499_8068_d00a84a800b6'
'8a082956_a2be_42ca_a14b_7d21ed3d5a2d'
'6185353f_5c13_47ab_af58_b72d4d07106b'
'fbeed366_6956_4de1_a217_487d75fd003c'
'019dcefb_0e1b_4086_96a9_982ae52fd88a'
'5d0b55e8_d0bd_4ee4_a2be_0bef88d94732'
'823beaed_1c2e_4203_ab5f_be294382d0c5'
'e4ee8620_5b53_4187_8f8c_7123ee8d3486'
'41551a1b_7dd9_4fd4_95bc_1fcdebb9dc85'
'12d517fc_e726_4512_b176_5d4075d0ecfe'
'd682eee4_cec0_4809_9efe_97aec7bf15d0'
'59ea679e_cbd3_4cf5_a974_67409b719391'
'5523c88e_310a_4fb6_b76f_48f624d1ce1f'
'0e189f7a_785d_46f1_bb56_8f422d421ecb'
'9143ba87_d742_4130_bbe0_2084e8ebad3f'
'3b823817_c2b6_49af_9a55_fc4706216501'
I need to identify the logical name for only the UUIDs on that list in order to create a LVM volume group out of them. But I'm don't understand how to proceed from there. Anyone can propose an example ?
After removing the 's from you list of UUIDs (can be done with tr) you could use simply grep to extract the corresponding lines from your list of devices. To extract only the values of NAME="..." from these lines use cut.
grep -Ff <(tr -d \' < fileListOfUUIDs) fileListOfDevices | cut -d\" -f2
If the inputs you showed us are not stored in files but generated by commands you can use
commandListOfDevices | grep -Ff <(commandListOfUUIDs | tr -d \') | cut -d\" -f2
You are trying to extract field values from match results generated by a set of tokens. sed or awk can do the trick.
To extract the value of a field called NAME from one of the match results simply do this:
josh#linux ~ $ result='NAME="sda" FSTYPE="ext4" SERIAL="NUTANIX_...dbd"'
josh#linux ~ $ echo $result | sed -e 's/.*NAME="//; s/".*//;'
sda
To extract the value of a field called FSTYPE from the same match result:
josh#linux ~ $ echo $result | sed -e 's/.*FSTYPE="//; s/".*//;'
ext4
What if the order of the field changes? ..well the same code works.
josh#linux ~ $ result='FSTYPE="ext4" NAME="sda" SERIAL="NUTANIX_...dbd"'
josh#linux ~ $ echo $result | sed -e 's/.*NAME="//; s/".*//;'
sda
With this in mind, extraction of field values from all match results can then be processed in a loop. I have created a small script to automate the process and I am posting it here in the hope that it would be useful.
#!/bin/bash
#
# usage: doodle -F <file-containing-pattern(s)>
# doodle -P <pattern(s)>
#
if [ $# -ne 2 ]; then
echo 'invalid argument...'; exit;
else
pattern=
fi
if [ "$1" == "-F" ]; then
if [ ! -r $2 ]; then
echo "unable to read - $2"; exit
else
pattern=$(cat $2)
fi
elif [ "$1" == "-P" ]; then
pattern=$2
else
echo 'invalid argument...'; exit;
fi
if [ -z "$pattern" ]; then
echo 'invalid argument...'; exit;
fi
declare -a buffer
readarray buffer
if [ ${#buffer[*]} -gt 0 ]; then
for token in $pattern; do
for i in ${!buffer[*]}; do
match=$(echo ${buffer[i]} | sed -n "/$token/=");
if [ -n "$match" ]; then
##### Option 1 ##############################################################
# Uncomment the next line to print the same result that 'grep' would print.
#echo -e $(echo ${buffer[i]} | sed "s/$token/\\\\e[91m$token\\\\e[0m/");
##### Option 2 ##############################################################
# Uncomment the next line to print characters before the first [:space:]
#echo ${buffer[i]} | awk -F' ' '{ print $1 }';
##### Option 3 ##############################################################
# Uncomment the next line to email device-id to NSA
#TODO...
##### Option 4 ##############################################################
# Uncomment the next line to extract value of 'NAME' field
echo ${buffer[i]} | sed -e 's/.*NAME="//; s/".*//;'
##### Additional Option #####################################################
# Uncomment the next line (break command) to process one match per token
break;
fi
done
done
fi
unset buffer
To use this script, simply:
Copy the text in the above snippet into a file called doodle.
Open terminal and type:
josh#linux ~ $ chmod a+x doodle
To process tokens saved in a file called device-id.txt:
josh#linux ~ $ lsblk -nodeps -no name,serial -PS /dev/sd* | ./doodle -F device-ids.txt
To process tokens supplied as a single string:
josh#linux ~ $ lsblk -nodeps -no name,serial -PS /dev/sd* | ./doodle -P 'ae748138_9bc0_4499_8068_d00a84a800b6
8a082956_a2be_42ca_a14b_7d21ed3d5a2d
6185353f_5c13_47ab_af58_b72d4d07106b
...
3b823817_c2b6_49af_9a55_fc4706216501'
The script is very generic and you an can easily adapt it for use in related problems. You will find lots of tutorials on bash, sed and awk on the internet. There is also a comprehensive bash reference at gnu.org.

Fetching data into an array

I have a file like this below:
-bash-4.2$ cat a1.txt
0 10.95.187.87 5444 up 0.333333 primary 0 false 0
1 10.95.187.88 5444 up 0.333333 standby 1 true 0
2 10.95.187.89 5444 up 0.333333 standby 0 false 0
I want to fetch the data from the above file into a 2D array.
Can you please help me with a suitable way to put into an array.
Also post putting we need put a condition to check whether the value in the 4th column is UP or DOWN. If it's UP then OK, if its down then below command needs to be executed.
-bash-4.2$ pcp_attach_node -w -U pcpuser -h localhost -p 9898 0
(The value at the end is getting fetched from the 1st column.
You could try something like that:
while read -r line; do
declare -a array=( $line ) # use IFS
echo "${array[0]}"
echo "${array[1]}" # and so on
if [[ "$array[3]" ]]; then
echo execute command...
fi
done < a1.txt
Or:
while read -r -a array; do
if [[ "$array[3]" ]]; then
echo execute command...
fi
done < a1.txt
This works only if field are space separated (any kind of space).
You could probably mix that with regexp if you need more precise control of the format.
Firstly, I don't think you can have 2D arrays in bash. But you can however store lines into a 1-D array.
Here is a script ,parse1a.sh, to demonstrate emulation of 2D arrays for the type of data you included:
#!/bin/bash
function get_element () {
line=${ARRAY[$1]}
echo $line | awk "{print \$$(($2+1))}" #+1 since awk is one-based
}
function set_element () {
line=${ARRAY[$1]}
declare -a SUBARRAY=($line)
SUBARRAY[$(($2))]=$3
ARRAY[$1]="${SUBARRAY[#]}"
}
ARRAY=()
while IFS='' read -r line || [[ -n "$line" ]]; do
#echo $line
ARRAY+=("$line")
done < "$1"
echo "Full array contents printout:"
printf "%s\n" "${ARRAY[#]}" # Full array contents printout.
for line in "${ARRAY[#]}"; do
#echo $line
if [ "$(echo $line | awk '{print $4}')" == "down" ]; then
echo "Replace this with what to do for down"
else
echo "...and any action for up - if required"
fi
done
echo "Element access of [2,3]:"
echo "get_element 2 3 : "
get_element 2 3
echo "set_element 2 3 left: "
set_element 2 3 left
echo "get_element 2 3 : "
get_element 2 3
echo "Full array contents printout:"
printf "%s\n" "${ARRAY[#]}" # Full array contents printout.
It can be executed by:
./parsea1 a1.txt
Hope this is close to what you are looking for. Note that this code will loose all indenting spaces during manipulation, but a formatted update of the lines could solve that.

How to deny empty array indexes in bash

I wrote the bash like so:
#!/bin/bash
GAP=1
Out=$1
ResultFile=$2
len=`wc -l $Out | awk '{print $1}'`
eval "(COMMAND) &"
pid=$!
i=0
while kill -0 $pid; do
if [ -N $Out ]; then
newlen=`wc -l $Out | awk '{print $1}'`
newlines=`expr $newlen - $len`
tail -$newlines $Out > temp
IP=( $(sed -n '<SomeThing>' temp) )
host=${IP[$i]}
echo "exit" | nc $host 23
if [ "$?" -eq "0" ]; then
(
<DoingSomeThing>
) | nc $host 23 1>>$ResultFile 2>&1
fi
len=$newlen
let i++
fi
sleep $GAP
done
When the command IP=( $(sed -n '<SomeThing>' temp) ) is running in my bash maybe the result of sed command is nothing and maybe the output is ip. I want only when output of sed command get ip write it into array and when the output of sed is empty does not write it to array.
Thank you
You're not doing your script right in many ways but about your question, the quick way is to store the output first on a variable:
SED_OUT=$(sed -n '<SomeThing>' temp)
[[ -n $SED_OUT ]] && IP=($SED_OUT) ## Would only alter IP if $SED_OUT has a value.

grep result into array

I want to count the number of IP's in a given file using this function, the IP's should go into an array so that I can use them later, but I get 'declare: not found' and 'cnt+=1: not found', why is this?
#!/bin/bash
searchString=$1
file=$2
countLines()
{
declare -a ipCount
cnt=0
while read line ; do
((cnt+=1))
ipaddr=$( echo "$line" | grep -o -E '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' )
((ipCount[$ipaddr]+=1))
done
for ip in ${ipCount[*]}
do
printf "%-15s %s\n" "$ip" "${ipCount[$ip]}"
done
echo "total count=$cnt"
}
if [ $# -lt 2 ]; then
echo "missing searchString and file"
else
grep "$searchString" $file | countLines
fi
This is a piece of the test file I am trying on
Apr 25 11:33:21 Admin CRON[2792]: pam_unix(cron:session): session opened for user 192.168.1.2 by (uid=0)
Apr 25 12:39:01 Admin CRON[2792]: pam_unix(cron:session): session closed for user 192.168.1.2
Apr 27 07:42:07 John CRON[2792]: pam_unix(cron:session): session opened for user 192.168.2.22 by (uid=0)
The desired output would be just the IP's inside an array, then also a 'count' on how many IP's there were.
I know I can get the ip's with a grep command but I would like to do more with it later, and it's important that it's in an array.
Your two main issues were that you were using declare -a but to declare an associative array, you need declare -A. Then, to iterate over the keys of an associative array, you need to use for foo in ${!ArrayName[#]}. I also added some quotes to your variables to be on the safe side:
#!/bin/bash
searchString="$1"
file="$2"
countLines()
{
## You need -A for associative arrays
declare -A ipCount
cnt=0
while read line ; do
(( cnt+=1 ))
ipaddr=$( echo "$line" | grep -o -E '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' )
(( ipCount["$ipaddr"]+=1 ))
done
## To iterate over the keys of an associative
## array, you need ${!ArrayName[#]}
for ip in "${!ipCount[#]}"
do
printf "%-15s %s\n" "$ip" "${ipCount[$ip]}"
done
echo "total count=$cnt"
}
if [ $# -lt 2 ]; then
echo "missing searchString and file"
else
grep "$searchString" "$file" | countLines
fi
This is the output of the above on your example file:
$ bash a.sh 27 a
192.168.2.22 1
192.168.1.2 2
total count=3

Resources