I want that output result to grep second and third column:
1 db1 ADM_DAT 300 yes 95.09
2 db2 SYSAUX 400 yes 94.52
and convert them like array for example:
outputres=("db1 ADM_DAT" "db2 SYSAUX")
and after that to be able to read those values in loop for example:
for i in "${outputres[#]}"; do read -r a b <<< "$i"; unix_command $(cat file|grep $a|awk '{print $1}') $a $b;done
file:
10.1.1.1 db1
10.1.1.2 db2
Final expectation:
unix_command 10.1.1.1 db1 ADM_DAT
unix_command 10.1.1.2 db2 SYSAUX
This is only a theoretical example, I am not sure if it is working.
I would use a simple bash while read and keep adding elements into the array with the += syntax:
outputres=()
while read -r _ a b _; do
outputres+=("$a $b")
done < file
Doing so, with your input file, I got:
$ echo "${outputres[#]}" #print all elements
db1 ADM_DAT db2 SYSAUX
$ echo "${outputres[0]}" #print first one
db1 ADM_DAT
$ echo "${outputres[1]}" #print second one
db2 SYSAUX
Since you want to use both values separatedly, it may be better to use an associative array:
$ declare -A array=()
$ while read -r _ a b _; do array[$a]=$b; done < file
And then you can loop through the values with:
$ for key in ${!array[#]}; do echo "array[$key] = ${array[$key]}"; done
array[db2] = SYSAUX
array[db1] = ADM_DAT
See a basic example of utilization of these arrays:
#!/bin/bash
declare -A array=([key1]='value1' [key2]='value2')
for key in ${!array[#]}; do
echo "array[$key] = ${array[$key]}"
done
echo ${array[key1]}
echo ${array[key2]}
So maybe this can solve your problem: loop through the file with columns, fetch the 2nd and 3rd and use them twice: firstly the $a to perform a grep in file and then as parameters to cmd_command:
while read -r _ a b _
do
echo "cmd_command $(awk -v patt="$a" '$0~patt {print $1}' file) $a, $b"
done < columns_file
For a sample file file:
$ cat file
hello this is db1
and this is another db2
I got this output (note I am just echoing):
$ while read -r _ a b _; do echo "cmd_command $(awk -v patt="$a" '$0~patt {print $1}' file) $a, $b"; done < a
cmd_command hello db1, ADM_DAT
cmd_command and db2, SYSAUX
Related
I have a basic array at the moment and it works as expected. But what I want to do is put every other value into a seperate array. I just need the 100 and 200 to be in array 1, and 1000 and 2000 to be in array 2. How do I do that?
$ cat scriptx.sh
while read line
do
my_array=("${my_array[#]}" $line)
done
echo ${my_array[#]}
$ ./scriptx.sh
100
1000
200
2000
100 1000 200 2000
For an arbitrary number of arrays n_arrays, you can use a counter of lines read so far, dynamically generate array names, such as array_0, array_1 etc., and append new lines to the right arrays based on counter % n_arrays. Obviously, n_arrays set to 2 gives an answer to the question.
#!/bin/bash
set -euo pipefail
declare -ri n_arrays=2 # number of arrays to use
declare -i i counter=0 # number of lines read so far
for ((i = 0; i < n_arrays; ++i)); do
declare -a "array_${i}" # optional; needed with set -u
done
while IFS= read -r line; do
declare -n my_array="array_$((counter++ % n_arrays))"
my_array+=("$line") # append line to the right array
done
for ((i = 0; i < n_arrays; ++i)); do
declare -n my_array="array_${i}"
echo "${my_array[#]#A}" # print a declare+assign command
done
Something along the lines of:
#!/bin/bash
while IFS= read -r line; do
array1+=("$line")
if IFS= read -r line; then
array2+=("$line")
fi
done
echo "array1: ${array1[*]}"
echo "array2: ${array2[*]}"
You could do something like this:
#!/bin/bash
i=0
while read line; do
if (( i++ % 2 )); then
odd_indexes+=($line)
else
even_indexes+=($line)
fi
done
echo "even_indexes: ${even_indexes[*]}"
echo "odd_indexes: ${odd_indexes[*]}"
Assuming that your input data is:
100
1000
200
2000
This script will output:
even_indexes: 100 200
odd_indexes: 1000 2000
But you could also do something like this if you don't mind reading the input file twice:
#!/bin/bash
mapfile -t odd_indexes < <(sed -n 'n;p' data.txt)
mapfile -t even_indexes < <(sed -n 'p;n' data.txt)
echo "even_indexes: ${even_indexes[*]}"
echo "odd_indexes: ${odd_indexes[*]}"
If you're reading from stdin you would need to write that out to a temporary file and use that file in the above script.
One idea:
$ cat runme
#!/bin/bash
cnt=${1:-2} # 1st (optional) arg states number of arrays; if not supplied default is '2'
for ((i=1;i<=cnt;i++))
do
unset array_"$i" # make sure arrays do not exist
done
n=0
while read -r line
do
(( ndx = n++%cnt +1 )) # calculate array index
declare -n curr_array=array_"${ndx}" # use nameref to dynamically reference correct array
curr_array+=( "${line}" ) # add line to array
done
echo ""
for ((i=1;i<=cnt;i++)) # display contents of our arrays
do
typeset -p array_"$i"
done
Taking for a test drive:
$ runme
100
1000
200
2000
declare -a array_1=([0]="100" [1]="200")
declare -a array_2=([0]="1000" [1]="2000")
$ runme 2
100
1000
200
2000
declare -a array_1=([0]="100" [1]="200")
declare -a array_2=([0]="1000" [1]="2000")
$ runme 3
1 a
2 b
3 c
4 d
5 e
declare -a array_1=([0]="1 a" [1]="4 d")
declare -a array_2=([0]="2 b" [1]="5 e")
declare -a array_3=([0]="3 c")
I have a file like this below:
-bash-4.2$ cat a1.txt
0 10.95.187.87 5444 up 0.333333 primary 0 false 0
1 10.95.187.88 5444 up 0.333333 standby 1 true 0
2 10.95.187.89 5444 up 0.333333 standby 0 false 0
I want to fetch the data from the above file into a 2D array.
Can you please help me with a suitable way to put into an array.
Also post putting we need put a condition to check whether the value in the 4th column is UP or DOWN. If it's UP then OK, if its down then below command needs to be executed.
-bash-4.2$ pcp_attach_node -w -U pcpuser -h localhost -p 9898 0
(The value at the end is getting fetched from the 1st column.
You could try something like that:
while read -r line; do
declare -a array=( $line ) # use IFS
echo "${array[0]}"
echo "${array[1]}" # and so on
if [[ "$array[3]" ]]; then
echo execute command...
fi
done < a1.txt
Or:
while read -r -a array; do
if [[ "$array[3]" ]]; then
echo execute command...
fi
done < a1.txt
This works only if field are space separated (any kind of space).
You could probably mix that with regexp if you need more precise control of the format.
Firstly, I don't think you can have 2D arrays in bash. But you can however store lines into a 1-D array.
Here is a script ,parse1a.sh, to demonstrate emulation of 2D arrays for the type of data you included:
#!/bin/bash
function get_element () {
line=${ARRAY[$1]}
echo $line | awk "{print \$$(($2+1))}" #+1 since awk is one-based
}
function set_element () {
line=${ARRAY[$1]}
declare -a SUBARRAY=($line)
SUBARRAY[$(($2))]=$3
ARRAY[$1]="${SUBARRAY[#]}"
}
ARRAY=()
while IFS='' read -r line || [[ -n "$line" ]]; do
#echo $line
ARRAY+=("$line")
done < "$1"
echo "Full array contents printout:"
printf "%s\n" "${ARRAY[#]}" # Full array contents printout.
for line in "${ARRAY[#]}"; do
#echo $line
if [ "$(echo $line | awk '{print $4}')" == "down" ]; then
echo "Replace this with what to do for down"
else
echo "...and any action for up - if required"
fi
done
echo "Element access of [2,3]:"
echo "get_element 2 3 : "
get_element 2 3
echo "set_element 2 3 left: "
set_element 2 3 left
echo "get_element 2 3 : "
get_element 2 3
echo "Full array contents printout:"
printf "%s\n" "${ARRAY[#]}" # Full array contents printout.
It can be executed by:
./parsea1 a1.txt
Hope this is close to what you are looking for. Note that this code will loose all indenting spaces during manipulation, but a formatted update of the lines could solve that.
I have a command, when I run it, it output a table that looks like;
Id Name File OS Version Annotation
10 MICKEY [MICKEY_01_001] MICKEY/MICKEY.vmx windows8Server64Guest vmx-08
13 DONALD [DONALD_01_001] DONALD/DONALD.vmx windows7Server64Guest vmx-10
2 GOOFY [GOOFY_01_001] GOOFY/GOOFY.vmx windows9Server64Guest vmx-09
I then store the table in an array call TABLE and list the TABLE array, the code looks like this;
readarray -t TABLE <<< "$(command)"
IFS='|'
for i in "${TABLE[#]}"
do
echo $I
done
How do I append to the end of each array element? I want the table to be presented as following;
Id Name File OS Version Annotation
10 MICKEY [MICKEY_01_001] MICKEY/MICKEY.vmx windows8Server64Guest vmx-08 ON
13 DONALD [DONALD_01_001] DONALD/DONALD.vmx windows7Server64Guest vmx-10. OFF
2 GOOFY [GOOFY_01_001] GOOFY/GOOFY.vmx windows9Server64Guest vmx-09. ON
If you want to append ON or OFF in your array
readarray -t TABLE <<< "$(command)"
#IFS='|' why ?
for ((i=1;i<"${#TABLE[#]}";i++))
# start i=1 to preserve header
do
# condition to ON or OFF
[ "${a:=OFF}" = 'ON' ] &&a='OFF'||a='ON'
TABLE["$i"]="${TABLE["$i"]} $a"
done
for i in "${TABLE[#]}"
do
echo "$i"
done
What does the command "$(command)" do? Should we assume, that one line of the output = one string = one element of the array? If so, then this should work for you:
readarray -t TABLE <<< "$(command)"
IFS='|'
for i in "${TABLE[#]}"
do
if <condition_for_on_met>; then
echo "$i ON"
elif <condition_for_off_met>;then
echo "$i OFF"
else
echo "$i"
fi
done
But this is a general answer. You could improve your question by showing us what your input is and how is it processed before it is printed.
I've looked, but have only seen answers to one array being passed in a script.
I want to pass multiple arrays to a bash script that assigns them as individual variables as follows:
./myScript.sh ${array1[#]} ${array2[#]} ${array3[#]}
such that: var1=array1 and var2=array2 and var3=array3
I've tried multiple options, but doing variableName=("$#") combines all arrays together into each variable. I hope to have in my bash script a variable that represents each array.
The shell passes a single argument vector (that is to say, a simple C array of strings) off to a program being run. This is an OS-level limitation: There exists no method to pass structured data between two programs (any two programs, written in any language!) in an argument list, except by encoding that structure in the contents of the members of this array of C strings.
Approach: Length Prefixes
If efficiency is a goal (both in terms of ease-of-parsing and amount of space used out of the ARG_MAX limit on command-line and environment storage), one approach to consider is prefixing each array with an argument describing its length.
By providing length arguments, however, you can indicate which sections of that argument list are supposed to be part of a given array:
./myScript \
"${#array1[#]}" "${array1[#]}" \
"${#array2[#]}" "${array2[#]}" \
"${#array3[#]}" "${array3[#]}"
...then, inside the script, you can use the length arguments to split content back into arrays:
#!/usr/bin/env bash
array1=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
array2=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
array3=( "${#:2:$1}" ); shift "$(( $1 + 1 ))"
declare -p array1 array2 array3
If run as ./myScript 3 a b c 2 X Y 1 z, this has the output:
declare -a array1='([0]="a" [1]="b" [2]="c")'
declare -a array2='([0]="X" [1]="Y")'
declare -a array3='([0]="z")'
Approach: Per-Argument Array Name Prefixes
Incidentally, a practice common in the Python world (particularly with users of the argparse library) is to allow an argument to be passed more than once to amend to a given array. In shell, this would look like:
./myScript \
"${array1[#]/#/--array1=}" \
"${array2[#]/#/--array2=}" \
"${array3[#]/#/--array3=}"
and then the code to parse it might look like:
#!/usr/bin/env bash
declare -a args array1 array2 array3
while (( $# )); do
case $1 in
--array1=*) array1+=( "${1#*=}" );;
--array2=*) array2+=( "${1#*=}" );;
--array3=*) array3+=( "${1#*=}" );;
*) args+=( "$1" );;
esac
shift
done
Thus, if your original value were array1=( one two three ) array2=( aye bee ) array3=( "hello world" ), the calling convention would be:
./myScript --array1=one --array1=two --array1=three \
--array2=aye --array2=bee \
--array3="hello world"
Approach: NUL-Delimited Streams
Another approach is to pass a filename for each array from which a NUL-delimited list of its contents can be read. One chief advantage of this approach is that the size of array contents does not count against ARG_MAX, the OS-enforced command-line length limit. Moreover, with an operating system where such is available, the below does not create real on-disk files but instead creates /dev/fd-style links to FIFOs written to by subshells writing the contents of each array.
./myScript \
<( (( ${#array1[#]} )) && printf '%s\0' "${array1[#]}") \
<( (( ${#array2[#]} )) && printf '%s\0' "${array2[#]}") \
<( (( ${#array3[#]} )) && printf '%s\0' "${array3[#]}")
...and, to read (with bash 4.4 or newer, providing mapfile -d):
#!/usr/bin/env bash
mapfile -d '' array1 <"$1"
mapfile -d '' array2 <"$2"
mapfile -d '' array3 <"$3"
...or, to support older bash releases:
#!/usr/bin/env bash
declare -a array1 array2 array3
while IFS= read -r -d '' entry; do array1+=( "$entry" ); done <"$1"
while IFS= read -r -d '' entry; do array2+=( "$entry" ); done <"$2"
while IFS= read -r -d '' entry; do array3+=( "$entry" ); done <"$3"
Charles Duffy's response works perfectly well, but I would go about it a different way that makes it simpler to initialize var1, var2 and var3 in your script:
./myScript.sh "${#array1[#]} ${#array2[#]} ${#array3[#]}" \
"${array1[#]}" "${array2[#]}" "${array3[#]}"
Then in myScript.sh
#!/bin/bash
declare -ai lens=($1);
declare -a var1=("${#:2:lens[0]}") var2=("${#:2+lens[0]:lens[1]}") var3=("${#:2+lens[0]+lens[1]:lens[2]}");
Edit: Since Charles has simplified his solution, it is probably a better and more clear solution than mine.
Here is a code sample, which shows how to pass 2 arrays to a function. There is nothing more than in previous answers except it provides a full code example.
This is coded in bash 4.4.12, i.e. after bash 4.3 which would require a different coding approach. One array contains the texts to be colorized, and the other array contains the colors to be used for each of the text elements :
function cecho_multitext () {
# usage : cecho_multitext message_array color_array
# what it does : Multiple Colored-echo.
local -n array_msgs=$1
local -n array_colors=$2
# printf '1: %q\n' "${array_msgs[#]}"
# printf '2: %q\n' "${array_colors[#]}"
local i=0
local coloredstring=""
local normalcoloredstring=""
# check array counts
# echo "msg size : "${#array_msgs[#]}
# echo "col size : "${#array_colors[#]}
[[ "${#array_msgs[#]}" -ne "${#array_colors[#]}" ]] && exit 2
# build the colored string
for msg in "${array_msgs[#]}"
do
color=${array_colors[$i]}
coloredstring="$coloredstring $color $msg "
normalcoloredstring="$normalcoloredstring $msg"
# echo -e "coloredstring ($i): $coloredstring"
i=$((i+1))
done
# DEBUG
# echo -e "colored string : $coloredstring"
# echo -e "normal color string : $normal $normalcoloredstring"
# use either echo or printf as follows :
# echo -e "$coloredstring"
printf '%b\n' "${coloredstring}"
return
}
Calling the function :
#!/bin/bash
green='\E[32m'
cyan='\E[36m'
white='\E[37m'
normal=$(tput sgr0)
declare -a text=("one" "two" "three" )
declare -a color=("$white" "$green" "$cyan")
cecho_multitext text color
Job done :-)
I do prefer using base64 to encode and decode arrays like:
encode_array(){
local array=($#)
echo -n "${array[#]}" | base64
}
decode_array(){
echo -n "$#" | base64 -d
}
some_func(){
local arr1=($(decode_array $1))
local arr2=($(decode_array $2))
local arr3=($(decode_array $3))
echo arr1 has ${#arr1[#]} items, the second item is ${arr1[2]}
echo arr2 has ${#arr2[#]} items, the third item is ${arr2[3]}
echo arr3 has ${#arr3[#]} items, the here the contents ${arr3[#]}
}
a1=(ab cd ef)
a2=(gh ij kl nm)
a3=(op ql)
some_func "$(encode_array "${a1[#]}")" "$(encode_array "${a2[#]}")" "$(encode_array "${a3[#]}")"
The output is
arr1 has 3 items, the second item is cd
arr2 has 4 items, the third item is kl
arr3 has 2 items, the here the contents op ql
Anyway, that will not work with values that have tabs or spaces. If required, we need a more elaborated solution. something like:
encode_array()
{
for item in "$#";
do
echo -n "$item" | base64
done | paste -s -d , -
}
decode_array()
{
local IFS=$'\2'
local -a arr=($(echo "$1" | tr , "\n" |
while read encoded_array_item;
do
echo "$encoded_array_item" | base64 -d;
echo "$IFS"
done))
echo "${arr[*]}";
}
test_arrays_step1()
{
local IFS=$'\2'
local -a arr1=($(decode_array $1))
local -a arr2=($(decode_array $2))
local -a arr3=($(decode_array $3))
unset IFS
echo arr1 has ${#arr1[#]} items, the second item is ${arr1[1]}
echo arr2 has ${#arr2[#]} items, the third item is ${arr2[2]}
echo arr3 has ${#arr3[#]} items, the here the contents ${arr3[#]}
}
test_arrays()
{
local a1_2="$(echo -en "c\td")";
local a1=("a b" "$a1_2" "e f");
local a2=(gh ij kl nm);
local a3=(op ql );
a1_size=${#a1[#])};
resp=$(test_arrays_step1 "$(encode_array "${a1[#]}")" "$(encode_array "${a2[#]}")" "$(encode_array "${a3[#]}")");
echo -e "$resp" | grep arr1 | grep "arr1 has $a1_size, the second item is $a1_2" || echo but it should have only $a1_size items, with the second item as $a1_2
echo "$resp"
}
Based on the answers to this question you could try the following.
Define the arrays as variable on the shell:
array1=(1 2 3)
array2=(3 4 5)
array3=(6 7 8)
Have a script like this:
arg1=("${!1}")
arg2=("${!2}")
arg3=("${!3}")
echo "arg1 array=${arg1[#]}"
echo "arg1 #elem=${#arg1[#]}"
echo "arg2 array=${arg2[#]}"
echo "arg2 #elem=${#arg2[#]}"
echo "arg3 array=${arg3[#]}"
echo "arg3 #elem=${#arg3[#]}"
And call it like this:
. ./test.sh "array1[#]" "array2[#]" "array3[#]"
Note that the script will need to be sourced (. or source) so that it is executed in the current shell environment and not a sub shell.
I am writing a shell script as below, that gets a list of files provided by the user in a file and then ftp's to a server and then compares the list of files to what is on the server. The issue I am having is that when I am calling my diff function, the list that is being returned are the files that are unique to both arrays. I want only those that are in unique_array1 but not in unique_array2. In short, a list that shows what files within the list the user provided are not on the ftp server. Please note, that in the list of files provided by the user, each line is a file name, separated by a new line character.My script is as below:
#!/bin/bash
SERVER=ftp://localhost
USER=anonymous
PASS=password
EXT=txt
FILELISTTOCHECK="ftpFileList.txt"
#create a list of files that is on the ftp server
listOfFiles=$(curl $SERVER --user $USER:$PASS 2> /dev/null | awk '{ print $9 }' | grep -E "*.$EXT$")
#read the list of files from the list provided##
#Eg:
# 1.txt
# 2.txt
# 3.txt
#################################################
listOfFilesToCheck=`cat $FILELISTTOCHECK`
unique_array1=$(echo $listOfFiles | sort -u)
unique_array2=$(echo $listOfFilesToCheck | sort -u)
diff(){
awk 'BEGIN{RS=ORS=" "}
{NR==FNR?a[$0]++:a[$0]--}
END{for(k in a)if(a[k])print k}' <(echo -n "${!1}") <(echo -n "${!2}")
}
#Call the diff function above
Array3=($(diff unique_array1[#] unique_array2[#]))
#get what files are in listOfFiles but not in listOfFilesToCheck
echo ${Array3[#]}
Based on this You may try comm command:
Usage: comm [OPTION]... FILE1 FILE2
Compare sorted files FILE1 and FILE2 line by line.
With no options, produce three-column output. Column one contains
lines unique to FILE1, column two contains lines unique to FILE2,
and column three contains lines common to both files.
-1 suppress column 1 (lines unique to FILE1)
-2 suppress column 2 (lines unique to FILE2)
-3 suppress column 3 (lines that appear in both files)
A test program:
#!/bin/bash
declare -a arr1
declare -a arr2
arr1[0]="this"
arr1[1]="is"
arr1[2]="a"
arr1[3]="test"
arr2[0]="test"
arr2[1]="is"
unique_array1=$(printf "%s\n" "${arr1[#]}" | sort -u)
unique_array2=$(printf "%s\n" "${arr2[#]}" | sort -u)
comm -23 <(printf "%s\n" "${unique_array1[#]}") <(printf "%s\n" "${unique_array2[#]}")
Output:
a
this