Is it possible to grep an array in bash? Basically I have two arrays, one array I want to loop through and the other array I want to check to see if each string is in the other array or not.
Rough example:
for CLIENTS in ${LIST[#]}; do
#lost as to best way to grep a array with out putting the values in a file.
grep ${server_client_list[#]}
done
You can use grep -f using process substitution:
grep -Ff <(printf "%s\n" "${LIST[#]}") <(printf "%s\n" "${server_client_list[#]}")
Bash provides substring matching. This eliminates the need to spawn external processes and subshells. Simply use:
for CLIENTS in ${LIST[#]}; do
if [[ "${server_client_list[#]}" =~ "$CLIENTS" ]]; then
echo "CLIENTS ($CLIENTS) is in server_client_list"
fi
done
Note: this works for bash, not sh.
You could use the following function :
function IsInList {
local i
for i in "${#:2}"; do
[ "$i" = "$1" ] && return 0
done
return 1
}
for CLIENT in "${LIST[#]}"; do
IsInList "$CLIENT" "${server_client_list[#]}" && echo "It's in the list!"
done
Note that the quotes around ${LIST[#]} and ${server_client_list[#]} are very important because they guarantee the array expansion to expand the way you want them to, instead of having a bad surprise when a space make its way into one of those array values.
Credits : I took this function from patrik's answer in this post when I needed something like that.
Edit :
If you were looking for something with regexp capabilities, you could also use the following :
function IsInList {
local i
for i in "${#:2}"; do
expr match "$i" "$1" && return 0
done
return 1
}
Related
I am trying to make an if statement where if array1 contains any of the strings in array2 it should print "match" else print "no match"
So far I have the following. Not sure how to complete it. Both loops should break as soon as the first match is found.
#!/bin/bash
array1=(a b c 1 2 3)
array2=(b 1)
for a in "${array1[#]}"
do
for b in "${array2[#]}"
do
if [ "$a" == "$b" ]; then
echo "Match!"
break
fi
done
done
Maybe this isn't even the best way to do it?
This illustrates the desired result
if [ array1 contains strings in array2 ]
then
echo "match"
else
echo "no match"
fi
To check whether array1 contains any entry from array2 you can use grep. This will be way faster and shorter than loops in bash.
The following commands exit with status code 0 if and only if there is a match. Use them as ...
if COMMAND FROM BELOW; then
echo match
else
echo no match
fi
Single-Line Array Entries
The simple version for strings without linebreaks is
printf %s\\n "${array1[#]}" | grep -qFxf <(printf %s\\n "${array2[#]}")
Multiline Array Entries
Sadly there doesn't seem to be a straightforward way to make this work for array entries with linebreaks. GNU grep has the option -z to set the "line" delimiters in the input to null, but apparently no option to do the same for the file provided to -f. Listing the entries from array2 as -e arguments to grep is not working either -- grep -F seems to be unable to match multiline patterns. However, we can use the following hack:
printf %q\\n "${array1[#]}" | grep -qFxf <(printf %q\\n "${array2[#]}")
Here we assume that bash's built-in printf %q always prints a unique single line -- which it currently does. However, future implementations of bash may change this. The documentation help printf only states that the output thas to be correctly quoted for bash.
For a fast solution, you're better off using an external tool that can process the entire array as a whole (such as the grep-based answers). Doing nested loops in pure bash is likely to be slower for any substantial amount of data (where the item-by-item processing in bash is likely to be more expensive than the external process start-up time).
However, if you do need a pure bash solution, I see that your current solution has no way to print out the "no match" scenario. In addition, it may print out "match" multiple times.
To fix that, you can just store the fact that a match has been found, and use that to both:
exit the outer loop early as well as the inner loop; and
print the correct string at the end.
To do this, you can use something like:
#!/bin/bash
# Test data.
array1=(a b c 1 2 3)
array2=(b 1)
# Default to not-found state.
foundMatch=false
for a in "${array1[#]}" ; do
for b in "${array2[#]}" ; do
# Any match switches to found state and exits inner loop.
[[ "$a" == "$b" ]] && foundMatch=true && break
done
# If found, exit outer loop as well.
${foundMatch} && break
done
# Output appropriate message for found/not-found state.
$foundMatch && echo "Match" || echo "No match"
For array elements which does not contain newlines, the grep -qf with printf "%s\n" would be a good option. For comparing arrays with any elements, I ended with this:
cmp -s /dev/null <(comm -z12 <(printf "%s\0" "${array1[#]}" | sort -z) <(printf "%s\0" "${array2[#]}" | sort -z))
The printf "%s\0" "${array[#]}" | sort -z print a sorted list of zero terminated array elements. The comm -z12 then extracts common elements in both lists. The cmp -s /dev/null checks if the output of comm is empty, which will not be empty if any element is in both lists. You could use [ -z "$(comm -z ...)" ] to check if the output of comm would be empty, but bash will complain that the output of a command captured with $(..) contains a null byte, so it's better to cmp -s /dev/null.
I think | is faster then <(), so your if could be:
if ! printf "%s\0" "${array1[#]}" | sort -z |
comm -z12 - <(printf "%s\0" "${array2[#]}" | sort -z) |
cmp -s /dev/null -; then
echo "Some elements are in both array1 and array2"
fi
The following could work:
printf "%s\0" "${array1[#]}" | eval grep -qzFx "$(printf " -e %q" "${array2[#]}")"
But I believe I found a bug in grepv3.1 when matching a newline character with -x flag. If you don't use the newline character, the above line works.
Would you try the following:
array1=(a b c 1 2 3)
array2=(b 1)
declare -A seen # set marks of the elements of array1
for b in "${array2[#]}"; do
(( seen[$b]++ ))
done
for a in "${array1[#]}"; do
(( ${seen[$a]} )) && echo "match" && exit
done
echo "no match"
It may be efficient by avoiding the double loop, although the discussion of efficiency may be meaningless as long as using bash :)
foo()
{
mapfile -t arr <<< "${1}"
for i in "${!arr[#]}"
do
if [ -z "${arr[$i]}" ]
then
unset arr[$i]
fi
done
}
When I pass a variable with some content in it ( a big string basically ), I would like to :
interpret the first word before the first whitespace as a key and everything after the first whitespace becomes the entry for that key in my associative array
skip empty lines or lines with just a newline
example ( empty lines not included for compactness )
google https://www.google.com
yahoo https://www.yahoo.com
microsoft https://www.microsoft.com
the array should look like
[ google ] == https://www.google.com
[ yahoo ] == https://www.yahoo.com
[ microsoft ] == https://www.microsoft.com
I haven't found any good solution in the bash manual for the 2 points, the function foo that you see it's kind of an hack that creates an array and only after that it goes through the entire array and deletes the entries where the string is null .
So point 2 gets a solution, probably an inefficient one, but it works, but point 1 still doesn't have a good solution, and the alternative solution is to just create an array while iterating with read, as far as I know .
Do you know how to improve this ?
mapfile doesn't build associative arrays (although if it could, the simplest solution to #2 would be to simply filter the input with, e.g., grep: mapfile -t arr < <(echo "$1" | grep -v "^$").
Falling back to an explicit loop using read, use the =~ operator to match and skip blank lines.
declare -A arr
while read key value; do
if [[ $value =~ "^\s*$" ]]; then # Or your favorite regex for skipping blank lines
continue
fi
arr["$key"]="$value"
done <<< "$1"
You can also skip blank lines using grep even with the while loop:
declare -A arr
while read key value; do
arr["$key"]="$value"
done < <(echo "$1" | grep '^\s*$')
I know that this particular volume should be coming out as true but it keeps returning as false. Can anyone see what is wrong with it? My guess is it has something to do with the if statement but i have no idea what. the "3D/Gather" are 2 seperate strings but i want it so that if they are together then it returns true. This is the start of what is going to be quite a long script so i want to make it a function that i can call upon later on. Thanks in advance for your help.
#!/bin/bash
session_path=$PWD/testSess/3DBPvol.proc
outPath=""
sess=$session_path
{
ThreeD="false"
while read line; do
IFS=" "
arr=$(echo ${line})
unset IFS
for i in ${arr[#]} ; do
if [[ "$i" =~ "3D" ]] && [[ "$i" =~ "Gather" ]]; then
ThreeD="true"
fi
done
done < "$sess"
echo "Is 3D? $ThreeD"
}
arr=$(echo ${line})
This doesn't create an array, you'd need extra parens:
arr=($(echo ${line}))
But you don't actually need the echo, this should be enough:
arr=(${line})
So you want to see if the file under /proc contains "3D Gather"? You can do that simply with a grep:
#!/bin/bash
session_path=$PWD/testSess/3DBPvol.proc
grep -q '3D Gather' "$session_path" && ThreeD=true || ThreeD=false
echo "Is 3D? $ThreeD"
It seems you don't need to break the line into a BASH array. Consider this rafactored script that does the same job but with a lot less code:
ThreeD="false"
while read line; do
[[ "$line" == *"3D Gather"* ]] && ThreeD="true" && break
done < "$sess"
echo "Is 3D? $ThreeD"
I'm struggling to wrap my head around Bash arrays, in particular I have this function where I need to load an array; What I have written is this:
function list_files() {
for f in *; do
[[ -e $f ]] || continue
done
}
function list_array() {
array=list_files
number=0
for items in "${array[#]}"
do
let "number +=1"
echo -e "\033[1m$number\033[0m) $items"
tput sgr0
let "number -=$(echo "${#array[*]}")"
done
}
The problem here is that the function only works once, however I need to run this several times in the script. I am unsure how to go about doing this. Either I have to empty and reload the array every time the function is invoked, or I have to supply a different array name in the function parameter (list_array myarrayname in stead of just list_array). However I have no idea how to accomplish either of these tasks, or if they are possible/feasible.
Any help would be very welcomed!
A bit unclear what you are trying to achieve; perhaps you can find some inspiration from below:
#!/bin/bash
list_files() {
number=0
for f in *
do
if [[ -f $f ]]
then
number=$((number+=1))
echo $f, $number
fi
done
}
list_files_array() {
array=($1/*)
number=0
for item in ${array[#]}
do
if [[ -f $item ]]
then
number=$((number+=1))
echo $item, $number
fi
done
}
list_files $(pwd)
list_files_array $(pwd)
Bash 4 on Linux ~
I have an array of possible values. I must restrict user input to these values.
Arr=(hello kitty goodbye quick fox)
User supplies value as argument to script:
bash myscript.sh -b var
Currently, I'm trying the following:
function func_exists () {
_var="$1"
for i in ${Arr[#]}
do
if [ "$i" == "$_var" ]
then
echo hooray for "$_var"
return 1
fi
done
return 0
}
func_exists $var
if [ $? -ne 1 ];then
echo "Not a permitted value."
func_help
exit $E_OPTERROR
fi
Seems to work fine, are there better methods for testing user input against an array of allowed values?
UPDATE: I like John K's answer ...can someone clarify the use of $#? I understand that this represents all positional parameters -- so we shift the first param off the stack and $# now represents all remaining params, those being the passed array ...is that correct? I hate blindly using code without understanding ...even if it works!
Your solution is what I'd do. Maybe using a few more shell-isms, such as returning 0 for success and non-0 for failure like UNIX commands do in general.
# Tests if $1 is in the array ($2 $3 $4 ...).
is_in() {
value=$1
shift
for i in "$#"; do
[[ $i == $value ]] && return 0
done
return 1
}
if ! is_in "$var" "${Arr[#]}"; then
echo "Not a permitted value." >&2
func_help
exit $E_OPTERROR
fi
Careful use of double quotes makes sure this will work even if the individual array entries contain spaces, which is allowed. This is a two element array: list=('hello world' 'foo bar').
Another solution. is_in is just a variable:
Arr=(hello kitty goodbye quick fox)
var='quick'
string=" ${Arr[*]} " # array to string, framed with blanks
is_in=1 # false
# try to delete the variable inside the string; true if length differ
[ "$string" != "${string/ ${var} /}" ] && is_in=0
echo -e "$is_in"
function func_exists () {
case "$1"
in
hello)
kitty)
goodbye)
quick)
fox)
return 1;;
*)
return 0;;
esac
}