Storing matching results from a regular expression in an array - arrays

I am trying to write a simple bash script that would grep for a string in a binary file and then using a regular expression extract all hexadecimal strings. However, I cannot seem to find a way in which I could store the output from the regex in an array which I could later on use to convert all of its elements from hexadecimal to ASCII values.
str=$(grep -ra "Html.Exploit.CVE.2015_6073" /var/lib);
hexstr=$([[ $STR =~ (?<=^|[*{};])[A-Fa-f0-9]+(?=$|[*;{}]) ]]);
converted=$(xxd -r -p <<< "$HEXSTR");
echo -e "\e[92m$converted \e[0m"
Can I do an if statement with a while loop in it that would create an array with the hexadecimal strings (returned from the regex) as elements?
UPDATE
Current code:
#!/bin/bash
str=$(grep -ra "Html.Exploit.CVE.2015_6073" /var/lib);
if [[ $str =~ (?<=^|[*{};])[A-Fa-f0-9]+(?=$|[*;{}]) ]]; then
Array+="$str"
fi
for hex in "${Array[#]}"; do
echo $hex
done

You can add values to an array like this in bash:
Array+=("$hexstr")
Then you can loop through the array like this:
for hex in "${Array[#]}"; do
#perform operation to convert hex here
converted="$(xxd -r -p <<< "$hex")"
done
To correct the if match logic you can make a statement like this so the $hexstr variable will get set.
if [[ $STR =~ (?<=^|[*{};])[A-Fa-f0-9]+(?=$|[*;{}]) ]]; then
hexstr="$STR"
fi
Putting this together would look like this:
if [[ $STR =~ (?<=^|[*{};])[A-Fa-f0-9]+(?=$|[*;{}]) ]]; then
Array+=("$STR")
fi
###Do work
###Use For loop etc
If you want to shorten your code even more (less code less chances to break) you can do all of the matching in the grep and add to the array like this:
Array=($( grep -ra "Html.Exploit.CVE.2015_6073" /var/lib | grep -oP "(?<=^|[*{};])[A-Fa-f0-9]+(?=$|[*;{}])"))
##Do For loop
The above will perform your grep and find only matching lines then it will get only the matching value of the regex and store each in the array.

Related

How to split string to array with specific word in bash

I have a string after I do a command:
[username#hostname ~/script]$ gsql ls | grep "Graph graph_name"
- Graph graph_name(Vertice_1:v, Vertice_2:v, Vertice_3:v, Vertice_4:v, Edge_1:e, Edge_2:e, Edge_3:e, Edge_4:e, Edge_5:e)
Then I do
IFS=", " read -r -a vertices <<< "$(gsql use graph ifgl ls | grep "Graph ifgl(" | cut -d "(" -f2 | cut -d ")" -f1)" to make the string splitted and append to array. But, what I want is to split it by delimiter ", " then append each word that contain ":v" to an array, its mean word that contain ":e" will excluded.
How to do it? without do a looping
Like this, using grep
mapfile -t array < <(gsql ls | grep "Graph graph_name" | grep -oP '\b\w+:v')
The regular expression matches as follows:
Node
Explanation
\b
the boundary between a word char (\w) and something that is not a word char
\w+
word characters (a-z, A-Z, 0-9, _) (1 or more times (matching the most amount possible))
:v
':v'
This bash script should work:
declare arr as array variable
arr=()
# use ", " as delimiter to parse the input fed through process substituion
while read -r -d ', ' val || [[ -n $val ]]; do
val="${val%)}"
val="${val#*\(}"
[[ $val == *:v ]] && arr+=("$val")
done < <(gsql ls | grep "Graph graph_name")
# check array content
declare -p arr
Output:
declare -a arr='([0]="Vertice_1:v" [1]="Vertice_2:v" [2]="Vertice_3:v" [3]="Vertice_4:v")'
Since there is a condition per element the logical way is to use a loop. There may be ways to do it, but here is a solution with a for loop:
#!/bin/bash
input="Vertice_1:v, Vertice_2:v, Vertice_3:v, Vertice_4:v, Edge_1:e, Edge_2:e, Edge_3:e, Edge_4:e, Edge_5:e"
input="${input//,/ }" #replace , with SPACE (bash array uses space as separator)
inputarray=($input)
outputarray=()
for item in "${inputarray[#]}"; do
if [[ $item =~ ":v" ]]; then
outputarray+=($item) #append the item to the output array
fi
done
echo "${outputarray[#]}"
will give output: Vertice_1:v Vertice_2:v Vertice_3:v Vertice_4:v
since the elements don't have space in them this works

Cat strings from file and compare it with associative bash array

I create associative array for example
declare -A vars=(["Test array here"]="POOLING TESTPOOL ")
And i have file with multiple lines which want to compare with array values and if match print array key.
I try this but it's doesn't work
match="POOLING TESTPOOL"
for key in "${vars[#]}"; do [[ $key = "$match" ]] && printf '%s\n' "${!vars[$key]}" ; done
Also I try glob
for key in "${vars[#]}"; do [[ $key = *"$match"* ]] && printf '%s\n' "${!vars[$key]}" ; done
But every time I got empty string
So if you want cat files and compare string with associative array value
and print its key you can use this nested loop
for i in `cat file.txt`; do for key in "${!vars[#]}"; do if [[ ${vars[$key]} == *"$i"* ]]; then echo $key fi done done
Thanks to #Fravadona to point me.

storing user values in an array then comparing these variables using bash

while read line
do
if [ $line -ge $zero ]
then
a+=($line) ##here I am attempting to store the values in an array
else
for i in ${a[#]}
do
echo $(a[$i]) ## here I am trying to print them
done
fi
what is going wrong? it is giving this error:
a[1]: command not found
a[2]: command not found
a[3]: command not found
done
From the begining
if [ $line -ge $zero ]
what data type should be in $line?
-ge used in numeric comparison. If $line is a string than use = or if you just wnt to check that it's not empty use this syntax if [[ "$line" ]]
Next.
a+=($line)
Again if $line is a string then you should wrap in "" like this a+=("$line")
coz line can contain spaces.
For loop.
for i in ${a[#]}
do
echo $(a[$i]) ## here I am trying to print them
You'r messing with syntax here for i in ${a[#]} will iterate over arrays values, not indexes. And again if values are strings use "" like this for i in "${a[#]}"
So this echo $(a[$i]) won't work by 2 reasons. First you should use {} here, like this echo ${a[$i]}, second $i is not index, but it's may actualy work if it's a digit but in a wrong way. So here you need just echo $i coz $i is alredy a value from a array. Or rewrite foor loop.
for i in ${!a[#]}
do
echo ${a[$i]} ## here I am trying to print them
done
And last but not least, there is no done at the end of this script or it's just a part? So in the and it should look like this.
while read line; do
if [[ $line ]]; then
a+=( "$line" ) ##here I am attempting to store the values in an array
else
for i in ${!a[#]}; do
echo ${a[$i]} ## here I am trying to print them
done
fi
done < data.txt
echo ${a[#]} # print rusult

How to replace a word from a file with elements in an array

I am trying to replace all the word "null" to elements in array. The problem is that after replacing one word of "null", I would like to replace the next "null" with next element in the array.
I am not very good with bash and I feel like this is quite a basic question.
Here is what I have so far:
for m in $(cat finalfile.csv)
do
if [ "$m" = "null" ]
then
m=cwearray[$counter]
let counter++
fi
done
This doesn't replace anything in the finalfile.csv.
For example if the file has:
"value1","value2","null","value3"\n
"value1","value2","null","value3"...
and the array has ["foo","bar"]
I would like it to be:
"value1","value2","foo","value3"\n
"value1","value2","bar","value3"...
can be done with bash, even with multiple nulls per line:
$ cat finalfile.csv
"value1","value2","null","null"
"value1","value2","null","value3"
$ cwearray=( foo bar baz )
$ idx=0
$ while read -r line; do
while [[ $line == *null* ]]; do
line=${line/null/${cwearray[idx++]}}
# ...............^^^^^^^^^^^^^^^^^^
# replace the _first_ "null" with the _next_ array element
done
echo "$line"
done < finalfile.csv > updatedfinalfile.csv
$ cat updatedfinalfile.csv
"value1","value2","foo","bar"
"value1","value2","baz","value3"
It's easier in Perl where you can increase the index directly in the replacement part of a substitution:
printf '%s\n' 1,2,3,null null,2,3,4 null,null,null,null \
| perl -pe 'BEGIN { #cwe = qw( A B C D E F ) }
s/(?:^|(?<=,))null(?=,|$)/$cwe[$i++]/g'
Update: It seems you've updated your question with a sample input. If nulls are double quoted, it gets even easier, as there's no need to check whether they're surrounded with commas or beginning/end of the line.
perl -pe 'BEGIN{ #cwe = qw( foo bar ) }
s/"null"/"$cwe[$i++]"/g'
An awk solution :
declare -a cwearray
cwearray=(foo bar)
awk -F, 'NR==FNR{repl[NR]=$0; next}{for(i=1;i<=NF;i++){if($i=="\"null\""){$i="\""repl[++counter]"\""}}}1' OFS="," <(for i in "${cwearray[#]}"; do echo "$i"; done) <file>
Read the file line by line. If a line contains null, then use sed to replace all occurrences of null with the corresponding value, retrieved via array index.
#!/bin/bash
file="finalfile.csv"
counter=0
array=(
"foo"
"bar"
)
while read -r line; do
item="${array[$counter]}"
echo "$line" | sed "s/null/$item/g"
((counter++))
done < "$file"

echo a variable and then grep to see if value exist in a file is not returning anything. Unix Shell Scripting

I'm trying to figure out how to determine if a variable contains a value from a file using grep, this is not returning anything, so I'm going to explain it.
I have my code that is this:
MyFiles="MyFile-I-20160606_141_Employees.txt"
DirFiles="/dev/fs/C/Users/salasfri/Desktop/MyFiles.txt"
for OutFile in $(cat $DirFiles); do
if [[ $( echo $MyFiles | grep -c $OutFile ) -gt 0 ]]; then
print "The file $OutFile exist!!"
fi
done
and the file in /dev/fs/C/Users/salasfri/Desktop/MyFiles.txt contains the following values:
MyFile-I-*_141_Employees.txt
MyFile-I-*_141_Products.txt
MyFile-I-*_141_Deparments.txt
the idea is verify if the variable "MyFiles" is found in the MyFiles.txt file, as you can see is using the pattern "*" due that is a date, it will change.
that solutions is not returning any count of files, there's something that I'm doing wrong?
You can try to change the searchstring before searching.
An example with three teststrings:
for teststring in MyFile-I-20160606_141_Employees.txt MyFile-I-20160606_142_Employees.txt MyFile-I-20160606_141_Others.txt
do
grepstr=$(sed 's/[0-9]\{8\}_/*_/' <<< "${teststring}")
fgrep "${grepstr}" "${DirFiles}"
found=$(fgrep "${grepstr}" "${DirFiles}")
if [ $? -eq 0 ]; then
echo "${found} matches ${teststring}."
fi
done
In your case you can make the code shorter with
fgrep -q "$(sed 's/[0-9]\{8\}_/*_/' <<< "${MyFiles}")" $DirFiles &&
echo "The file $(sed 's/[0-9]\{8\}_/*_/' <<< "${MyFiles}") exist!!"
Your patterns are glob-style patterns, not regular expressions. The pattern abc-*_X.txt will not match the string abc-1234_X.txt.
You want to use a shell construct that does glob matching.
MyFiles="MyFile-I-20160606_141_Employees.txt"
sed 's/\r$//' "/dev/fs/C/Users/salasfri/Desktop/MyFiles.txt" \
| while IFS= read -r Pattern; do
if [[ $MyFiles == $Pattern ]]; then
print "$MyFiles matches pattern $Pattern"
break
fi
done

Resources