i have a a files containing usernames and users sent count mail per line . for example (dont know how many line have ) :
info.txt >
500 example1
40 example2
20 example3
....
..
.
if the number was greater than X , i want to run commands containing the user name and act on user .
getArray() {
users=() # Create array
while IFS= read -r line # Read a line
do
users+=("$line") # Append line to the array
done < "$1"
}
getArray "/root/.myscripts/spam1/info.txt"
# i know this part is incorrect and need help here :
if [ "${users[1$]}" -gt "50" ]
then
echo "${users[2$] has sent ${users[1$]} emails"
fi
please Help
Thanks
Not knowing how many lines of input you have is no reason to use an array. Indeed, it is generally more useful if you assume your input is infinite (an input stream), so reading into an array is impossible. Just read each line and take action if necessary:
#!/bin/sh
while read -r count user; do
if test "$count" -gt 50; then
echo "$user has sent $count emails"
fi
done < /root/.myscripts/spam1/info.txt
Related
I am new to shell and usually using single line commands. However, I would like to learn more in bash but got stuck with this problem of looping.
I have two file- A and B. I would like to paste A (one line at a time) to file B when it find pattern ("Hit") and so on. Here is the example.
file A:
contig543
conting432
conting32
file B:
Hit E-value
WP_0134620.1 1.63E-167
WP_0134619.1 7.31E-126
Hit E-value
WP_0134644.1 0
WP_0134625.1 2.53E-108
Hit E-value
WP_0134620.1 1.63E-167
WP_0134619.1 7.31E-126
Output file:
contig543
Hit E-value
WP_0134620.1 1.63E-167
WP_0134619.1 7.31E-126
contig432
Hit E-value
WP_0134620.1 1.63E-167
WP_0134619.1 7.31E-126
contig32
Hit E-value
WP_0134620.1 1.63E-167
WP_0134619.1 7.31E-126
here is my code:
#!/bin/zsh
fileA='contig1'
n=1
while read line; do
# reading each line
echo "$line"
fileB='F3.txt'
x=1
while read line1; do
echo "$line1"
if [["$line1" = ^"Hit"]]; then
echo "$line1"
fi
done < $fileB
done < $fileA
n=$((n+1))
Start with shellcheck.net for syntax errors.
(I'm using bash - I need to compare zsh syntax...)
The root of your problem is here: [["$line1" = ^"Hit"]]
You need spaces inside the [[ and ]], and a tilde on your operator. I'd also remove the quotes on your pattern here; they technically aren't a problem, but they are a complication you don't need in this case. The result:
[[ "$line1" =~ ^Hit ]]
The other problem is that this is going to read and print out ALL of F3.txt for every entry in contig1, echoing the contig1 value at the top each time, and then just double-printing the Hit lines. You need to interleave and coordinate your data, so it needs more logic. Likewise, you aren't using x and n; why are they there?
Fortunately, there are simpler and more efficient solutions.
A quick hack, focusing on just fixing the loop logic:
set -- $(<A) # A to arg array
while read b # read each line of B (c.f. "done<B")
do case "$b" in # string match the line
Hit*) printf "%s\n\n%s\n" $1 "$b" # print formatted on match
shift ;; # remove the used value from A
*) echo "$b" ;; # for anything else, just print it
esac
done<B # set input on the loop
This makes a lot of unchecked assumptions, such as that you don't need $# for something else, that there are an appropriate number of record in A for groups in B, that values in A have no inappropriate characters such as whilespaces that would throw off the array load (note I didn't quote $1 either), that both A & B have newlines at the end of the last line, and a good bit more... but depending on your situation, those might be reasonably safe assumptions. If they are not, please edit accordingly.
It avoids the nested loop which reads all of B for every value in A, though.
This loads A into a sequential queue and shifts off the next element to collate into your output of B each time you "Hit" the trigger string.
If you want a slightly less quick & dirty version that still reads through each file only once, accounts for possible whitespace in the keys of A, does less haphazard reads, and uses pattern matching as your were doing:
aLst=()
while read -r a || # by-line raw read of A
[[ -n "$a" ]] # allowing for no newline on last record
do aLst+=("$a") # quoted assignment into named indexed array
done<A
while read -r b || [[ -n "$b" ]] # safer read like A
do if [[ "$b" =~ ^Hit ]] # check for trigger string
then printf "%s\n\n%s\n" "${aLst[0]}" "$b" # quoted element access
aLst=( "${aLst[#]:1}" ) # "shift" for arrays
else echo "$b" # just print nonmatches
fi
done<B
Output of both:
contig543
Hit E-value
WP_0134620.1 1.63E-167
WP_0134619.1 7.31E-126
conting432
Hit E-value
WP_0134644.1 0
WP_0134625.1 2.53E-108
conting32
Hit E-value
WP_0134620.1 1.63E-167
WP_0134619.1 7.31E-126
Hope that helps. :)
I have wasted so much of my time with this, I hope someone can help me. I am editing a script that is used to send values to an executable and then scrubs the output from the executable for tabulation. I have created two arrays that are filled with user inputted ranges, these ranges are then used to create a nested loop that I have put into a function (since I need to create 4 files from the output depending on another value). I admit that my code is abysmal, but it does the primary thing and that is obtaining the data I need and puts into the correct file. All I am trying to do is to get it to actually make a table with row and column labels, I just can't understand why this is so difficult.
This is the problematic area:
# Function to loop user inputted ranges in order to generate report data.
function repeat()
{
printf "%22s" 'density (10^18 m^-3)'
for a in "${density_array[#]}" # for loop to list density values in the range set by the user.
do
printf "%13s" "$a"
done
echo -e "\n" 'speed (m/s)'
#printf "%s\n" "${speed_array[#]}"
for i in "${speed_array[#]}"
do
echo "$i"
for j in "${density_array[#]}"
do
echo $j > SCATINPUT # generates a temporary file named SCATINPUT, with density value as first line.
echo $temp >> SCATINPUT # appends a new line with temperature value to SCATINPUT file.
echo $i >> SCATINPUT # appends a new line with speed value to SCATINPUT file.
echo $1 >> SCATINPUT # appends a new line with rate type from argument to SCATINPUT file.
# pipes contents of SCATINPUT file to executable, extracts value from STDOUT to RATE variable.
RATE=`path_of_executable < SCATINPUT | awk '/0\./'`
RATEF=$(printf "%.4e" $RATE) # converts number in RATE variable to scientific notation with 4 digits after decimal and sets RATEF variable.
echo -ne "\t$RATEF"
rm -f SCATINPUT # quietly deletes SCATINPUT file.
done
done
}
I am getting this output in a file:
density (10^18 m^-3) 2.0000e+00 4.0000e+00 6.0000e+00
speed (m/s)
8.0000e+06
7.6164e+04 1.4849e+05 2.1936e+059.0000e+06
5.7701e+04 1.1249e+05 1.6619e+051.0000e+07
4.3469e+04 8.4747e+04 1.2520e+051.1000e+07
3.3078e+04 6.4488e+04 9.5269e+041.2000e+07
2.5588e+04 4.9886e+04 7.3697e+04
But it should be this:
density (10^18 m^-3) 2.0000e+00 4.0000e+00 6.0000e+00
speed (m/s)
8.0000e+06 7.6164e+04 1.4849e+05 2.1936e+05
9.0000e+06 5.7701e+04 1.1249e+05 1.6619e+05
1.0000e+07 4.3469e+04 8.4747e+04 1.2520e+05
1.1000e+07 3.3078e+04 6.4488e+04 9.5269e+04
1.2000e+07 2.5588e+04 4.9886e+04 7.3697e+04
The general idea would be to replace the echo commands with comparable printf commands with formats that match the formats used to print the first/header line ...
Start by replacing echo "$i" with printf "%22s" "$i" => this should leave the cursor on the same line as $i and lined up under the 2.0000e+00
After you finish the j loop and before getting the next i do printf "\n" => this should move the cursor to the next line and ready for the next printf "%22s" "$i".
That should get you started.
If things don't line up quite right then consider replacing the echo -ne "\tRATEF" with a printf "%#s" "$RATEF" (adjust the number # to line up output as desired).
I have a grep output and I'm trying to make an associative array from the output that I get.
Here is my grep output:
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
I want to use that output to define an associative array, like this:
array[123456789101]=devid1234
array[11213141516]=devid5678
Is that possible? I'm new at making arrays. I hope someone could help me in my problem.
Either pipe your grep output to a helper script with a while loop containing a simple "0/1" toggle to read two lines taking the last field of each to fill your array, e.g.
#!/bin/bash
declare -A array
declare -i n=0
arridx=
while read -r label value; do # read 2 fields
if [ "$n" -eq 0 ]
then
arridx="${value:1}" # strip 1st and lst 2 chars
arridx="${arridx:0:(-2)}" # save in arridx (array index)
((n++)) # increment toggle
else
arrval="${value:1}" # strip 1st and lst 2 chars
arrval="${arrval:0:(-2)}" # save in arrval (array value)
array[$arridx]="$arrval" # assign to associative array
n=0 # zero toggle
fi
done
for i in ${!array[#]}; do # output array
echo "array[$i] ${array[$i]}"
done
Or you can use process substitution containing the grep command within the script to do the same thing, e.g.
done < <( your grep command )
You can also add a check under the else clause that if [[ $label =~ DeviceId ]] to validate you are on the right line and catch any variation in the grep output content.
Example Input
$ cat dat/grepout.txt
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
Example Use/Output
$ cat dat/grepout.txt | bash parsegrep2array.sh
array[123456789101] devid1234
array[111213141516] devid5678
Parsing out the values is easy, and once you have them you can certainly use those values to build up an array. The trickiest part comes from the fact that you need to combine input from separate lines. Here is one approach; note that this script is verbose on purpose, to show what's going on; once you see what's happening, you can eliminate most of the output:
so.input
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
so.sh
#!/bin/bash
declare -a hardwareInfo
while [[ 1 ]]; do
# read in two lines of input
# if either line is the last one, we don't have enough input to proceed
read lineA < "${1:-/dev/stdin}"
# if EOF or empty line, exit
if [[ "$lineA" == "" ]]; then break; fi
read lineB < "${1:-/dev/stdin}"
# if EOF or empty line, exit
if [[ "$lineB" == "" ]]; then break; fi
echo "$lineA"
echo "$lineB"
hwsn=$lineA
hwsn=${hwsn//HardwareSerialNumber/}
hwsn=${hwsn//\"/}
hwsn=${hwsn//:/}
hwsn=${hwsn//,/}
echo $hwsn
# some checking could be done here to test that the value is numeric
devid=$lineB
devid=${devid//DeviceId/}
devid=${devid//\"/}
devid=${devid//:/}
devid=${devid//,/}
echo $devid
# some checking could be done here to make sure the value is valid
# populate the array
hardwareInfo[$hwsn]=$devid
done
# spacer, for readability of the output
echo
# display the array; in your script, you would do something different and useful
for key in "${!hardwareInfo[#]}"; do echo $key --- ${hardwareInfo[$key]}; done
cat so.input | ./so.sh
"HardwareSerialNumber": "123456789101",
"DeviceId": "devid1234",
123456789101
devid1234
"HardwareSerialNumber": "111213141516",
"DeviceId": "devid5678",
111213141516
devid5678
111213141516 --- devid5678
123456789101 --- devid1234
I created the input file so.input just for convenience. You would probably pipe your grep output into the bash script, like so:
grep-command | ./so.sh
EDIT #1: There are lots of choices for parsing out the key and value from the strings fed in by grep; the answer from #David C. Rankin shows another way. The best way depends on what you can rely on about the content and structure of the grep output.
There are also several choices for reading two separate lines that are related to each other; David's "toggle" approach is also good, and commonly used; I considered it myself, before going with "read two lines and stop if either is blank".
EDIT #2: I see declare -A in David's answer and in examples on the web; I used declare -a because that's what my version of bash wants (I'm using a Mac). So, just be aware that there can be differences.
I know that reading a .csv file can be done simply in bash with this loop:
#!/bin/bash
INPUT=data.cvs
OLDIFS=$IFS
IFS=,
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read flname dob ssn tel status
do
echo "Name : $flname"
echo "DOB : $dob"
echo "SSN : $ssn"
echo "Telephone : $tel"
echo "Status : $status"
done < $INPUT
IFS=$OLDIFS
But I want to slightly modify this- I want to make the columns be defined by the programmer in the bash file.
For example:
declare -a columns=("Name", "Surname", "ID", "Gender")
while read columns
do
//now echo everything that has been read
done < $INPUT
So I want to specify the list of variables that should be used as the container to the read CSV data with an array and then access this array inside the while body.
Is there a way to do it?
The key to this solution is the comment before the while statement below. read is a built-in, but it is still a command, and command arguments are expanded by the shell before executing the command. After expansion of ${columns[#]}, the command becomes
read Name Surname ID Gender
Example:
# Don't use commas in between array values (since they become part of the value)
# Values not quoted because valid names don't need quotes, and these
# value must be valid names
declare -a columns=(Name Surname ID Gender)
Then, we can try:
# Read is a command. Arguments are expanded.
# The quotes are unnecessary but it's hard to break habits :)
while read "${columns[#]}"; do
echo Name is "$Name"
# etc
done <<< "John Doe 27 M"
Output:
Name is John
This same approach would work even in a shell without arrays; the column names can just be a space separated list. (Example run in dash, a Posix shell)
$ columns="Name Surname ID Gender"
$ # Here it is vital that $columns not be quoted; we rely on word-splitting
$ while read $columns; do
> echo Name is $Name
> done
John Doe 27 M
Name is John
...
Read the line into an array, then loop through that array and create an associative array that uses the column names.
while read -r line
do
vals=($line)
declare -A colmap
i=0
for col in ${columns[#]}
do
colmap[col]=${vals[$i]}
let i=i+1
done
# do stuff with colmap here
# ...
unset colmap # Clear colmap before next iteration
done < $INPUT
I read the files of a directory and put each file name into an array (SEARCH)
Then I use a loop to go through each file name in the array (SEARCH) and open them up with a while read line loop and read each line into another array (filecount). My problem is its one huge array with 39 lines (each file has 13 lines) and I need it to be 3 seperate arrays, where
filecount1[line1] is the first line from the 1st file and so on. here is my code so far...
typeset -A files
for file in ${SEARCH[#]}; do
while read line; do
files["$file"]+="$line"
done < "$file"
done
So, Thanks Ivan for this example! However I'm not sure I follow how this puts it into a seperate array because with this example wouldnt all the arrays still be named "files"?
If you're just trying to store the file contents into an array:
declare -A contents
for file in "${!SEARCH[#]}"; do
contents["$file"]=$(< $file)
done
If you want to store the individual lines in a array, you can create a pseudo-multi-dimensional array:
declare -A contents
for file in "${!SEARCH[#]}"; do
NR=1
while read -r line; do
contents["$file,$NR"]=$line
(( NR++ ))
done < "$file"
done
for key in "${!contents[#]}"; do
printf "%s\t%s\n" "$key" "${contents["$key"]}"
done
line 6 is
$filecount[$linenum]}="$line"
Seems it is missing a {, right after the $.
Should be:
${filecount[$linenum]}="$line"
If the above is true, then it is trying to run the output as a command.
Line 6 is (after "fixing" it above):
${filecount[$linenum]}="$line"
However ${filecount[$linenum]} is a value and you can't have an assignment on a value.
Should be:
filecount[$linenum]="$line"
Now I'm confused, as in whether the { is actually missing, or } is the actual typo :S :P
btw, bash supports this syntax too
filecount=$((filecount++)) # no need for $ inside ((..)) and use of increment operator ++
This should work:
typeset -A files
for file in ${SEARCH[#]}; do # foreach file
while read line; do # read each line
files["$file"]+="$line" # and place it in a new array
done < "$file" # reading each line from the current file
done
a small test shows it works
# set up
mkdir -p /tmp/test && cd $_
echo "abc" > a
echo "foo" > b
echo "bar" > c
# read files into arrays
typeset -A files
for file in *; do
while read line; do
files["$file"]+="$line"
done < "$file"
done
# print arrays
for file in *; do
echo ${files["$file"]}
done
# same as:
echo ${files[a]} # prints: abc
echo ${files[b]} # prints: foo
echo ${files[c]} # prints: bar