Counting and manipulating occurrences in text file (Perl) - arrays

Question

I have a tab separated text file that is like
1J L 0.5
1J P 0.4
1J K 0.2
1J L 0.3
1B K 0.7
1B L 0.2
1B P 0.3
1B L 0.6
1B L 0.3
And I want to manipulate it in order to get the following information:
For each element in the 1st column, count how many repeated elements in the second column there are, and do the average of all numbers in the third column for each element of the second column. The desired output can be another tab separated text file, where "Average" is the average number for that element in the 2nd column:
1st K# Average L# Average P# Average
1J 1 0.2 2 0.4 1 0.4
1B 1 0.7 3 0.38 1 0.3
How should I proceed? I thought about doing a Hash of Arrays with key = 1st column, but I don't think this would be too advantageous.
I also thought about creating multiple arrays named #L, #P, #K to count the occurrences of each of these elements, for each element of the 1st column; and other arrays #Ln, #Pn, #Kn that would get all numbers for each of these. In the end, the sum of each number divided by scalar #L would give me the average number.
But my main problem in these is: how can I do all of this processing for each element of the 1st column?
Edit: another possibility (that I am trying right now) is to create an array of all unique elements of the first column. Then, greping each one and do the processing. But there may be easier ways?
Edit2: it may happen that some elements of the second column do not exist for some elements in the first column - problem: division by 0. E.g.:
1J L 0.5
1J P 0.4
1J K 0.2
1J L 0.3
1B K 0.7
1B L 0.2
1B L 0.3 <- note that this is not P as in the example above.
1B L 0.6
1B L 0.3

Answer 1

Here is a way to go:
my $result;
while(<DATA>){
chomp;
my #data = split;
$result->{$data[0]}{$data[1]}{sum} += $data[2];
$result->{$data[0]}{$data[1]}{nbr}++;
}
say "1st\tK#\tavg\tL#\tavg\tP#\tavg";
foreach my $k(keys %$result) {
print "$k\t";
for my $c (qw(K L P)) {
if (exists($result->{$k}{$c}{nbr}) && $result->{$k}{$c}{nbr} != 0) {
printf("%d\t%.2f\t",$result->{$k}{$c}{nbr},$result->{$k}{$c}{sum}/$result->{$k}{$c}{nbr});
} else {
printf("%d\t%.2f\t",0,0);
}
}
print "\n";
}
__DATA__
1J L 0.5
1J P 0.4
1J K 0.2
1J L 0.3
1B K 0.7
1B L 0.2
1B P 0.3
1B L 0.6
1B L 0.3
output:
1st K# avg L# avg P# avg
1B 1 0.70 3 0.37 1 0.30
1J 1 0.20 2 0.40 1 0.40

Answer 2

Untested code:
while (<>) {
chomp;
($x, $y, $z) = split /\t/;
push #{$f{$x}{$y}}, $z; # E.g. $f{'1J'}{'L'}[1] will be 0.3
}
#cols = qw/L P K/;
foreach $x (sort keys %f) {
print "$x\t";
foreach $y (#cols) {
$t = $n = 0;
foreach $z (#{$f{$x}{$y}}) {
$t += $z;
++$n;
}
$avg = $n ? $t / $n : 'N/A';
print "$n\t$avg\t";
}
print "\n";
}

Answer 3

For each of the count and sum I would use a Hash of Hashes where the first column is the key to the outer hash and the second column is the key to the inner hash. So something like:
my (%count, %sum);
while(<>) {
my #F = split / /, $_;
$count{$F[0]}->{$F[1]}++;
$sum{$F[0]}->{$F[1]} += $F[2];
}
for my $key (keys %count) {
print $key;
for my $subkey ("K", "L", "P") {
my $average = defined($count{$key}->{$subkey}) ? $sum{$key}->{$subkey} / $count{$key}->{$subkey} : 0;
...; # and print the result
}
print "\n";
}

Answer 4

I am sorry I did this - really - but here is a "one-liner" (ahem) that I will try to translate into a real script and explain - as an exercise for myself :-) I hope this admittedly artificial example of a one line solution adds something to the more clearly written and scripted examples submitted by the others.
perl -anE '$seen{$F[0]}->{$F[1]}++; $sum{$F[0]}->{$F[1]} += $F[2];}{
for(keys %seen){say " $_:"; for $F1(sort keys $seen{$_}) {
say "$F1","s: $seen{$_}->{$F1} avg:",$sum{$_}->{$F1}/$seen{$_}->{$F1}}}' data.txt
See perlrun(1) for a more detailed explanation of Perl's switches. Essentially, perl -anE starts Perl in "autosplit" mode (-a) and creates a while <> loop to read input (-n) for the code that is executed between the ' ' quotes. The -E turns on all the newest bells and whistles for execution (normally one uses -e). Here's my attempt at explaining what it does.
First, in the while loop this (sort of) "oneliner":
autosplits input into an array (#F ... awkish for "fields" I guess) using a space as the delimitter.
uses the %seen{} trick to count occurrences of matching lines in part of the array. Here it increments the value of the %seen hash key created from column one ($F[0]) of #F each time it sees a line in column two ($F[1]) of #F that repeats
uses a hash %sum or %total to add the values in column three ($F[2]) using the =+ operator. See this perlmonks node for another example.
Then it breaks out of the while <> loop created with -n by using a "butterfly" }{ that acts like an END block allowing a nested for loop to spit everything out. I use $F1 as the subkey for the inner for loop to remind myself that I'm getting it from the second column of the autosplit array #F.
Output (we need printf to get nicer numerical results);
1B:
Ks: 1 avg:0.7
Ls: 3 avg:0.366666666666667
Ps: 1 avg:0.3
1J:
Ks: 1 avg:0.2
Ls: 2 avg:0.4
Ps: 1 avg:0.4
This makes the numbers look nicer (using printf to format)
perl -anE '$seen{$F[0]}->{$F[1]}++; $sum{$F[0]}->{$F[1]} += $F[2];}{
for(keys %seen){say " $_:"; for $F1(sort keys $seen{$_}) {
printf("%ss %d avg: %.2f\n", $F1, $seen{$_}->{$F1}, $sum{$_}->{$F1}/$seen{$_}->{$F1})}}' data.txt
Script version. It increments values of repeated keys drawn from column two ($field[1]) of the data; in a second hash it sums the key values drawn from column three ($field[2]). I wanted to impress you with a more functional style or the exact right CPAN module for the job but had to $work. Cheers and be sure to ask more Perl questions!
#!/usr/bin/env perl
use strict;
use warnings;
my %seen ;
my %sum ;
while(<DATA>){
my #fields = split ;
$seen{$fields[0]}{$fields[1]}++ ;
$sum{$fields[0]}{$fields[1]} += $fields[2];
}
for(keys %seen) {
print " $_:\n";
for my $f(sort keys $seen{$_}) {
printf("%ss %d avg: %.2f\n",
$f, $seen{$_}->{$f}, $sum{$_}->{$f}/$seen{$_}->{$f} );
}
}
__DATA__
1J L 0.5
1J P 0.4
1J K 0.2
1J L 0.3
1B K 0.7
1B L 0.2
1B L 0.3
1B L 0.6
1B L 0.3