HoHOA behaves differently when values are pushed - arrays

Question

I have the following data:
eya XLOC_000445_Change:10.3_q:0.003 atonal1
six XLOC_00099_Change:70.0_q:0.095 atonal1
six-eya XLOC_0234324_Change:19.8_q:0.05 atonal1
eya XLOC_00010_Change:6.5_q:0.22 c-myc
six XLOC_025437_Change:1.1_q:0.018 c-myc
six-eya XLOC_001045_Change:2.3_q:0.0001 c-myc
eya XLOC_000115_Change:7.3_q:0.03 ezrin
six XLOC_000001_Change:7.9_q:0.00006 ezrin
six-eya XLOC_0234322_Change:9.0_q:0.0225 ezrin
six-eya XLOC_091345_Change:9.3_q:0.005 slc12a2
eya XLOC_000445_Change:9.9_q:0.3 atonal1
six XLOC_00099_Change:7.0_q:0.95 atonal1
six-eya XLOC_0234324_Change:9.8_q:0.5 atonal1
And have tried building a HoHoA as follows:
#!/usr/bin/perl
use warnings;
use strict;
Method 1: Pushing array values onto HoH:
while (<$input>) {
chomp;
push #xloc, $1 if ($_ =~ /(XLOC_\d+)/);
push #change_val, $1 if ($_ =~ /Change:(-?\d+\.\d+|-?inf)/);
push #q_value, $1 if ($_ =~ /q:(\d+\.\d+)/);
my #split = split('\t');
push #condition, $split[0];
push #gene, $split[2];
}
push #{ $experiment{$gene[$_]}{$condition[$_]} }, [ $xloc[$_], $change_val[$_], $q_value[$_] ] for 0 .. $#change_val;
Method 2: Assigning values to HoHoA on the fly:
while (<$input>) {
chomp;
my $xloc = $1 if ($_ =~ /(XLOC_\d+)/);
my $change = $1 if ($_ =~ /Change:(-?\d+\.\d+|-?inf)/);
my $q_value = $1 if ($_ =~ /q:(\d+\.\d+)/);
my #split = split('\t');
my $condition = $split[0];
my $gene = $split[2];
$experiment{$gene}{$condition} = [ $xloc, $change, $q_value ];
}
Both work fine - insofar as I get the data structure I want. However, only the first method (pushing) ensures that genes that exist as duplicates (in this case atonal1) are represented twice in the HoHoA.
My downstream code was originally made to handle HoHoA built in the second fashion, and I can't for the life of me work out why both approaches are handled differently in the follwing code:
Downstream code:
my (%change, %seen, $xloc, $change_val, $q_value);
for my $gene (sort keys %experiment) {
for my $condition (sort keys %{$experiment{$gene}}) {
$seen{$gene}++; # Counts for each occurrence of gene
if ( (not exists $change{$gene}) || (abs $change{$gene} < abs $experiment{$gene}{$condition}[1]) ) { # Has a larger change value
$change{$gene} = $experiment{$gene}{$condition}[1];
}
}
}
print Dumper \%change;
When I run the above code on either approach I get:
Output for method 1:
$VAR1 = {
'atonal1' => [
'XLOC_0234324',
'9.8',
'0.5'
],
'c-myc' => undef,
'ezrin' => undef,
'slc12a2' => undef,
};
Output for method 2:
$VAR1 = {
'atonal1' => '9.9', # i.e. the largest change value for each condition/gene
'c-myc' => '6.5',
'ezrin' => '9.0',
'slc12a2' => '9.3',
};
What I want is:
$VAR1 = {
'atonal1' => [
'9.9',
'70.0' # This is the difference - i.e the both values are added to the hash `%change`
],
'c-myc' => '6.5',
'ezrin' => '9.0',
'slc12a2' => '9.3',
};
I have no idea what's creating the difference
UPDATE
I'll post the Dumper output for %experiment after values have been pushed on using Method 1:
$VAR1 = {
'atonal1' => {
'eya' => [
[
'XLOC_000445',
'10.3',
'0.003'
],
[
'XLOC_000445',
'9.9',
'0.3'
]
],
'six' => [
[
'XLOC_00099',
'70.0',
'0.095'
],
[
'XLOC_00099',
'7.0',
'0.95'
]
],
'six-eya' => [
[
'XLOC_0234324',
'19.8',
'0.05'
],
[
'XLOC_0234324',
'9.8',
'0.5'
]
]
},
'c-myc' => {
'eya' => [
[
'XLOC_00010',
'6.5',
'0.22'
]
],
'six' => [
[
'XLOC_025437',
'1.1',
'0.018'
]
],
'six-eya' => [
[
'XLOC_001045',
'2.3',
'0.0001'
]
]
},
'ezrin' => {
'eya' => [
[
'XLOC_000115',
'7.3',
'0.03'
]
],
'six' => [
[
'XLOC_000001',
'7.9',
'0.00006'
]
],
'six-eya' => [
[
'XLOC_0234322',
'9.0',
'0.0225'
]
]
},
'slc12a2' => {
'six-eya' => [
[
'XLOC_091345',
'9.3',
'0.005'
]
]
},
};

Answer 1

Let's take your data and reformat it a bit. I'm not saying this is the way you need to format your data. I'm just doing it this way to get a better understanding of what it represents:
GENE XLOC CHANGE Q VALUE CONDITION
======== ==================== ======= ======== ==========
eya XLOC_000445_Change: 10.3_q: 0.003 atonal1
six XLOC_00099_Change: 70.0_q: 0.095 atonal1
six-eya XLOC_0234324_Change: 19.8_q: 0.05 atonal1
eya XLOC_00010_Change: 6.5_q: 0.22 c-myc
six XLOC_025437_Change: 1.1_q: 0.018 c-myc
six-eya XLOC_001045_Change: 2.3_q: 0.0001 c-myc
eya XLOC_000115_Change: 7.3_q: 0.03 ezrin
six XLOC_000001_Change: 7.9_q: 0.00006 ezrin
six-eya XLOC_0234322_Change: 9.0_q: 0.0225 ezrin
six-eya XLOC_091345_Change: 9.3_q: 0.005 slc12a2
eya XLOC_000445_Change: 9.9_q: 0.3 atonal1
six XLOC_00099_Change: 7.0_q: 0.95 atonal1
six-eya XLOC_0234324_Change: 9.8_q: 0.5 atonal1
Are my column assumptions correct?
First, I recommend that you use split to split up your values instead of regular expressions. Do this on a line-by-line basis. Perl is pretty efficient at optimization. 90% of programming is debugging and supporting your program. Trying to be efficient by compressing multiple steps into a single step just make things harder to understand with very little value returned in optimization.
Let's take each line, and break it out:
while ( my $line = <$input> ) {
chomp $line;
my ( $gene, $more_data, $condition ) = split /\s+/, $line;
At this point:
$gene = 'eye'
$more_data = 'XLOC_000445_Change:10.3_q:0.003';
$condition = 'atonal1` # I'm not sure what this is...
Now, we can split out $more_data:
my ( $xloc, $change, $q_value ) = split /:/, $more_data;
$xloc =~ s/^XLOC_//;
$change =~ s/_q$//;
Now we have:
$xloc = '000445';
$change = '10.3';
$q_value = '0.003';
Does this make more sense?
One of your problems is you're attempting to store data in a very, very complex structure without really thinking about what that data represents.
Let's say your data is this:
a gene may contain multiple conditions,
Each gene-condition combination can have a result.
This result contains an xloc, q_value, and change.
That means your data should look like this:
$experiment{$gene}->{$condition}->{XLOC} = $xloc;
$experiment{$gene}->{$condition}->{Q_VALUE} = $q_value;
$experiment{$gene}->{$condition}->{CHANGE} = $change;
However, I see gene = eya, condition = atonal1 twice in your list. Maybe you need something more along the lines of this:
a gene may contain multiple conditions,
Each gene-condition combination can have multiple results.
Each result contains an xloc, q_value, and change.
If that's the case, your data structure should look something like this:
$experment{$gene}->{$condition}->[0]->{XLOC} = $xloc;
$experment{$gene}->{$condition}->[0]->{Q_VALUE} = $q_value;
$experment{$gene}->{$condition}->[0]->{CHANGE} = $change;
This isn't an answer. I'm just trying to get a handle on what your data is and what you are trying to store in that data. Once we have that settled, I can help you with the rest of your program.
Let me know if my understanding of what your data represents is accurate. Add a comment to this answer, and update your question a bit.
Once I know I'm on the right track, I'll show you how you can more easily manage this structure and keep track of everything in it.
Now that I knew I was on the right track, the solution was fairly simple: Object Oriented Programming!
Let me explain: Each experiment consists of a Gene-Condition pair. This is what I key my experiments on.
I create a Local::Condition for each of these Gene-Condition pairs. Inside of this I store my array of results.
My results contain three item.
Behold: The Answer!
What I decided to do is to create a results object. This object contains the XLoc, Change, and Q Value of that result. By packing my results into an object, I have fewer issues trying to keep track of it.
So what we have is this:
We have experiments.
Each _experiment consists of a gene/condition pair which we key our experiments on.
Each gene/condition experiment consists of an array of results.
Now, it's a lot easier to keep track of what is going on. For each line, I create a Local::Result type of object that contains the set of results of that gene/condition pair.
So, all I have to do is push my results onto that gene/condition array which represents my set of results
#
# Create a Result for this experiment
#
my $result = Local::Result->new( $xloc, $change, $q_value );
#
# Push this result onto your $gene/condition experiment
#
push #{ $experiments{$gene}->{$condition} }, $result;
Note my syntax here is very spelled out. I have a hash called %experiments that are keyed by Genes. Each gene contains Conditions for that gene. This Gene/Condition pair is an array of results.
Object Oriented syntax can be a bit complex to get use to, but there is an excellent tutorial in the Perl documentation. By using object oriented programming, you group together details that you otherwise must track.
#! /usr/bin/env perl
use strict;
use warnings;
use feature qw(say);
use autodie;
use Data::Dumper;
my %experiments;
while ( my $line = <DATA> ) {
my ($condition, $more_data, $gene) = split /\s+/, $line;
my ($xloc, $change, $q_value) = split /:/, $more_data;
$xloc =~ s/^XLOC_(.*)_Change/$1/;
$change =~ s/_q$//;
my $result = Local::Result->new( $xloc, $change, $q_value );
push #{ $experiments{$gene}->{$condition} }, $result;
}
printf "%-10.10s %-10.10s %10.10s %-4s %-7s\n\n",
"Gene", "Condition", "XLoc", "Chng", "Q Value";
for my $gene ( sort keys %experiments ) {
for my $condition ( sort keys %{ $experiments{$gene} } ) {
for my $result ( #{ $experiments{$gene}->{$condition} } ) {
printf "%-10.10s %-10.10s %10.10s %-4.1f %-7.1f\n",
$gene, $condition, $result->xloc, $result->change, $result->q_value;
}
}
}
package Local::Result;
sub new {
my $class = shift;
my $xloc = shift;
my $change = shift;
my $q_value = shift;
my $self = {};
bless $self, $class;
$self->xloc($xloc);
$self->change($change);
$self->q_value($q_value);
return $self;
}
sub xloc {
my $self = shift;
my $xloc = shift;
if ( defined $xloc ) {
$self->{XLOC} = $xloc;
}
return $self->{XLOC};
}
sub change {
my $self = shift;
my $change = shift;
if ( defined $change ) {
$self->{CHANGE} = $change;
}
return $self->{CHANGE};
}
sub q_value {
my $self = shift;
my $q_value = shift;
if ( defined $q_value ) {
$self->{Q_VALUE} = $q_value;
}
return $self->{Q_VALUE};
}
package main;
__DATA__
eya XLOC_000445_Change:10.3_q:0.003 atonal1
six XLOC_00099_Change:70.0_q:0.095 atonal1
six-eya XLOC_0234324_Change:19.8_q:0.05 atonal1
eya XLOC_00010_Change:6.5_q:0.22 c-myc
six XLOC_025437_Change:1.1_q:0.018 c-myc
six-eya XLOC_001045_Change:2.3_q:0.0001 c-myc
eya XLOC_000115_Change:7.3_q:0.03 ezrin
six XLOC_000001_Change:7.9_q:0.00006 ezrin
six-eya XLOC_0234322_Change:9.0_q:0.0225 ezrin
six-eya XLOC_091345_Change:9.3_q:0.005 slc12a2
eya XLOC_000445_Change:9.9_q:0.3 atonal1
six XLOC_00099_Change:7.0_q:0.95 atonal1
six-eya XLOC_0234324_Change:9.8_q:0.5 atonal1