Perl: Comparing 2 hash of arrays with another array - arrays

I have written the code below in Perl but it's not giving the desirable output. I am dealing with the comparison between one array and two hash of arrays.
Given sample input files:
1) file1.txt
A6416 A2318
A84665 A88
2) hashone.pl
%hash1=(
A6416=>['E65559', 'C11162.1', 'c002gnj.3',],
A88=>['E77522', 'M001103', 'C1613.1', 'c001hyf.2',],
A84665=>['E138347', 'M032578', 'C7275.1', 'c009xpt.3',],
A2318=>['E128591', 'C43644.1', 'C47705.1', 'c003vnz.4',],
);
3) hashtwo.pl
%hash2=(
15580=>['C7275.1', 'E138347', 'M032578', 'c001jnm.3', 'c009xpt.2'],
3178=>['C1613.1', 'E77522','M001103', 'c001hyf.2', 'c001hyg.2'],
24406=>['C11162.1', 'E65559', 'M003010', 'c002gnj.2'],
12352=>['C43644.1', 'C47705.1', 'E128591','M001458', 'c003vnz.3'],
);
My aim is to achieve the task described:
From file1.txt, I have to locate the corresponding ID in %hash1. For instance,A6416 (file1.txt) is the key in %hash1. Next, I have to find the values of A6416 ['E65559', 'C11162.1', 'c002gnj.3',] in %hash2. If majority (more than 50%) of the values are found in %hash2, I replace A6416 with corresponding key from %hash2.
Example:
A6416 A2318
A84665 A88
Output:
24406 12352
15580 3178
Please note that the keys for %hash1 and %hash2 are different (they don't overlap). But the values are the same (they overlap).
#!/usr/bin/perl -w
use strict;
use warnings;
open FH, "file1.txt" || die "Error\n";
my %hash1 = do 'hashone.pl';
my %hash2 = do 'hashtwo.pl';
chomp(my #array=<FH>);
foreach my $amp (#array)
{
if ($amp =~ /(\d+)(\s?)/)
{
if (exists ($hash1{$1}))
{
for my $key (keys %hash2)
{
for my $i ( 0 .. $#{ $hash2{$key} } )
{
if ((#{$hash1{$1}}) eq ($hash2{$key}[$i]))
{
print "$key";
}
}
}
}
}
}
close FH;
1;
Any guidance on this problem is highly appreciated. Thank you!

I think you should invert %hash2 into this structure:
$hash2{'C7275.1'} = $hash2{'E138347'} = $hash2{'M032578'}
= $hash2{'c001jnm.3'} = $hash2{'c009xpt.2'} = 15580;
$hash2{'C1613.1'} = $hash2{'E77522'} = $hash2{'M001103'}
= $hash2{'c001hyf.2'} = $hash2{'c001hyg.2'} = 3178;
$hash2{'C11162.1'} = $hash2{'E65559'}
= $hash2{'M003010'} = $hash2{'c002gnj.2'} = 24406;
$hash2{'C43644.1'} = $hash2{'C47705.1'} = $hash2{'E128591'}
= $hash2{'M001458'} = $hash2{'c003vnz.3'} = 3178;
So that you can perform these look-ups much more effectively, rather than having to iterate over every element of every element of %hash2.

Building on the responses from ruakh and zock here is the code you need to build the look-up table for hash2
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my %hash2=(
15580=>['C7275.1', 'E138347', 'M032578', 'c001jnm.3', 'c009xpt.2'],
3178=>['C1613.1', 'E77522','M001103', 'c001hyf.2', 'c001hyg.2'],
24406=>['C11162.1', 'E65559', 'M003010', 'c002gnj.2'],
12352=>['C43644.1', 'C47705.1', 'E128591','M001458', 'c003vnz.3'],
);
# Build LUT for hash2
my %hash2_lut;
foreach my $key (keys %hash2)
{
foreach my $val (#{$hash2{$key}})
{
$hash2_lut{$val} = $key
}
}
print Dumper(\%hash2_lut);
Please select ruakh's post as the answer, just trying to clarify the code for you. Use Data::Dumper...it is your friend.
Here is the output:
$VAR1 = {
'C47705.1' => '12352',
'M032578' => '15580',
'E138347' => '15580',
'E77522' => '3178',
'C7275.1' => '15580',
'c001jnm.3' => '15580',
'E65559' => '24406',
'C1613.1' => '3178',
'M001458' => '12352',
'c002gnj.2' => '24406',
'c009xpt.2' => '15580',
'c001hyf.2' => '3178',
'C43644.1' => '12352',
'E128591' => '12352',
'c001hyg.2' => '3178',
'M003010' => '24406',
'c003vnz.3' => '12352',
'C11162.1' => '24406',
'M001103' => '3178'
};

Related

Strange behaviour of Perl's push function

I am writing a dedicated ICS (iCalendar file) parser.
I pass an array to a subroutine. All variables are single values apart from $notdates which is a comma-separated list of dates.
#entryl = ($dtstart, $dtend, $attendee, $lastmod, $uid, $notdates);
&entrytoarray(#entryl);
sub entrytoarray {
# print Dumper #_;
my $shiftdur = (&stamptoepoc($_[1]) - &stamptoepoc($_[0])) / 60 / 60;
my $attendee = $_[2];
my $deleted = $_[5];
$attendee =~ /ATTENDEE;USER-KEY=([^;]*);CN=([^;]*);.*:(.*)/;
my %ehash = (
"STARTDATE" , &stamptodate($_[0]),
"ENDDATE" , &stamptodate($_[1]),
"STARTSTAMP" , $_[0],
"ENDSTAMP" , $_[1],
"USERKEY" , $1,
"CN" , $2,
"EMAIL" , $3,
"LASTMOD" , $_[3],
"UID" , $_[4],
"DURATION" , $shiftdur
);
# Only keep current data
my $fdays = 4;
my $tdays = 7;
chomp(my $curstamp = `TZ="UTC" date -d "$fdays days" +"%Y%m%d%H%M00"`);
chomp(my $stpstamp = `TZ="UTC" date -d "$tdays days" +"%Y%m%d%H%M00"`);
if (($_[0] > $curstamp) && ($_[1] < $stpstamp)) {
if (defined($deleted)) {
my #deleted = split /,/, $deleted;
foreach (#deleted) {
if ($_ ne $_[0]) {
push(#entry, \%ehash);
}
}
}
else {
push(#entry, \%ehash);
}
}
print Dumper #entry;
This works mostly as expected:
$VAR1 = {
'DURATION' => '5',
'STARTSTAMP' => '20141122230000',
'UID' => '20141114T010539Z--1092363482',
'LASTMOD' => '20141118214419',
'STARTDATE' => '2014-11-22 23:00:00',
'EMAIL' => 'xxxxxxxxxxxxx',
'ENDDATE' => '2014-11-23 04:00:00',
'CN' => 'xxxxxxxxxxx',
'ENDSTAMP' => '20141123040000',
'KEY' => 'xxxxxxxxxxxxxxxxxx'
};
$VAR2 = {
'EMAIL' => 'xxxxxxxxxxxxx',
'ENDDATE' => '2014-11-23 23:00:00',
'ENDSTAMP' => '20141123230000',
'KEY' => 'xxxxxxxxxxx',
'CN' => 'xxxxxxxxxxxxxx',
'STARTDATE' => '2014-11-23 19:00:00',
'LASTMOD' => '20141118205901',
'UID' => '20141114T010456Z--1092363482',
'DURATION' => '4',
'STARTSTAMP' => '20141123190000'
};
$VAR3 = $VAR2;
Where is the $VAR3 = $VAR2 coming from?
My guess is that this section is the culprit:
foreach (#deleted) {
if ($_ ne $_[0]) {
push(#entry, \%ehash);
}
}
If you have several values in the array, the if-statement can be true twice, and thus push a value twice. Unless this is wanted behaviour, I would make sure that only one value is pushed. You can do this by using grep instead:
if (grep { $_ ne $_[0] } #deleted) {
push #entry, \%ehash;
}
Note that this replaces the foreach loop.
Your array #entry contains hash references. Data::Dumper is saying that the first and second elements of the array refer to two different hashes, while the third refers to the same hash as the second.
You don't show where #entry comes from, but I would expect all three elements to be references to %ehash.
The problem is that, if you keep pushing a reference to %ehash onto #entry, they all point to the same data item, and the intermediate states of the hash won't be recorded.
Unless you mean entrytoarray to push only one copy of %ehash (in which case there's a separate problem that we can't see) you need to fix it by either writing
push #entry, { %ehash }
which copies the hash and returns a reference to the copy, or you can declare and populate %ehash inside the foreach loop, which will create a new hash each time around the loop.

Converting and printing array of hashes - Perl

I really dont know how to do it so I ended up here.
I want to convert this input:
my #sack_files_1 = (
'mgenv/1_2_3/parent.dx_environment',
'mgenv/1_2_3/doc/types.dat',
'u5env/1_2_3/parent.dx_environment',
'u5env/1_2_3/doc/types.dat',
);
To this:
my $sack_tree_1 = {
'mgenv' => {
'1_2_3' => [ 'parent.dx_environment', 'doc/types.dat' ],
},
'u5env' => {
'1_2_3' => [ 'parent.dx_environment', 'doc/types.dat' ],
}
};
Something like this should do the trick:
use strict;
use warnings;
use Data::Dumper;
my #sack_files_1 = (
'mgenv/1_2_3/parent.dx_environment',
'mgenv/1_2_3/doc/types.dat',
'u5env/1_2_3/parent.dx_environment',
'u5env/1_2_3/doc/types.dat',
);
my %sack_tree_1;
foreach (#sack_files_1) {
my ( $env, $number, #everything_else ) = split('/');
push( #{ $sack_tree_1{$env}{$number} }, join( "/", #everything_else ) );
}
print Dumper \%sack_tree_1
This will do as you ask. It uses File::Spec::Functions to split each path into its components.
The first two elements of the hash are used directly as hash keys, relying on autovivication to create the necessary hash elements.
A simple push to an implied array reference also autovivifies the lowest-level hash element.
I have used Data::Dump to display the resulting hash. It is not part of the core Perl installation and you may need to install it, but it is much superior to Data::Dumper.
use strict;
use warnings;
use File::Spec::Functions qw/ splitdir catfile /;
my #sack_files_1 = (
'mgenv/1_2_3/parent.dx_environment',
'mgenv/1_2_3/doc/types.dat',
'u5env/1_2_3/parent.dx_environment',
'u5env/1_2_3/doc/types.dat',
);
my %paths;
for my $path (#sack_files_1) {
my ($p1, $p2, #path) = splitdir $path;
push #{ $paths{$p1}{$p2} }, catfile #path;
}
use Data::Dump;
dd \%paths;
output
{
mgenv => { "1_2_3" => ["parent.dx_environment", "doc\\types.dat"] },
u5env => { "1_2_3" => ["parent.dx_environment", "doc\\types.dat"] },
}
my $sack_tree_1 = {};
foreach my $data (#sack_files_1) {
my #path = split '/', $data;
my ($file,$last_part) = pop #path, pop #path; # get the file name and last part of the path
my $hash_part = $sack_tree_1;
foreach my $path (#path) { # For every element in the remaining part of the path
$hash_part->{$path} //= {}; # Make sure we have a hash ref to play with
$hash_part = $hash_part->{$path} # Move down the hash past the current path element
}
$hash_part->{$last_part} = $file; # Add the file name to the last part of the path
}
This handles all path lengths of 2 or more

Trying to figure out how to push specific links contained in each link of separate list of links into an array

GENERAL IDEA
Here is a snippet of what I'm working with:
my $url_temp;
my $page_temp;
my $p_temp;
my #temp_stuff;
my #collector;
foreach (#blarg_links) {
$url_temp = $_;
$page_temp = get( $url_temp ) or die $!;
$p_temp = HTML::TreeBuilder->new_from_content( $page_temp );
#temp_stuff = $p_temp->look_down(
_tag => 'foo',
class => 'bar'
);
foreach (#temp_stuff) {
push(#collector, "http://www.foobar.sx" . $1) if $_->as_HTML =~ m/href="(.*?)"/;
};
};
Hopefully it is clear that what I'm hopelessly trying to do is push the link endings found in each of a list of links into an array called #temp_stuff. So the first link in #blarg_links, when visited, has greater than or equal to 1 foo tag with an associated bar class that when acted on by as_HTML will match something I want in the href equality to then pump into an array of links which have the data I'm really after... Does that make sense?
ACTUAL DATA
my $url2 = 'http://www.chemistry.ucla.edu/calendar-node-field-date/year';
my $page2 = get( $url2 ) or die $!;
my $p2 = HTML::TreeBuilder->new_from_content( $page2 );
my #stuff2 = $p2->look_down(
_tag => 'div',
class => 'year mini-day-on'
);
my #chem_links;
foreach (#stuff2) {
push(#chem_links, $1) if $_->as_HTML =~ m/(http:\/\/www\.chemistry\.ucla\.edu\/calendar-node-field-date\/day\/[0-9]{4}-[0-9]{2}-[0-9]{2})/;
};
my $url_temp;
my $page_temp;
my $p_temp;
my #temp_stuff;
my #collector;
foreach (#chem_links) {
$url_temp = $_;
$page_temp = get( $url_temp ) or die $!;
$p_temp = HTML::TreeBuilder->new_from_content( $page_temp );
#temp_stuff = $p_temp->look_down(
_tag => 'span',
class => 'field-content'
);
};
foreach (#temp_stuff) {
push(#collector, "http://www.chemistry.ucla.edu" . $1) if $_->as_HTML =~ m/href="(.*?)"/;
};
n.b. - I want to use HTML::TreeBuilder. I'm aware of alternatives.
This is a rough attempt at what I think you want.
It fetches all the links on the first page and visits each of them in turn, printing the link in each <span class="field-content"> element.
use strict;
use warnings;
use 5.010;
use HTML::TreeBuilder;
STDOUT->autoflush;
my $url = 'http://www.chemistry.ucla.edu/calendar-node-field-date/year';
my $tree = HTML::TreeBuilder->new_from_url($url);
my #chem_links;
for my $div ( $tree->look_down( _tag => 'div', class => qr{\bmini-day-on\b} ) ) {
my ($anchor)= $div->look_down(_tag => 'a', href => qr{http://www\.chemistry\.ucla\.edu});
push #chem_links, $anchor->attr('href');
};
my #collector;
for my $url (#chem_links) {
say $url;
my $tree = HTML::TreeBuilder->new_from_url($url);
my #seminars;
for my $span ( $tree->look_down( _tag => 'span', class => 'field-content' ) ) {
my ($anchor) = $span->look_down(_tag => 'a', href => qr{/});
push #seminars, 'http://www.chemistry.ucla.edu'.$anchor->attr('href');
}
say " $_" for #seminars;
say '';
push #collector, #seminars;
};
For a more modern framework for parsing webpages, I would suggest you take a look at Mojo::UserAgent and Mojo::DOM. Instead of having to manually march through each section of your html tree, you can use the power of css selectors to zero in on the specific data that you want. There's a nice 8 minute introductory video on the framework at Mojocast Episode 5.
# Parses the UCLA Chemistry Calendar and displays all seminar links
use strict;
use warnings;
use Mojo::UserAgent;
use URI;
my $url = 'http://www.chemistry.ucla.edu/calendar-node-field-date/year';
my $ua = Mojo::UserAgent->new;
my $dom = $ua->get($url)->res->dom;
for my $dayhref ($dom->find('div.mini-day-on > a[href*="/day/"]')->attr('href')->each) {
my $dayurl = URI->new($dayhref)->abs($url);
print $dayurl, "\n";
my $daydom = $ua->get($dayurl->as_string)->res->dom;
for my $seminarhref ($daydom->find('span.field-content > a[href]')->attr('href')->each) {
my $seminarurl = URI->new($seminarhref)->abs($dayurl);
print " $seminarurl\n";
}
print "\n";
}
Output is identical to that of Borodin's solution using HTML::TreeBuilder:
http://www.chemistry.ucla.edu/calendar-node-field-date/day/2014-01-06
http://www.chemistry.ucla.edu/seminars/nano-rheology-enzymes
http://www.chemistry.ucla.edu/calendar-node-field-date/day/2014-01-09
http://www.chemistry.ucla.edu/seminars/imaging-approach-biology-disease-through-chemistry
http://www.chemistry.ucla.edu/calendar-node-field-date/day/2014-01-10
http://www.chemistry.ucla.edu/seminars/arginine-methylation-%E2%80%93-substrates-binders-function
http://www.chemistry.ucla.edu/seminars/special-inorganic-chemistry-seminar
http://www.chemistry.ucla.edu/calendar-node-field-date/day/2014-01-13
http://www.chemistry.ucla.edu/events/robert-l-scott-lecture-0
...

Perl: Create a hash from an array

If I have the following array
my #header_line = ('id', 'name', 'age');
How do I create a hash from it equivalent to the line below?
my %fields = { id => 0, name => 1, age => 2};
The reason I want to do this is so that I can use meaningful names rather than magic numbers for indexes. For example:
$row->[$fields{age}]; # rather than $row->[2]
my %fields;
#fields{#header_line} = (0 .. $#header_line);
my %fields = map { $header_line[$_] => $_ } 0..$#header_line;
You said in reply to a comment that this is coming from Text::CSV. This module has a way to import this into a hash for you.
$csv->column_names( #header_line );
$row = $csv->getline_hr( $FH );
print $row->{ 'id' };
my %fields = ();
for (my $i = 0; $i < scalar(#header_line); $i++) {
$fields{$header_line[$i]} = $i;
}
TIMTOWTDI
my %fields = ();
foreach my $field(#header_line)
{
$fields{$field} = scalar(keys(%fields));
}

Search for hash in an array by value

I have a function which extracts Excel data into an array of hashes like so:
sub set_exceldata {
my $excel_file_or = '.\Excel\ORDERS.csv';
if (-e $excel_file_or) {
open (EXCEL_OR, $excel_file_or) || die("\n can't open $excel_file_or: $!\n");
while () {
chomp;
my ( $id, $date, $product, $batchid, $address, $cost ) = split ",";
my %a = ( id => $id
, date => $date
, product => $product
, batchid => $batchid
, address => $address
, cost => $cost
);
push ( #array_data_or, \%a );
}
close EXCEL_OR;
}
}
Populating the array of hashes is fine. However, the difficult part is searching for a particular item (hash) in the array. I can't seem to locate items that might have an id or 21, or a batchid of 15, or a cost > $20 etc.
How would I go about implementing such a search facility?
Thanks to all,
With the power of grep
my #matching_items = grep {
$_->{id} == 21
} #array_data_or;
If you know there will be only one item returned you can just do this:
my ($item) = grep {
$_->{id} == 21
} #array_data_or;
(Untested, and I haven't written one of these in a while, but this should work)
If you're sure that the search always returns only one occurence or if you're interested in only the first match then you could use the 'first' subroutine found in List::Util
use List::Util;
my %matching_hash = %{ first { $_->{id} == 21 } #array_data_or };
I enclosed the subroutine call in the %{ } block to ensure that the RHS evaluates to a hash.

Resources