How do I breakdown common elements in hash of arrays in perl? - arrays

I am trying to find any intersections of elements within a hash of arrays in Perl
For example
my %test = (
Lot1 => [ "A","B","C"],
Lot2 => [ "A","B","C"],
Lot3 => ["C"],
Lot4 => ["E","F"],
);
The result I would be after is
Lot1 and Lot2 have AB
Lot1,Lot2 and Lot3 have C
Lot4 has E and F.
I think this could be done with a recursive function that effectively moves its way through the arrays and if an intersection between two arrays is found it calls itself recursively with the intersection found and the next array. The stopping condition would be running out of arrays.
Once the function is exited I would have to iterate through the hash to get the arrays that contain these values.
Does this sound like a good approach? I have been struggling with the code, but was going to use List::Compare to determine the intersection.
Thank you.

Array::Utils has an intersection operation where you can test the intersect of two arrays. But that's only the start point of what you're trying to do.
So I would be thinking that you need to first invert your lookup:
my %member_of;
foreach my $key ( keys %test ) {
foreach my $element ( #{$test{$key}} ) {
push ( #{$member_of{$element}}, $key );
}
}
print Dumper \%member_of;
Giving:
$VAR1 = {
'A' => [
'Lot1',
'Lot2'
],
'F' => [
'Lot4'
],
'B' => [
'Lot1',
'Lot2'
],
'E' => [
'Lot4'
],
'C' => [
'Lot1',
'Lot2',
'Lot3'
]
};
Then collapse that, into a key set:
my %new_set;
foreach my $element ( keys %member_of ) {
my $set = join( ",", #{ $member_of{$element} } );
push( #{ $new_set{$set} }, $element );
}
print Dumper \%new_set;
Giving:
$VAR1 = {
'Lot1,Lot2,Lot3' => [
'C'
],
'Lot1,Lot2' => [
'A',
'B'
],
'Lot4' => [
'E',
'F'
]
};
So overall:
#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
my %test = (
Lot1 => [ "A", "B", "C" ],
Lot2 => [ "A", "B", "C" ],
Lot3 => ["C"],
Lot4 => [ "E", "F" ],
);
my %member_of;
foreach my $key ( sort keys %test ) {
foreach my $element ( #{ $test{$key} } ) {
push( #{ $member_of{$element} }, $key );
}
}
my %new_set;
foreach my $element ( sort keys %member_of ) {
my $set = join( ",", #{ $member_of{$element} } );
push( #{ $new_set{$set} }, $element );
}
foreach my $set ( sort keys %new_set ) {
print "$set contains: ", join( ",", #{ $new_set{$set} } ), "\n";
}
I don't think there's a more efficient way to tackle it, because you're comparing each array to each other array, and forming a new compound key out of it.
This gives you:
Lot1,Lot2 contains: A,B
Lot1,Lot2,Lot3 contains: C
Lot4 contains: E,F

This can be done as two simple hash conversions:
Build a hash that lists all of the lots each item is in
Convert that to a hash that lists all items for each lot combination
Then just dump the last hash in a convenient form
This is the code.
use strict;
use warnings 'all';
use feature 'say';
my %test = (
Lot1 => [ "A", "B", "C" ],
Lot2 => [ "A", "B", "C" ],
Lot3 => ["C"],
Lot4 => [ "E", "F" ],
);
my %items;
for my $lot ( keys %test ) {
for my $item ( #{ $test{$lot} } ) {
push #{ $items{$item} }, $lot;
}
}
my %lots;
for my $item ( keys %items ) {
my $lots = join '!', sort #{ $items{$item} };
push #{ $lots{$lots} }, $item;
}
for my $lots ( sort keys %lots ) {
my #lots = split /!/, $lots;
my $items = join '', #{ $lots{$lots} };
$lots = join ', ', #lots;
$lots =~ s/.*\K,/ and/;
printf "%s %s %s\n", $lots, #lots > 1 ? 'have' : 'has', $items;
}
output
Lot1 and Lot2 have AB
Lot1, Lot2 and Lot3 have C
Lot4 has EF
It generates an %items hash that looks like this
{
A => ["Lot2", "Lot1"],
B => ["Lot2", "Lot1"],
C => ["Lot2", "Lot3", "Lot1"],
E => ["Lot4"],
F => ["Lot4"],
}
and from that a %lots hash that looks like this
{
"Lot1!Lot2" => ["A", "B"],
"Lot1!Lot2!Lot3" => ["C"],
"Lot4" => ["E", "F"],
}

Related

perl call variable name using another variable(array)

my #CLASS_TYPES = ("INTRA", "BB", "CAT");
my #INTRA_NEIGH = ("1.1.1.1/32","2.2.2.2/32");
my #BB_NEIGH = ("3.3.3.3/32","4.4.4.4/32" );
foreach my $class (#CLASS_TYPES) {
my $csv = #.$class._NEIGH;
print($csv);
when I print $csv I am expecting to print the array values how do I achieve that
That's not the right approach. See Why it's stupid to `use a variable as a variable name' and A More Direct Explanation of the Problem.
The very fact that #CAT_NEIGH doesn't exist illustrates part of the problem.
Solution:
my #CLASS_TYPES = ("INTRA", "BB", "CAT");
my %NEIGH = (
INTRA => [ "1.1.1.1/32", "2.2.2.2/32" ],
BB => [ "3.3.3.3/32", "4.4.4.4/32" ],
);
for my $class (#CLASS_TYPES) {
next if !$NEIGH{$class};
print "$_\n" for #{ $NEIGH{$class} };
}
or just
my %NEIGH = (
INTRA => [ "1.1.1.1/32", "2.2.2.2/32" ],
BB => [ "3.3.3.3/32", "4.4.4.4/32" ],
);
for my $class (keys(%NEIGH)) {
print "$_\n" for #{ $NEIGH{$class} };
}

Convert an array of strings into a array of arrays of strings

My goal is to convert this
my #array=("red", "blue", "green", ["purple", "orange"]);
into this
my #array = ( ["red"], ["blue"], ["green"], ["purple", "orange"]);
Current test code:
my #array = ("red", "blue", "green", ["purple", "orange"] );
foreach $item ( #array ) {
#if this was Python, it would be as simple as:
# if is not instance(item, array):
# # item wasn't a list
# item = [item]
if(ref($item) ne 'ARRAY'){
#It's an array reference...
#you can read it with $item->[1]
#or dereference it uisng #newarray = #{$item}
#print "We've got an array!!\n";
print $item, "\n";
# keep a copy of our string
$temp = $item;
# re-use the variable but convert to an empty list
#item = ();
# add the temp-copy as first list item
#item[0] = $temp;
# print each list item (should be just one item)
print "$_\n" for $item;
}else{
#not an array in any way...
print "ALREADY an array!!\n";
}
}
# EXPECTED my #array=(["red"], ["blue"], ["green"], ["purple", "orange"]);
print #array , "\n";
foreach $item (#array){
if(ref($item) ne 'ARRAY'){
#
#say for $item;
print "didn't convert properly to array\n";
}
}
The comment about python maps pretty directly to perl.
my #array = ("red", "blue", "green", ["purple", "orange"] );
foreach $item ( #array ) {
#if this was Python, it would be as simple as:
# if is not instance(item, array):
# # item wasn't a list
# item = [item]
if (ref $item ne 'ARRAY') {
$item = [ $item ];
}
}
though using map as in Borodin's answer would be more natural.
I'm wondering why you want to do this, but it's
#array = map { ref ? $_ : [ $_ ] } #array
And please don't call arrays #array; that's what the # is for.
Your comment is ridiculous
#if this was Python, it would be as simple as:
# if is not instance(item, array):
# # item wasn't a list
# item = [item]
If you were familiar with Perl then you wouldn't need to ask the question. You must be aware that there is no one-to-one translation from Python to Perl. Python is much less expressive than either Perl or C, but I can't imagine you demanding a simple conversion to C.
Please get over your bigotry.
If you push the values to a new array, you don't need to do more than evaluate whether or not $item is an arrayref:
#! perl
use strict;
use warnings;
use Data::Dumper;
my #array=("red", "blue", "green", ["purple", "orange"]);
my #new_array;
foreach my $item (#array) {
if ( ref($item) eq 'ARRAY' ) {
push #new_array, $item;
}
else {
push #new_array, [$item];
}
}
print Dumper \#new_array;
Output from Dumper:
$VAR1 = [
[
'red'
],
[
'blue'
],
[
'green'
],
[
'purple',
'orange'
]
];
After a long day of learning more Perl than I ever thought/wanted to learn... here's what I think is a workable solution:
#! perl
use strict;
use warnings;
use Data::Dumper;
my %the_dict = (duts =>
{dut_a => {UDF => 'hamnet'},
dut_b => {UDF => [ '1', '2', '3', ]},
dut_c => {UDF => [ 'Z' ], }});
print Dumper \%the_dict;
foreach my $dut (keys %{$the_dict{duts}}) {
# convert the dut's UDF value to an array if it wasn't already
if ( 'ARRAY' ne ref $the_dict{duts}->{$dut}{UDF} ) {
$the_dict{duts}->{$dut}{UDF} = [ $the_dict{duts}->{$dut}{UDF} ];
}
# now append a new value to the array
push(#{$the_dict{duts}{$dut}{UDF}}, 'works');
}
print Dumper \%the_dict;
when run we see these print-outs:
$VAR1 = {
'duts' => {
'dut_a' => {
'UDF' => 'hamnet'
},
'dut_c' => {
'UDF' => [
'Z'
]
},
'dut_b' => {
'UDF' => [
'1',
'2',
'3'
]
}
}
};
$VAR1 = {
'duts' => {
'dut_a' => {
'UDF' => [
'hamnet',
'works'
]
},
'dut_c' => {
'UDF' => [
'Z',
'works'
]
},
'dut_b' => {
'UDF' => [
'1',
'2',
'3',
'works'
]
}
}
};

perl: deep merge with per-element arrays merge

I'm trying to merge two hashes and Hash::Merge does almost exactly what I need, except for arrays. Instead of concatenating arrays I need it to do per-element merge.
For example:
use Hash::Merge qw (merge);
my %a = ( 'arr' => [ { 'a' => 'b' } ] );
my %b = ( 'arr' => [ { 'c' => 'd' } ] );
my %c = %{ merge( \%a, \%b) };
Desired result is ('arr'=>[{'a'=>'b','c'=>'d'}]), actual result is ('arr'=>[{'a'=>'b'},{'c'=>'d'}])
Can this be done by using specify_behavior or is there some other way?
I think that specify_behaviour is used to specify how to handle conflicts, or uneven structures to merge. The documentation doesn't actually say much. But try it, go through defined shortcuts, or try to set them yourself. For your data structure you could try
SCALAR => ARRAY => sub { [ %{$_0}, %{$_[0]} ] }
SCALAR => ARRAY => HASH => sub { [ $_[0], $_[0] ] }
If you tried and it didn't work you may have found a bug in the module? By what you show it just didn't go "deep" enough. Here it is without the module. I've enlarged your sample structures.
use warnings;
use strict;
my %a = (
'arr1' => [ { a => 'A', a1 => 'A1' } ],
'arr2' => [ { aa => 'AA', aa1 => 'AA1' } ]
);
my %b = (
'arr1' => [ { b => 'B', b1 => 'B1' } ],
'arr2' => [ { bb => 'BB', bb1 => 'BB1' } ]
);
# Copy top level, %a to our target %c
my %c;
#c{keys %a} = values %a;
# Iterate over hash keys, then through array
foreach my $key (sort keys %c) {
my $arr_len = #{$c{$key}};
foreach my $i (0..$arr_len-1) {
my %hb = %{ ${$b{$key}}[$i] };
# merge: add %b to %c
#{ ${$c{$key}}[$i] }{keys %hb} = values %hb;
}
}
# Print it out
foreach my $key (sort keys %c) {
print "$key: ";
my $arr_len = #{$c{$key}};
foreach my $i (0..$arr_len-1) {
my %hc = %{ ${$c{$key}}[$i] };
print "$_ => $hc{$_}, " for sort keys %hc;
}
print "\n";
}
This prints the contents of %c (aligned manually here)
arr1: a => A, a1 => A1, b => B, b1 => B1,
arr2: aa => AA, aa1 => AA1, bb => BB, bb1 => BB1,
Code does not handle arrays/hashes of unequal sizes but checks can be added readily.
Another solution (that handles uneven hash elements in %a and %b).
my %c;
foreach my $key (keys %a, keys %b) {
my $a_ref = $a{$key};
my $b_ref = $b{$key};
$c{$key} = { map %$_, #$a_ref, #$b_ref };
}
use Data::Dumper;
print Dumper \%c;

Perl: Getting difference between two arrays of hashes?

I have two array references that contain hashes:
$A = [
{
"t" => "1419054300000",
"v" => "28.1"
},
{
"t" => "1419053400000",
"v" => "28.2"
},
{
"t" => "1419052500000",
"v" => "28.4"
}
];
$B = [
{
"t" => "1419053400000",
"v" => "28.2"
},
{
"t" => "1419052500000",
"v" => "28.4"
}
];
I want to get only the hashes from $A where their value of t doesn't already exist in one of the hashes in $B (the t values are unique per arrayref, v isn't).
I assume there's some obvious method of doing this, but I've been banging my head against this all day without success.
You can use the perl5i diff method.
use perl5i::2;
...initialize $A and $B...
say $A->diff($B)->mo->as_perl;
__END__
[
{
't' => '1419054300000',
'v' => '28.1'
}
]
As always you can build hash look up where keys are elements you want to filter out,
my %seen;
#seen{ map $_->{t}, #$B } = ();
my $C = [
grep { !exists $seen{$_->{t}} } #$A
];

Perl multi hash from list

I have a list with some values which are connected. I need to create a hashmap with keys and values from the list and merge together. But i don't really know how to do it.
Input:
my #in =(
'mgenv/1_2_3/parent.dx_environment',
'mgenv/1_2_3/doc/types.dat');
Expected output:
"{ $env => { $ver => [ $file1, $file2, ... ] } }"
I've tried these:
(1)
my #sack_files = (
'mgenv/1_2_3/parent.dx_environment',
'mgenv/1_2_3/doc/types.dat');
my $sack_tree = {};
my %hash=();
for( my $i=0; $i<scalar #sack_files; $i++){
my #array = split(/[\/]+/,$sack_files[$i]);
for(my $i=0;$i<(scalar #array)-1;$i++){
my $first = $array[$i];
my $second = $array[$i+1];
$hash{$first}=$second;
}
# merge
}
(2)
use Data::Dumper;
my #sack_files = (
'mgenv/1_2_3/parent.dx_environment',
'mgenv/1_2_3/doc/types.dat',
);
my $sack_tree = {};
my %hash=();
for( my $i=0; $i<scalar #sack_files; $i++){
my #array = split(/[\/]+/,$sack_files[$i]);
nest(\%hash,#array);
}
In the second case I get an error because when the loop variable i=1 ,the key/values already exists so maybe i have to check the previously added key/values. But I don't really know how.
I would really appreciate any ideas.
Just use push to add new members to an existing array in a hash of hashes. You have to dereference the array reference with #{ ... }.
#!/usr/bin/perl
use warnings;
use strict;
use Data::Dumper;
my #sack_files = qw( mgenv/1_2_3/parent.dx_environment
mgenv/1_2_3/doc/types.dat
mgenv/1_2_3/doc/etc.dat
mgenv/4_5_6/parent.dx_environment
mgenv/4_5_6/doc/types.dat
u5env/1_2_3/parent.dx_environment
u5env/1_2_3/doc/types.dat
u5env/4_5_6/parent.dx_environment
u5env/4_5_6/doc/types.dat
);
my %hash;
for my $sack_file (#sack_files) {
my ($env, $ver, $file) = split m{/}, $sack_file, 3;
push #{ $hash{$env}{$ver} }, $file;
}
print Dumper \%hash;
output
$VAR1 = {
'mgenv' => {
'1_2_3' => [
'parent.dx_environment',
'doc/types.dat',
'doc/etc.dat'
],
'4_5_6' => [
'parent.dx_environment',
'doc/types.dat'
]
},
'u5env' => {
'4_5_6' => [
'parent.dx_environment',
'doc/types.dat'
],
'1_2_3' => [
'parent.dx_environment',
'doc/types.dat'
]
}
};

Resources