Perl: Dump data from a hash into excel - arrays

I have a hash with key and Values(array). I want to dump them to a spreadsheet, but having difficulty in arranging them .
%hash
key1 -> foo bar
key2-> john adam gill
key3-> apple banana mango orange
Code:
use strict;
use warnings;
use Excel::Writer::XLSX;
my $pattern = "BEGIN_";
my $format;
my #keys = qw(key1 key2 key3);
foreach my $key(#keys){
open my $fh, "<","filename.txt" or die $!;
while ( <$fh> ) {
if (/$pattern/) {
push(#matching_lines, $_);
}
}
$hash{$key} = [#matching_lines] ;
for (#matching_lines) { $_ = undef } ; #Emptying the array contents,to reuse it for for all the other keys
}
my $workbook = Excel::Writer::XLSX->new( 'c:\TEMP\filename.xlsx' );
if (not defined $workbook)
{
die "Failed to create spreadsheet: $!";
}
my $worksheet = $workbook->add_worksheet();
# Add and define a format
$format = $workbook->add_format();
$format->set_bg_color( 'yellow' );
my $row = 1;
my $col = 0;
foreach my $k (keys %hash)
{
$worksheet->write($row, $col, $k, $format); # title
$worksheet->write_col($row+1, $col, $hash{$k}); #value
$col++;
}
$workbook->close() or die "Error closing file: $!";
Current Output
Desired Output

Edit: Now you've actually updated your program to clarify that the problem is how you're reading your data, the below is moot. But it does illustrate an alternative approach.
OK, the core problem here is what you're trying to do is 'flip' a hash. You're printing row by row, but your hash is organised in columns.
Using comma sep as a quick proxy for printing actual excel:
#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
#initialise
my %hash = (
key1 => [qw ( foo bar )],
key2 => [qw ( john adam gill )],
key3 => [qw ( apple banana mango orange )],
);
#print for debug
print Dumper \%hash;
#get header row. Sort it, because hashes are unordered.
#could instead:
#my #keys = qw ( key1 key2 key3 );
my #keys = sort keys %hash;
#print header row
print join ",", #keys, "\n";
#iterate until every element of the hash is gone
while ( map { #{ $hash{$_} } } #keys ) {
#cycle the keys, shifting a value of the top of each array.
#replace any undefined values with ''.
print shift( #{ $hash{$_} } ) // '', "," for #keys;
print "\n";
}
This prints:
key1,key2,key3,
foo,john,apple,
bar,adam,banana,
,gill,mango,
,,orange,
Which if you load it as csv into Excel, should give your desired result. I'm pretty sure you could use a similar 'write row' with the module.
So this actually seems to do what you want:
#!/usr/env/perl
use strict;
use warnings;
use Excel::Writer::XLSX;
#initialise
my %hash = (
key1 => [qw ( foo bar )],
key2 => [qw ( john adam gill )],
key3 => [qw ( apple banana mango orange )],
);
my $workbook = Excel::Writer::XLSX->new('c:\TEMP\filename.xlsx');
if ( not defined $workbook ) {
die "Failed to create spreadsheet: $!";
}
my $worksheet = $workbook->add_worksheet();
# Add and define a format
my $format = $workbook->add_format();
$format->set_bg_color('yellow');
my #keys = sort keys %hash;
my $row = 0;
$worksheet->write_row( $row++, 0, \#keys, $format );
while ( map { #{ $hash{$_} } } #keys ) {
my $col = 0;
$worksheet->write( $row, $col++, shift( #{ $hash{$_} } ) // '' )
for #keys;
$row++;
}
$workbook->close() or die "Error closing file: $!";

You're not correctly emptying your #matching_lines array. This line:
for (#matching_lines) { $_ = undef }
Sets the array values to undef, but does not remove them.
For example, if #matching_lines was ('foo', 'bar'), now it becomes (undef, undef). When you add baz and qux to it later, it becomes (undef, undef, 'baz', 'qux'). These undefs become blank cells when you add them to the worksheet.
To correctly empty the array, use:
#matching_lines = ();

Related

perl read data from textfile

I wanted to get the fruit and info from an input file in order (reading from line 1 and proceeds). The code below is somehow printing the fruit and info in any random order, everytime running the script generate different order, not reading from line 1. Any recommendation?
I am having an input file something like below
apple
text1
text2
grape
text3
text4
jackfruit
text5
and this is the code I am having to call out each fruit and info
use strict;
use warnings;
my %hash;
open FILE, "config.txt" or die $!;
my $key;
while (my $line = <FILE>) {
chomp($line);
if ($line !~ /^\s/) {
$key = $line;
#$hash{$key} = [];
} else {
$line =~ s/\s//g;
push (#{ $hash{$key} }, $line);
}
}
close FILE;
my %final;
foreach my $fruit (keys %hash){
foreach my $info (values #{$hash{$fruit}}){
print "Fruit: $fruit\n";
print "Info for $fruit = $info\n";
}
}
keys %hash
Gives you an array of the keys from the hash, but not sorted. You can sort it with the command sort
The whole line would be
foreach my $fruit (sort(keys %hash)){
Use perldoc -f sort to get help of the sort function.
If you want to keep things in order that they are in the file, use an array. Each element of that array can be a hash that organizes the data:
#!perl
use v5.26;
my #items;
while( <DATA> ) {
chomp;
if( /\A(\S+)/ ) {
push #items, { fruit => $1, info => [] }
}
elsif( /\A\s+(.+)/ ) {
push $items[-1]{info}->#*, $1
}
}
foreach my $item ( #items ) {
print "Fruit: $item->{fruit}\n";
foreach my $info ( $item->{info}->#* ) {
print "\tInfo: $info\n";
}
}
__END__
apple
text1
text2
grape
text3
text4
jackfruit
text5
cranberry
text6
The output maintains the order in the file:
Fruit: apple
Info: text1
Info: text2
Fruit: grape
Info: text3
Info: text4
Fruit: jackfruit
Info: text5
Fruit: cranberry
Info: text6
However, if you want to keep them in order of the file and merely output them, you don't need a data structure:
my #items;
while( <DATA> ) {
chomp;
if( /\A(\S+)/ ) {
print "Fruit: $1\n";
}
elsif( /\A\s+(.+)/ ) {
print "\tInfo: $1\n";
}
}
If you wanted slightly different output where each line needed to know the fruit, store that name in a persistent variable in the while loop:
my #items;
while( <DATA> ) {
state $fruit;
chomp;
if( /\A(\S+)/ ) {
$fruit = $1;
print "Fruit: $fruit\n";
}
elsif( /\A\s+(.+)/ ) {
print "\t$fruit: $1\n";
}
}

perl array not populating, despite hours of tinkering

Um...I have the following code snippet, and was wondering why the second subroutine, search($$) fails to yield results...while the first routine, search_item($$$), performs admirably (imo).
########################
# generate and return a list of users which match only
# one criteria (eg: eyes=2)
#
# $users is a reference to an array of 6-digit hexidecimal user IDs (eg: 000001, 000002, etc)
# $name is the name of the key (or field) to find (eg: 'eyes')
# $value (eg: 2) is compared with the value stored in the key
# if $value matches what's in the $name'd key, then add the uid to a list
sub search_item($$$) {
my ($users, $name, $value) = #_;
my #searched;
foreach my $uid (#$users) {
my %ustats = user::getUserStats($uid);
if ($ustats{$name} eq $value) { push #searched, $uid; }
}
return #searched;
}
########################
# generate and return a list of users which match
# many criteria (eg: eyes=2, hair=1, etc)
#
# $users is a reference to an array of user IDs (eg: 000001, 000002, etc)
# $terms is a reference to an array of search terms (eg: $terms[0] = "eyes=2";)
sub search($$) {
my $users = $_[0]; # an array reference
my $terms = $_[1]; # an array reference
my #searched;
my $first = 1;
foreach my $term (#$terms) {
# since #$terms is an array of scalars, in the format of 'name=value' pairs
my $name = $term; $name =~ s/=(.)*//;
my $value = $term; $value =~ s/$name=//;
if ($first) {
# search the given list reference ($users)
#searched = search_item($users, $name, $value);
$first = 0; # set to 0 cause now we gotta use #searched
} else {
# otherwise use a reference to #searched
#searched = search_item(\#searched, $name, $value);
}
}
return #searched;
}
i have setup the data so the code should return 1 hit. the data is correct and underlying functions (eg: getUserStats($)) also perform flawlessly.
both user 000001 and 969696 have eyes=2 all others eyes=1 and
user ID 000001 is gender=1, all others gender=0
so...if i write:
my #users = getUsers();
foreach my $uid (search_item(\#users, 'eyes', 2)) {
print "$uid<br>\n";
}
i get 2 hits of a total of 6 users in my database (this IS a correct result, of course). satisfied with those results, I run the search routine.
my #terms = ('eyes=2', 'gender=1'); # gender=0 is a boy. 1 is a girl
my #sResults = search(\#users, \#terms);
if (#sResults) {
foreach my $uid (#sResults) {
print "$uid<br>\n";
}
} else {
print "nothing found!<br>\n";
}
i always see "nothing found!" when i pray and hope to see "000001" instead... :(
this seems like legit code....so....whud am i doin wrong guys??? am i not derefencing something correctly? or...is the dereferencing / referencing the source of my dilema? i loath pointers...however incredibly useful :p
Your code does actually work if you pass correct parameters. My best guess is that the strings like eyes=2 that you're using contain spurious whitespace such as a trailing newline
Here's the test program that I used to work on your subrouitines
use strict;
use warnings;
use 5.010;
my %users = (
'000001' => { eyes => 2, gender => 1 },
'000002' => { eyes => 1, gender => 0 },
'000003' => { eyes => 1, gender => 0 },
'000004' => { eyes => 1, gender => 0 },
'969696' => { eyes => 2, gender => 0 },
);
sub user::getUserStats {
my ( $uid ) = #_;
%{ $users{$uid} };
}
########################
# generate and return a list of users which match only
# one criteria (eg: eyes=2)
#
# $users is a reference to an array of 6-digit hexidecimal user IDs (eg: 000001, 000002, etc)
# $name is the name of the key (or field) to find (eg: 'eyes')
# $value (eg: 2) is compared with the value stored in the key
# if $value matches what's in the $name'd key, then add the uid to a list
sub search_item($$$) {
my ( $users, $name, $value ) = #_;
my #searched;
foreach my $uid ( #$users ) {
my %ustats = user::getUserStats( $uid );
if ( $ustats{$name} eq $value ) { push #searched, $uid; }
}
return #searched;
}
########################
# generate and return a list of users which match
# many criteria (eg: eyes=2, hair=1, etc)
#
# $users is a reference to an array of user IDs (eg: 000001, 000002, etc)
# $terms is a reference to an array of search terms (eg: $terms[0] = "eyes=2";)
sub search($$) {
my $users = $_[0]; # an array reference
my $terms = $_[1]; # an array reference
my #searched;
my $first = 1;
foreach my $term ( #$terms ) {
# since #$terms is an array of scalars, in the format of 'name=value' pairs
my $name = $term;
$name =~ s/=(.)*//;
my $value = $term;
$value =~ s/$name=//;
if ( $first ) {
# search the given list reference ($users)
#searched = search_item( $users, $name, $value );
$first = 0; # set to 0 cause now we gotta use #searched
}
else {
# otherwise use a reference to #searched
#searched = search_item( \#searched, $name, $value );
}
}
return #searched;
}
my $users = [ keys %users ];
say for search( $users, [ 'eyes=2', 'gender=1' ] );
output
000001
Here's how I would write similar subroutines that behave identically and take the same parameters, but there is a lot in the design of this application that is less that optimal
sub search_item {
my ( $users, $name, $value ) = #_;
grep {
my %ustats = user::getUserStats( $_ );
$ustats{$name} eq $value;
} #$users;
}
sub search {
my ($users, $terms) = #_;
my #searched;
for my $term ( #$terms ) {
my ($name, $value) = split /=/, $term;
#searched = search_item( $users, $name, $value );
$users = \#searched;
}
#searched;
}
but I think user::getUserStats should be called User::get_user_stats (because Perl reserves capital letters for global identifiers such as package names) and it should return a reference to a hash instead of just a list

perl hash with array

I did same hash like this:
my %tags_hash;
Then I iterate some map and add value into #tags_hash:
if (#tagslist) {
for (my $i = 0; $i <= $#tagslist; $i++) {
my %tag = %{$tagslist[$i]};
$tags_hash{$tag{'refid'}} = $tag{'name'};
}}
But I would like to have has with array, so when key exists then add value to array.
Something like this:
e.g. of iterations
1,
key = 1
value = "good"
{1:['good']}
2,
key = 1
value = "bad"
{1:['good', 'bad']}
3,
key = 2
value = "bad"
{1:['good', 'bad'], 2:['bad']}
And then I want to get array from the key:
print $tags_hash{'1'};
Returns: ['good', 'bad']
An extended example:
#!/usr/bin/perl
use strict;
use warnings;
my $hash = {}; # hash ref
#populate hash
push #{ $hash->{1} }, 'good';
push #{ $hash->{1} }, 'bad';
push #{ $hash->{2} }, 'bad';
my #keys = keys %{ $hash }; # get hash keys
foreach my $key (#keys) { # crawl through hash
print "$key: ";
my #list = #{$hash->{$key}}; # get list associate within this key
foreach my $item (#list) { # iterate through items
print "$item ";
}
print "\n";
}
output:
1: good bad
2: bad
So the value of the hash element to be an array ref. Once you have that, all you need to do is push the value onto the array.
$hash{$key} //= [];
push #{ $hash{$key} }, $val;
Or the following:
push #{ $hash{$key} //= [] }, $val;
Or, thanks to autovivification, just the following:
push #{ $hash{$key} }, $val;
For example,
for (
[ 1, 'good' ],
[ 1, 'bad' ],
[ 2, 'bad' ],
) {
my ($key, $val) = #$_;
push #{ $hash{$key} }, $val;
}

Sum all values in each(hash of Array) perl

I have a hash of array named %hash
$VAR1 = {
'reboot' => [
4442,
3483,
541
],
'prod-dev' => [
0,
485,
3421,
242,
425,
425,
484,
1,
244
]
};
How to add all the values in each key and print them all at once like this
reboot : sum
prod-dev : sum
thanks cheers
#!/usr/binkj/perl -w
use strict;
use warnings;
use List::Util qw( sum );
use Data::Dumper;
my ($substr, $new_line);
my #fields;
my %hash =();
open(my kj$logs, ">STDOUT") or die $!;
my ($total_sum, $total_t, $tsum);
my (#array, $key, $val);
while (<STDIN>) {
my #matches;
chomp;
next if $_ =~ m/still logged in/;
next if $_ =~ m/wtmp/;
next if $_ =~ m/\(-\d.+?\)/;
next if $_ =~ m/^$/g;
$_ =~ /(^.*?)\s.*?(\(.*?\))/g;
($key, $val) = ($1,$2);
$val =~ s/(\(|\))//g;
if($val =~ m/\d\+.*/){
$val =~ s/\+/\:/;
#$val =~ m/(^\d.)(:\d.:)(:\d.\s)/g;
my ($days, $hrs, $mins) = $val =~ /^(\d+):(\d+):(\d+).$/g;
$days = $days * 24 * 60;
$hrs = $hrs * 60;
$total_t = $days + $hrs + $mins;
#print $days . ":" . $hrs . ":" . $mins. "\n";
print "$total_t \n";
}else {
my ($hrs, $mins) = $val =~ /^(\d+):(\d+).$/g;
$hrs = $hrs * 60;
$total_t = $hrs + $mins;
print "$total_t \n";
}
push (#{$hash{$key}}, $total_t);
for my $k (keys(%hash)) {
printf("%-8s : %d\n", $k,sum( #{ $hash->{$k} } ),);
}
print Dumper (\%hash);
close $logs;
here is the whole perl program
im really having a hard time solving this one wew I hope you guys can help me
use List::Util qw( sum );
for my $k (keys(%$VAR1)) {
printf("%-8s : %d\n",
$k,
sum( #{ $VAR1->{$k} } ),
);
}
minor but important difference from above answer in accordance with question and without any external module.
foreach $val (keys(%$VAR1)){
$s=0;
foreach ( #{$VAR1->{$val}}){
$s = $_+$s;
}
print "$val :- $s ";
}
I managed to answer it guys, thanks you very much!
foreach my $key ( keys %hash ) {
my $total_sum = 0;
foreach ( #{$hash{$key}} ) {
$total_sum += $_;
}
print $key . "" . $total_sum . "\n";
}
The function sum from List::Util can be used for the sum:
#!/usr/bin/env perl
use strict;
use warnings;
my %hash = (
'reboot' => [ 4442, 3483, 541 ],
'prod-dev' => [ 0, 485, 3421, 242, 425, 425, 484, 1, 244 ],
);
use List::Util qw( sum );
my #sums;
for my $key ( keys %hash ) {
my $sum = sum #{ $hash{$key} };
push #sums, "$key: $sum\n";
}
print #sums;

PERL dynamically match arrays based on unique key

I'm trying to compare 2 huge arrays and want to use map. I am using unique key concept here to match the indexes.
My Arrays:
my #array1 = ( ['a','b','c','d'], ['e','f','g','h'], ['i','j','k','l'], ['m','n','o','p'], ['q','r','s','t']);
my #array2 = ( ['r','q','s','t'], ['b','a','c','d'], ['n','m','o','p'], ['f','e','g','h'], ['j','i','k','l']);
My unique Keys:
my #uk1 = (0,2,3);
my #uk2 = (1,2,3);
These arrays will be huge in size, over 30,000 indexes in each with over 20 elements in each index.
So effectively i create a map where
for ( my $j = 0; $j <= $#array1 ; $j++ )
{
my searchString;
for ( my $k = 0; $k <= $#uk1; $k++ )
{
if ( $k != 0 )
{
$searchString .= ","
}
$my searchString .= $array1[$j][$uk[$k];
}
my #result = map { $_ }
grep { join (",",$array2[$_][1],$array2[$_][2],$array2[$_][3]) ) =~ join(",",$array1[$j][0],$array1[$j][1],$array1[$j][2]) }
0 .. $#array;
}
returns matched indexes.
My problem is, how do i make this dependant on the unique keys? as the length of the unique key will keep changing and as far as i know i cannot dynamically create the $array2[$_] join part.
Hope my question is clear enough.
I want to have the logic that compares
$array1[$uk1[0]],$array1[$uk1[1]],$array1[$uk1[2]] and so on (depending on the number of keys in UK) with
$array2[$uk2[0]],$array2[$uk2[1]],$array2[$uk2[2]].......
Perhaps,
my #array1 = ( ['a','b','c','d'], ['e','f','g','h'], ['i','j','k','l'], ['m','n','o','p'], ['q','r','s','t']);
my #array2 = ( ['r','q','s','t'], ['b','a','c','d'], ['n','m','o','p'], ['f','e','g','h'], ['j','i','k','l']);
my #result;
for my $i (0 .. $#array1) {
push #result,
map { [$i, $_] }
grep {
"#{ $array1[$i] }[1,2,3]" eq "#{ $array2[$_] }[0,2,3]"
}
0 .. $#array2;
}
use Data::Dumper; print Dumper \#result;
output
$VAR1 = [
[
0,
1
],
[
1,
3
],
[
2,
4
],
[
3,
2
],
[
4,
0
]
];
What you want to use is an array slice:
But lets also make life easier:
for my $sample ( #array1 )
{
my $test= join(",", #$sample[#uk1]) ;
my #result = grep { $_ eq $test } map { join(",", #$_[#uk2] ) } #array2 ;
say "huzzah" if #result ;
}
Perl lets you specify multiple elements from an array via the "array slice":
my #list= ('a', 'b', 'c', 'd') ;
my #pieces= #list[1,3] ;

Resources