I want to write a little "DBQuery" function in perl so I can have one-liners which send an SQL statement and receive back and an array of hashes, i.e. a recordset. However, I'm running into an issue with Perl syntax (and probably some odd pointer/reference issue) which is preventing me from packing out the information from the hash that I'm getting from the database. The sample code below demonstrates the issue.
I can get the data "Jim" out of a hash inside an array with this syntax:
print $records[$index]{'firstName'}
returns "Jim"
but if I copy the hash record in the array to its own hash variable first, then I strangely can't access the data anymore in that hash:
%row = $records[$index];
$row{'firstName'};
returns "" (blank)
Here is the full sample code showing the problem. Any help is appreciated:
my #records = (
{'id' => 1, 'firstName' => 'Jim'},
{'id' => 2, 'firstName' => 'Joe'}
);
my #records2 = ();
$numberOfRecords = scalar(#records);
print "number of records: " . $numberOfRecords . "\n";
for(my $index=0; $index < $numberOfRecords; $index++) {
#works
print 'you can print the records like this: ' . $records[$index]{'firstName'} . "\n";
#does NOT work
%row = $records[$index];
print 'but not like this: ' . $row{'firstName'} . "\n";
}
The nested data structure contains a hash reference, not a hash.
# Will work (the -> dereferences the reference)
$row = $records[$index];
print "This will work: ", $row->{firstName}, "\n";
# This will also work, by promoting the hash reference into a hash
%row = %{ $records[$index] };
print "This will work: ", $row{firstName}, "\n";
If you're ever presented with a deep Perl data structure, you may profit from printing it using Data::Dumper to print it into human-readable (and Perl-parsable) form.
The array of hashes doesn't actually contain hashes, but rather an references to a hash.
This line:
%row = $records[$index];
assigns %row with one entry. The key is the scalar:
{'id' => 1, 'firstName' => 'Jim'},
Which is a reference to the hash, while the value is blank.
What you really want to do is this:
$row = $records[$index];
$row->{'firstName'};
or else:
$row = %{$records[$index];}
$row{'firstName'};
Others have commented on hashes vs hashrefs. One other thing that I feel should be mentioned is your DBQuery function - it seems you're trying to do something that's already built into the DBI? If I understand your question correctly, you're trying to replicate something like selectall_arrayref:
This utility method combines "prepare", "execute" and "fetchall_arrayref" into a single call. It returns a reference to an array containing a reference to an array (or hash, see below) for each row of data fetched.
To add to the lovely answers above, let me add that you should always, always, always (yes, three "always"es) use "use warnings" at the top of your code. If you had done so, you would have gotten the warning "Reference found where even-sized list expected at -e line 1."
what you actually have in your array is a hashref, not a hash. If you don't understand this concept, its probably worth reading the perlref documentation.
to get the hash you need to do
my %hash = %{#records[$index]};
Eg.
my #records = (
{'id' => 1, 'firstName' => 'Jim'},
{'id' => 2, 'firstName' => 'Joe'}
);
my %hash = %{$records[1]};
print $hash{id}."\n";
Although. I'm not sure why you would want to do this, unless its for academic purposes. Otherwise, I'd recommend either fetchall_hashref/fetchall_arrayref in the DBI module or using something like Class::DBI.
Also note a good perl idiom to use is
for my $rowHR ( #records ) {
my %row = %$rowHR;
#or whatever...
}
to iterate through the list.
Related
I currently have an referenced hash and an array of keys that the hash contains. I want to get an array of the values corresponding to my array of keys.
I know how to do this in multiple lines:
# Getting hash reference and array of keys.
my $hashRef = {
one => 'foo',
two => 'bar',
three => 'baz'
};
my #keys = ('one', 'three');
# Getting corresponding array of values.
my #values;
foreach my $key (#keys) {
push #values, $hashRef->{$key};
}
However, I believe that there must be a much better way that doesn't make use of a loop. But unfortunately I just can't figure it out. How can I efficiently
get an array of values from a referenced hash and an array of keys; ideally in one line if possible?
Easily:
my #values = #$hashRef{#keys};
Or, on Perl 5.24+:
my #values = $hashRef->#{#keys};
Or, on Perl 5.20+ by enabling some additional features:
use feature qw(postderef);
no warnings qw(experimental::postderef);
my #values = $hashRef->#{#keys};
This takes advantage of the fact that you can get the values for multiple keys (a "slice") of a %hash with the #hash{LIST} syntax. You just have to dereference it first. See perldoc for more information.
I've come across an obstacle in one of my perl scripts that I've managed to solve, but I don't really understand why it works the way it works. I've been scouring the internet but I haven't found a proper explanation.
I have a subroutine that returns a reference to a hash of arrays. The hash keys are simple strings, and the values are references to arrays.
I print out the elements of the array associated with each key, like this
for my $job_name (keys %$build_numbers) {
print "$job_name => ";
my #array = #{#$build_numbers{$job_name}}; # line 3
for my $item ( #array ) {
print "$item \n";
}
}
While I am able to print out the keys & values, I don't really understand the syntax behind line 3.
Our data structure is as follows:
Reference to a hash whose values are references to the populated arrays.
To extract the elements of the array, we have to:
- dereference the hash reference so we can access the keys
- dereference the array reference associated to a key to extract elements.
Final question being:
When dealing with perl hashes of hashes of arrays etc; to extract the elements at the "bottom" of the respective data structure "tree" we have to dereference each level in turn to reach the original data structures, until we obtain our desired level of elements?
Hopefully somebody could help out by clarifying.
Line 3 is taking a slice of your hash reference, but it's a very strange way to do what you're trying to do because a) you normally wouldn't slice a single element and b) there's cleaner and more obvious syntax that would make your code easier to read.
If your data looks something like this:
my $data = {
foo => [0 .. 9],
bar => ['A' .. 'F'],
};
Then the correct version of your example would be:
for my $key (keys(%$data)) {
print "$key => ";
for my $val (#{$data->{$key}}) {
print "$val ";
}
print "\n";
}
Which produces:
bar => A B C D E F
foo => 0 1 2 3 4 5 6 7 8 9
If I understand your second question, the answer is that you can access precise locations of complex data structures if you use the correct syntax. For example:
print "$data->{bar}->[4]\n";
Will print E.
Additional recommended reading: perlref, perlreftut, and perldsc
Working with data structures can be hard depending on how it was made.
I am not sure if your "job" data structure is exactly this but:
#!/usr/bin/env perl
use strict;
use warnings;
use diagnostics;
my $hash_ref = {
job_one => [ 'one', 'two'],
job_two => [ '1','2'],
};
foreach my $job ( keys %{$hash_ref} ){
print " Job => $job\n";
my #array = #{$hash_ref->{$job}};
foreach my $item ( #array )
{
print "Job: $job Item $item\n";
}
}
You have an hash reference which you iterate the keys that are arrays. But each item of this array could be another reference or a simple scalar.
Basically you can work with the ref or undo the ref like you did in the first loop.
There is a piece of documentation you can check for more details here.
So answering your question:
Final question being: - When dealing with perl hashes of hashes of
arrays etc; to extract the elements at the "bottom" of the respective
data structure "tree" we have to dereference each level in turn to
reach the original data structures, until we obtain our desired level
of elements?
It depends on how your data structure was made and if you already know what you are looking for it would be simple to get the value for example:
%city_codes = (
a => 1, b => 2,
);
my $value = $city_codes{a};
Complex data structures comes with complex code.
In Perl, I've always been confused about how to cleanly assign multiple local variables from array entries.
I use the following syntax in subs all the time, so I'm somewhat familiar with it:
my ($var1, $var2) = #_
but other variations of this confuse me. For instance, I have the following code that works:
for my $ctr (0 .. $#matchingLines) {
my $lineNo = $matchingLines[$ctr][0];
my $text = $matchingLines[$ctr][1];
Where "#matchingLines" is an array of two-element arrays.
I wish I could convert the last two lines to the obvious:
my ($lineNo, $text) = $matchingLines[$ctr];
That of course does not work. I've tried numerous variations, but I can't find anything that works.
Just dereference the array ref:
my ( $lineNo, $text ) = #{ $matchingLines[$ctr] };
Check out Perl Data Structures Cookbook for additional examples.
It sounds like you have an array of arrays. this means that the inner arrays will be array references. If you want to allocate them to vars then you need to derference them.
use strict;
use warnings;
my #matchingLines = (['y','z'],['a','b']);
for my $ctr (0 .. $#matchingLines) {
my ($lineNo, $text) = #{$matchingLines[$ctr]};
print "#Array index: $ctr - lineno=$lineNo - text=$text\n"
}
this produces the output
#Array index: 0 - lineno=y - text=z
#Array index: 1 - lineno=a - text=b
Perhaps I have made this more complicated than I need it to be but I am currently trying to store an array that contains, among other things, an array inside a hash in Perl.
i.e. hash -> array -> array
use strict;
my %DEVICE_INFORMATION = {}; #global hash
sub someFunction() {
my $key = 'name';
my #storage = ();
#assume file was properly opened here for the foreach-loop
foreach my $line (<DATA>) {
if(conditional) {
my #ports = ();
$storage[0] = 'banana';
$storage[1] = \#ports;
$storage[2] = '0';
$DEVICE_INFORMATION{$key} = \#storage;
}
elsif(conditional) {
push #{$DEVICE_INFORMATION{$key}[1]}, 5;
}
}#end foreach
} #end someFunction
This is a simplified version of the code I am writing. I have a subroutine that I call in the main. It parses a very specifically designed file. That file guarantees that the if statement fires before subsequent elsif statement.
I think the push call in the elsif statement is not working properly - i.e. 5 is not being stored in the #ports array that should exist in the #storage array that should be returned when I hash the key into DEVICE_INFORMATION.
In the main I try and print out each element of the #storage array to check that things are running smoothly.
#main execution
&someFunction();
print $DEVICE_INFORMATION{'name'}[0];
print $DEVICE_INFORMATION{'name'}[1];
print $DEVICE_INFORMATION{'name'}[2];
The output for this ends up being... banana ARRAY(blahblah) 0
If I change the print statement for the middle call to:
print #{$DEVICE_INFORMATION{'name'}[1]};
Or to:
print #{$DEVICE_INFORMATION{'name'}[1]}[0];
The output changes to banana [blankspace] 0
Please advise on how I can properly update the #ports array while it is stored inside the #storage array that has been hash'd into DEVICE_INFORMATION and then how I can access the elements of #ports. Many thanks!
P.S. I apologize for the length of this post. It is my first question on stackoverflow.
I was going to tell you that Data::Dumper can help you sort out Perl data structures, but Data::Dumper can also tell you about your first problem:
Here's what happens when you sign open-curly + close-curly ( '{}' ) to a hash:
use Data::Dumper ();
my %DEVICE_INFORMATION = {}; #global hash
print Dumper->Dump( [ \%DEVICE_INFORMATION ], [ '*DEVICE_INFORMATION ' ] );
Here's the output:
%DEVICE_INFORMATION = (
'HASH(0x3edd2c)' => undef
);
What you did is you assigned the stringified hash reference as a key to the list element that comes after it. implied
my %DEVICE_INFORMATION = {} => ();
So Perl assigned it a value of undef.
When you assign to a hash, you assign a list. A literal empty hash is not a list, it's a hash reference. What you wanted to do for an empty hash--and what is totally unnecessary--is this:
my %DEVICE_INFORMATION = ();
And that's unnecessary because it is exactly the same thing as:
my %DEVICE_INFORMATION;
You're declaring a hash, and that statement fully identifies it as a hash. And Perl is not going to guess what you want in it, so it's an empty hash from the get-go.
Finally, my advice on using Data::Dumper. If you started your hash off right, and did the following:
my %DEVICE_INFORMATION; # = {}; #global hash
my #ports = ( 1, 2, 3 );
# notice that I just skipped the interim structure of #storage
# and assigned it as a literal
# * Perl has one of the best literal data structure languages out there.
$DEVICE_INFORMATION{name} = [ 'banana', \#ports, '0' ];
print Data::Dumper->Dump(
[ \%DEVICE_INFORMATION ]
, [ '*DEVICE_INFORMATION' ]
);
What you see is:
%DEVICE_INFORMATION = (
'name' => [
'banana',
[
1,
2,
3
],
'0'
]
);
So, you can better see how it's all getting stored, and what levels you have to deference and how to get the information you want out of it.
By the way, Data::Dumper delivers 100% runnable Perl code, and shows you how you can specify the same structure as a literal. One caveat, you would have to declare the variable first, using strict (which you should always use anyway).
You update #ports properly.
Your print statement accesses $storage[1] (reference to #ports) in wrong way.
You may use syntax you have used in push.
print $DEVICE_INFORMATION{'name'}[0], ";",
join( ':', #{$DEVICE_INFORMATION{'name'}[1]}), ";",
$DEVICE_INFORMATION{'name'}[2], "\n";
print "Number of ports: ", scalar(#{$DEVICE_INFORMATION{'name'}[1]})),"\n";
print "First port: ", $DEVICE_INFORMATION{'name'}[1][0]//'', "\n";
# X//'' -> X or '' if X is undef
I'm just starting out Perl (about 15 minutes ago) using a tutorial online. I made a small subroutine to test a few Perl features and would like to know if it is possible to determine at runtime if parameters that were passed to the sub' call are arrays or scalars. Let's use the sub I wrote as an example:
#!/usr/bin/perl
sub somme_prod {
if (scalar(#_) > 1) {
$facteur = shift(#_);
foreach my $nb (#_) {
$resultat += $nb
}
return ($resultat * $facteur);
}
else {
return "ERREUR";
}
}
print somme_prod(2, 2, 3, 7);
This is a basic sum-product subroutine which does exactly what its name says. Now, would it be possible to modify this subroutine to allow for a mix of arrays and scalars like this ?
somme_prod(2, (2,3), 7);
somme_prod(2, (2,3,7));
#...
Also, any comment on the style of Perl coding demonstrated here is much welcome. I have a background of amateur C++ coding so I may not be thinking in Perl.
Edit: I'm so sorry. I actually tried it after posting and it seems that Perl does process my sub as I want it to. Now I guess my question would be more "how does Perl know how to process this" ?
Edited code for a more Perl-ish version.
Yes; in Perl you can create references to arrays (or hashes, or anything else) to stuff several values into a single parameter.
For example:
somme_prod(2, [2, 3], 7);
...would resolve to:
sub somme_prod {
foreach my $arg (#_) {
if (ref($arg) eq 'ARRAY') {
my #values = #$arg; # dereference, e.g. [2, 3] -> (2, 3)
. . .
} else {
# single value, e.g. "2" or "7"
}
}
}
You can read the page perldoc perlref to learn all about references.
Perl handles lists and arrays differently, and a useful document for you to read is What is the difference between a list and an array?
Perl will always flatten nested lists (and so arrays within lists) so
my #data1 = (2, (2, 3), 7);
or
my #data2 = (2, 3);
my #data1 = (2, #data2, 7);
is equivalent to
my #data1 = (2, 2, 3, 7);
As Kevin says, if you want nested arrays you have to place an array reference in the place where the sublist appears. Because a reference is a scalar it won't get expanded.
Your subroutine is fine, but using some de-facto standards would help others to follow your program. Firstly the convention is that a subroutine will return undef if there is an error, so that you can write
sous_routine($p1, $p2) or die "Erreur";
In this case the possibility that zero is a valid result spoils this, but it is still best to stick to the rules. A plain return without a parameter indicates an error
A little bit of tidying up and using unless and if as statement modifiers gives this
sub somme_prod {
return unless #_ > 1;
my $facteur = shift;
my $somme = 0;
$somme += $_ for #_;
return $somme * $facteur;
}
print somme_prod(2, 2, 3, 7);
You've known Perl for 15 minutes? Forget about references for now.
Basically, everything passed to a subroutine is an array. In fact, it's stored in an array called #_.
# /usr/bin/env perl
use strict; #ALWAYS USE!
use warnings; #ALWAYS USE!
my #array = qw(member1 member2 member3 member4);
foo(#array, 'scalar', 'scalar', 'scalar');
sub foo {
print "My input is " . join (":", #_) . "\n";
This will print:
my input is member1:member2:member3:member4:scalar:scalar:scalar
There is no way to tell which entries are from an array and which are from a scalar. As far as your subroutine is concerned, they're all members of the array #_.
By the way, Perl comes with a command called perldoc. When someone says to see perlref, you can type in perldoc perlref at the command line and see the documentation. You can also go to the site http://perldoc.perl.org which will also contain the same information found in the perldoc command.
Now about references....
A data element of a Perl array or the value of a hash can only contain a single value. That could be a string, it could be a real number, it could be an integer, and it could be a reference to another Perl data structure. That's where all the fun and money is.
For example, the same subroutine foo above could have taken this information:
foo(\#array, 'scalar', 'scalar', 'scalar'); #Note the backslash!
In this case, you're not passing in the values of #array into foo. Instead, a reference to the array is passed as the first data element of #_. If you attempted to print out $_[0], you'd get something like ARRAY:6E43832 which says the data element is an array and where it's located in memory.
Now, you can use the ref function to see whether an piece of data is a reference and the type of reference it is:
sub foo {
foreach my $item (#_) {
if (ref $item eq 'ARRAY') {
print "This element is a reference to an array\n";
}
elsif (ref $item eq 'HASH') {
print "This element is a reference to a hash\n";
}
elsif (ref $item) { #Mysterious Moe Reference
print "This element is a reference to a " . lc (ref $item) . "\n";
}
else {
print "This element is a scalar and it's value is '$item'\n";
}
}
}
Of course, your reference to an array might be an array that contains references to hashes that contain references to arrays and so on. There's a module that comes with Perl called Data::Dumper (you can use perldoc to see information about it) that will print out the entire data structure.
This is how object orient Perl works, so it's really quite common to have references to other Perl data structures embedded in a piece of Perl data.
Right now, just get use to basic Perl and how it works. Then, start looking at the various tutorials about Perl references in Perldoc.