Directly access nested JSON data in perl? - arrays

I'm not familiar with hash/reference syntax with Perl and it makes my eyes hurt trying.
I have the following JSON:
{
"Arg":"Custom_Light state alias protocol",
"Results": [
{
"Name":"Custom_Light",
"Internals": { },
"Readings": {
"protocol": { "Value":"V3", "Time":"2017-01-14 18:49:18" },
"state": { "Value":"off", "Time":"2017-03-05 10:39:50" }
},
"Attributes": { "alias": "Kitchen light" }
} ],
"totalResultsReturned":1
}
How do I directly get the Reading > Protocol Value and Reading > state Value as well as the Attributes > Alias?
I am using the default JSON encoder/decoder and it works splendid. Using Dumper($json) I get all the JSON, but I have no clue how to directly access it without using foreach with all the arrays within arrays in this.
I have tried the following:
my $json = from_json( $readout, { utf8 => 1 } );
print "No. Entries:", scalar(keys($json)); #works, returns 3
my #results = %$json{Results};
Dumper(#results[1]); #I get the Results array
From here it already is ugly. What's that %$ doing there? I thought I could do something like print ${ $json->{'Results'}->[1] }{'Readings'}; but that leads me nowhere.
Give me wisdom. How do I access the Protocol value directly? How do I access the state value directly? And finally, how to get to the alias Attribute?
I don't know what I'm doing but I'm getting somewhere with my $test = %{${%$json{Results}}[0]}{Name}; #I get "Custom_Light", nice. Is this the way to go with a gazillion of weird % and $ just randomly thrown in?

You want
$json->{Results}[0]{Readings}{protocol}{Value}
$json->{Results}[0]{Readings}{state}{Value}
$json->{Results}[0]{Attributes}{alias}
However, since the Results item is an array, you are likely to want to iterate over all of its elements, although in this case there is only one element
I find it useful to extract one level of reference at a time into temporary variables. It would look like this
my $results = $json->{Results};
for my $result ( #$results ) {
my $readings = $result->{Readings};
my $attributes = $result->{Attributes};
printf "Protocol: %s\n", $readings->{protocol}{Value};
printf "State: %s\n", $readings->{state}{Value};
printf "Alias: %s\n", $attributes->{alias};
print "\n";
}

Have a look at perlreftut, perldsc, and perlref, it will help you understand how to access deeply nested structures in Perl.
print "No. Entries:", scalar(keys($json)); #works, returns 3
Actually, this will no longer work. Using keys on a scalar, was an experimental feature added in Perl 5.14 that allowed each, keys, push, pop, shift, splice, unshift, and values to be called with a scalar argument. This experiment was considered unsuccessful, and was removed in 5.23. See also Experimental values on scalar is now forbidden. So, you should dereference the hash reference $json before applying keys:
print "No. Entries:", scalar keys %$json;
As described in perlref, %$ref dereferences the hash reference $ref. Next, lets look at this line:
my #results = %$json{Results};
This actually first creates a new (anonymous) hash ( Result => $json->{Result} ) and then assigns this to #results making #result = ( 'Result', $json->{Result} ). So that is why you now can refer to $json->{Result}[0] as $result[1].
But this is obscure coding, and probably not intended as well. So to return to your question, to get the Value field you could write:
my $value = $json->{Results}[0]{Readings}{state}{Value};
And to get the alias field:
my $alias = $json->{Results}[0]{Attributes}{alias};

Related

What is the difference between {} and ->{} in Perl?

So if I have some data object and I want to access whats inside the element of that object
Whats the difference between
$Data{isEnabled})
$Data->{isEnabled}
my data is basically like this
for my $characterData (#{$AllCharacters->{'characters'}}) {
$Data{isEnabled})
$Data->{isEnabled}
and i want to access certain elements of my characterData but I'm not sure when to use
$Data{isEnabled})
vs
$Data->{isEnabled}
Like for example why does the first print work but the second fails?
use strict;
use warnings;
my %info = (NAME => "John", HOST => "Local", PORT => 80);
print $info{PORT};
print $info->{PORT};
The first expression accesses a key within a hash:
my %data = (is_enabled => 1);
print $data{is_enabled}), "\n";
In the second expression, data is not a hash, but a *hash reference. It would typically be declared as:
my $data = { is_enabled => 1 };
Since this is a reference, we need to use the dereferencing operator (->) to access the hash content:
print $data->{is_enabled}, "\n";
If you are iterating through an array of hashes, as your code seems to show, then each element is a hash reference. You need to use the second syntax:
my #all_data = ( { is_enabled => 1 }, { is_enabled => 0 } );
for my $data (#all_data) {
print $data->{is_enabled}, "\n";
}
You can read more about references in the perlref documentation page.
$Data->{isEnabled}
is equivalent to
${ $Data }{isEnabled}
I prefer the "arrow" notation, but it serves my explanation better to compare
$Data{isEnabled}
with
${ $Data }{isEnabled}
In the first case ($Data{isEnabled}), we are accessing an element of a hash %Data.
In the second case, we also appear to have a hash lookup, but we have a block ({ $Data }) where a name would normally be expected. It is indeed a hash lookup, but instead of accessing a named hash, we are accessing a referenced hash. The block (or the expression to the left of the ->) is expected to return a reference to the hash the program should access.
A reference is a means of referencing a variable through it's location in memory rather than by its name. Consider the following example:
my $ref;
if (condition()) {
$ref = \%hash1;
} else {
$ref = \%hash2;
}
say $ref->{id};
This will print $hash1{id} or $hash2{id} based on whether condition() returns a true value or not.

Read MongoDB array into perl and walk data

I am trying to capture an array from my MongoDB database into my Perl script and read each element. This is something that I thought would be simple, but for some dumb reason it is kicking my rearend.
My MongoDB Document (in part)
"members" : [
"5713b2d46d210e51836de591",
"me",
"you",
"him",
"her"
],
Perl code
$document = $database -> get_collection('my_collection')->find_one({_id => $oid});
#members = $document->{'members'};
print Dumper #members;
foreach $member (#members)
{
print "member = $member\n";
}
exit;
Output I am getting:
$VAR1 = [
'5713b2d46d210e51836de591',
'me',
'you',
'him',
'her'
];
member = ARRAY(0x47fa398)
Looking at the last line I see that I am being passed a reference to the array instead of the values. So I tried accessing via $member[0] or $member[1] but that just returns the same ARRAY(0x*****).
PLEASE HELP, I am sure it is something stupid.
Thanks!
Steven
I'm not familiar with Mongo, but looking at the output, your #members array has one element - an array ref (as you suspected). Since Mongo is returning an arrayref, you're best to store that in a scalar and access it like so;
my $members = $document->{'members'};
print "second item returned is: ", $members->[1];
print "The complete contents:\n";
for my $item ( #$members ) {
print " ", $item;
}

Hash in array in a hash

I'm trying to identify the output of Data::Dumper, it produces the output below when used on a hash in some code I'm trying to modify:
print Dumper(\%unholy_horror);
$VAR1 = {
'stream_details' => [
{
'file_path' => '../../../../tools/test_data/',
'test_file' => 'test_file_name'
}
]
};
Is this a hash inside an array inside a hash? If not what is it? and what is the syntax to access the "file path" and "test_file" keys, and their values.
I want to iterate over that inner hash like below, how would I do that?
while ( ($key, $value) = each %hash )
{
print "key: $key, value: $hash{$key}\n";
}
You're correct. It's a hash in an array in a hash.
my %top;
$top{'stream_details'}[0]{'file_path'} = '../../../../tools/test_data/';
$top{'stream_details'}[0]{'test_file'} = 'test_file_name';
print Dumper \%top;
You can access the elements as above, or iterate with 3 levels of for loop - assuming you want to iterate the whole thing.
foreach my $topkey ( keys %top ) {
print "$topkey\n";
foreach my $element ( #{$top{$topkey}} ) {
foreach my $subkey ( keys %$element ) {
print "$subkey = ",$element->{$subkey},"\n";
}
}
}
I would add - sometimes you get some quite odd seeming hash topologies as a result of parsing XML or JSON. It may be worth looking to see if that's what's happening, because 'working' with the parsed object might be easier.
The above might be the result of:
#JSON
{"stream_details":[{"file_path":"../../../../tools/test_data/","test_file":"test_file_name"}]}
Or something similar from an API. (I think it's unlikely to be XML, since XML doesn't implicitly have 'arrays' in the way JSON does).

hash inside perl structures

I try to create an array of perl structures. Each struct contains two scalars and a hash.
Later I want to find an item of the array, get the hash and find a scalar inside the hash.
I can find the item inside the array and get the scalars.
But I don't know hot to correctly get the hash and a value inside it.
I tried with/without reference.
Thanks a lot
#hash
%latestInfo = (
8 => '30',
);
#struct
package Myobj;
use Class::Struct;
struct( name => '$', majorVer => '$', latestVer => '%');
$w1 = new Myobj;
$w1->name('test');
$w1->majorVer(5);
$w1->latestVer($latestInfo);
#array with all version information
#versions=($w1, ...);
sub getVersionFromMajor
{
foreach $ver (#versions) {
if ($ver->majorVer eq $_[0]) {
return $ver;
}
}
}
#
#main
#
#ok: get version info from structures/array
local($ver) = getVersionFromMajor(5);
local($n) = $ver->name;
#fail: get hash inside item
my $latest = \$ver->latestVer;
%lat = $ver->latestVer;
#fail: get value inside hash
local($m) = $latest{8};
This bit:
$w1->latestVer($latestInfo);
Should be:
$w1->latestVer(\%latestInfo);
%latestInfo and $latestInfo are two unrelated variables - %latestInfo is your hash, and $latestInfo is an undeclared (and thus undef) scalar. \%latestInfo is a scalar reference to %latestInfo, which is what the latestVer method (created by Class::Struct) wants you to give it.
Perl would have told you about $latestInfo not existing if you'd done use strict and declared all your variables.
Also, this bit:
%lat = $ver->latestVer;
Should be:
%lat = %{ $ver->latestVer };

How do I consolidate a hash in Perl?

I have an array of hash references. The hashes contain 2 keys, USER and PAGES. The goal here is to go through the array of hash references and keep a running total of the pages that the user printed on a printer (this comes from the event logs). I pulled the data from an Excel spreadsheet and used regexes to pull the username and pages. There are 182 rows in the spreadsheet and each row contains a username and the number of pages they printed on that job. Currently the script can print each print job (all 182) with the username and the pages they printed but I want to consolidate this down so it will show: username 266 (i.e. just show the username once, and the total number of pages they printed for the whole spreadsheet.
Here is my attempt at going through the array of hash references, seeing if the user already exists and if so, += the number of pages for that user into a new array of hash references (a smaller one). If not, then add the user to the new hash ref array:
my $criteria = "USER";
my #sorted_users = sort { $a->{$criteria} cmp $b->{$criteria} } #user_array_of_hash_refs;
my #hash_ref_arr;
my $hash_ref = \#hash_ref_arr;
foreach my $index (#sorted_users)
{
my %hash = (USER=>"",PAGES=>"");
if(exists $index{$index->{USER}})
{
$hash{PAGES}+=$index->{PAGES};
}
else
{
$hash{USER}=$index->{USER};
$hash{PAGES}=$index->{PAGES};
}
push(#hash_ref_arr,{%hash});
}
But it gives me an error:
Global symbol "%index" requires explicit package name at ...
Maybe my logic isn't the best on this. Should I use arrays instead? It seems as though a hash is the best thing here, given the nature of my data. I just don't know how to go about slimming the array of hash refs down to just get a username and the total pages they printed (I know I seem redundant but I'm just trying to be clear). Thank you.
my %totals;
$totals{$_->{USER}} += $_->{PAGES} for #user_array_of_hash_refs;
And then, to get the data out:
print "$_ : $totals{$_}\n" for keys %totals;
You could sort by usage too:
print "$_ : $totals{$_}\n" for sort { $totals{$a} <=> $totals{$b} } keys %totals;
As mkb mentioned, the error is in the following line:
if(exists $index{$index->{USER}})
However, after reading your code, your logic is faulty. Simply correcting the syntax error will not provide your desired results.
I would recommend skipping the use of temporary hash within the loop. Just work with the a results hash directly.
For example:
#!/usr/bin/perl
use strict;
use warnings;
my #test_data = (
{ USER => "tom", PAGES => "5" },
{ USER => "mary", PAGES => "2" },
{ USER => "jane", PAGES => "3" },
{ USER => "tom", PAGES => "3" }
);
my $criteria = "USER";
my #sorted_users = sort { $a->{$criteria} cmp $b->{$criteria} } #test_data;
my %totals;
for my $index (#sorted_users) {
if (not exists $totals{$index->{USER}}) {
# initialize total for this user
$totals{$index->{USER}} = 0;
}
# add to user's running total
$totals{$index->{USER}} += $index->{PAGES}
}
print "$_: $totals{$_}\n" for keys %totals;
This produces the following output:
$ ./test.pl
jane: 3
tom: 8
mary: 2
The error comes from this line:
if(exists $index{$index->{USER}})
The $ sigil in Perl 5 with {} after the name means that you are getting a scalar value out of a hash. There is no hash declared by the name %index. I think that you probably just need to add a -> operator so the problem line becomes:
if(exists $index->{$index->{USER}})
but not having the data makes me unsure.
Also, good on you for using use strict or you would be instantiating the %index hash silently and wondering why your results didn't make any sense.
my %total;
for my $name_pages_pair (#sorted_users) {
$total{$name_pages_pair->{USER}} += $name_pages_pair->{PAGES};
}
for my $username (sort keys %total) {
printf "%20s %6u\n", $username, $total{$username};
}

Resources