perl -> Parsing json string of hash - arrays

so I took a curl output of a json formatted output and assigned it to a variable.
my $catcherJSON = decode_json $response->content;
print Dumper $catcherJSON;
When I view the $catcherJSON, I want to see an array of hashes, but I just get a massive string of json.
$VAR1 = [
{
'priority' => 5,
'ingestPath' => '/vg/24CHVOD_en',
...
...
...
},
{
'priority' => 5,
'ingestPath' =>
...
...
},
{
'priority' => 5,
'ingestPath' =>
...
...
},
{
'priority' => 5,
'ingestPath' =>
...
...},
{
'priority' => 5,
'ingestPath' =>
....
...
}
];
The hashes repeat (There are about 300 unique results) and I'm trying to figure out in perl how I can split this 1 string into my array of hashes simply.
Any way that I can easily convert this into a proper array of hashes so I can iterate through it?
Thanks in advance.

Any way that I can easily convert this into a proper array of hashes so I can iterate through it?
You've already done it.
so I took a curl output of a json formatted output and assigned it to a variable.
my $catcherJSON = decode_json $response->content;`
That's not correct.
You took the JSON formatted string from $response->content, decoded the JSON into a Perl data structure, and stored it in $catcherJSON.
When I view the $catcherJSON, I want to see an array of hashes, but I just get a massive string of json.
$catcherJSON is not a string of JSON. $catcherJSON is a Perl array of hashes. You've already decoded the JSON into a Perl data structure with decode_json. It's been formatted by Dumper() back into Perl code so you can read it.
Your code is fine, you have what you want. For example, $catcherJSON->[0]{ingestPath} will be '/vg/24CHVOD_en'.

my $catcherJSON = $response->content;
$catcherJSON =~ s/\n//g;
$catcherJSON = decode_json($catcherJSON);
This was my eventual solution.
When the curl response comes in, all json has line breaks. As soon as line breaks are removed, my decode_json will now refence all as $catcherJSON->[0]{id}, etc...
Thanks everyone for the help here.
Much appreciated.

Related

Read MongoDB array into perl and walk data

I am trying to capture an array from my MongoDB database into my Perl script and read each element. This is something that I thought would be simple, but for some dumb reason it is kicking my rearend.
My MongoDB Document (in part)
"members" : [
"5713b2d46d210e51836de591",
"me",
"you",
"him",
"her"
],
Perl code
$document = $database -> get_collection('my_collection')->find_one({_id => $oid});
#members = $document->{'members'};
print Dumper #members;
foreach $member (#members)
{
print "member = $member\n";
}
exit;
Output I am getting:
$VAR1 = [
'5713b2d46d210e51836de591',
'me',
'you',
'him',
'her'
];
member = ARRAY(0x47fa398)
Looking at the last line I see that I am being passed a reference to the array instead of the values. So I tried accessing via $member[0] or $member[1] but that just returns the same ARRAY(0x*****).
PLEASE HELP, I am sure it is something stupid.
Thanks!
Steven
I'm not familiar with Mongo, but looking at the output, your #members array has one element - an array ref (as you suspected). Since Mongo is returning an arrayref, you're best to store that in a scalar and access it like so;
my $members = $document->{'members'};
print "second item returned is: ", $members->[1];
print "The complete contents:\n";
for my $item ( #$members ) {
print " ", $item;
}

Accessing returned values as an array

I have simple XML that I want to read in Perl and make hash containing all read keys.
Consider this code:
my $content = $xml->XMLin("filesmap.xml")->{Item};
my %files = map { $_->{Path} => 1 } #$content;
This snippet works great when XML file contains many Item tags. Then $content is a reference to array. But when there is only one Item I get an error when dereferencing as array. My assumption is that $content is a reference to the scalar, not array.
What is the practice to make sure I get array of values read from XML?
What you need is to not use XML::Simple and then it's really trivial. My favourite for fairly straightforward XML samples is XML::Twig
use XML::Twig;
my $twig = XML::Twig -> new -> parsefile ( 'filesmap.xml' );
my #files = map { $_ -> trimmed_text } $twig -> get_xpath ( '//Path' );
With a more detailed XML sample (and desired result) I'll be able to give you a better answer.
Part of the problem with XML::Simple is it tries to turn an XML data structure into perl data structures, and - because hashes are key-values and unordered but arrays are ordered, it has to guess. And sometimes it does it wrong, and other times it's inconsistent.
If you want it to be consistent, you can set:
my $xml = XMLin( "filesmap.xml", ForceArray => 1, KeyAttr => [], ForceContent => 1 );
But really - XML::Simple is just a route to pain. Don't use it. If you don't like XML::Twig, try XML::LibXML instead.
What I would say you need is a flatten-ing step.
my %files
= map { $_->{Path} => 1 }
# flatten...
map { ref() eq 'ARRAY' ? #$_ : $_ }
$xml->XMLin("filesmap.xml")->{Item}
;
You can do a check and force the return into an array reference if necessary:
my $content = $xml->XMLin("filesmap.xml")->{Item};
$content = ref $content eq 'ARRAY'
? $content
: [$content];

Json creation in ruby for a list

I am new to Ruby.
I want to create a JSON file for a group of elements.
For this, I am using eachfunction to retrieve the datas. I want to create json as follows for the 4 length array,
'{
"desc":{
"1":"1st Value",
"2":"2nd value"
"3":"3rd Value",
"4":"4th value"
},
}'
This is my array iteration,
REXML::XPath.each( doc, "//time" ) { |element1|
puts element1.get_text
}
I know here is the simple code to generate a JSON,
require 'json/add/core'
class Item < Struct.new(:id, :name); end
chair = Item.new(1, 'chair')
puts JSON.pretty_generate(chair)
This syntax will generate a json as follows,
{
"json_class": "Item",
"v": [
1,
"chair"
]
}
But I'm not sure how to do that to make JSON for my elements as stated above. Google search didn't give me a proper way to do this.
Can anyone help me here?
it means this?:
require 'json'
my_arr= ["1st Value","2nd Value","3rd Value","4th Value"]
tmp_str= {}
tmp_str["desc"] = {}
my_arr.each do |x|
tmp_str["desc"]["#{x[0]}"] = x
end
puts JSON.generate(tmp_str)
you can iterate the string array ,then take the strings to hash object.JSON can easy to parse Hash objcect .

Hash in array in a hash

I'm trying to identify the output of Data::Dumper, it produces the output below when used on a hash in some code I'm trying to modify:
print Dumper(\%unholy_horror);
$VAR1 = {
'stream_details' => [
{
'file_path' => '../../../../tools/test_data/',
'test_file' => 'test_file_name'
}
]
};
Is this a hash inside an array inside a hash? If not what is it? and what is the syntax to access the "file path" and "test_file" keys, and their values.
I want to iterate over that inner hash like below, how would I do that?
while ( ($key, $value) = each %hash )
{
print "key: $key, value: $hash{$key}\n";
}
You're correct. It's a hash in an array in a hash.
my %top;
$top{'stream_details'}[0]{'file_path'} = '../../../../tools/test_data/';
$top{'stream_details'}[0]{'test_file'} = 'test_file_name';
print Dumper \%top;
You can access the elements as above, or iterate with 3 levels of for loop - assuming you want to iterate the whole thing.
foreach my $topkey ( keys %top ) {
print "$topkey\n";
foreach my $element ( #{$top{$topkey}} ) {
foreach my $subkey ( keys %$element ) {
print "$subkey = ",$element->{$subkey},"\n";
}
}
}
I would add - sometimes you get some quite odd seeming hash topologies as a result of parsing XML or JSON. It may be worth looking to see if that's what's happening, because 'working' with the parsed object might be easier.
The above might be the result of:
#JSON
{"stream_details":[{"file_path":"../../../../tools/test_data/","test_file":"test_file_name"}]}
Or something similar from an API. (I think it's unlikely to be XML, since XML doesn't implicitly have 'arrays' in the way JSON does).

Referenced array dropped in size to one element

Dear fellow perl programmers,
I wanted to access to this array
my #vsrvAttribs = qw(
Code
Description
vsrv_id
vsrv_name
vsrv_vcpu_no
vsrv_vmem_size
vsrv_vdspace_alloc
vsrv_mgmt_ip
vsrv_os
vsrv_virt_platf
vsrv_owner
vsrv_contact
vsrv_state
);
through a variable composed of a variable and a string suffix, which of course led to the error message like this
Can't use string ("#vsrvAttribs") as an ARRAY ref while "strict refs" in use at cmdbuild.pl line 262.`
Therefore I decided to get the reference to the array through a hash
my %attribs = ( vsrv => #vsrvAttribs );
And this is the code where I need to get the content of aforementioned array
foreach my $classTypeKey (keys %classTypes) {
my #attribs = $attribs{$classTypeKey};
print Dumper(\#attribs);
}
It seems I can get the reference to the array #vsrvAttribs, but when I checked the content of the array with Dumper , the array have got only one element
$VAR1 = [
'Code'
];
Do you have any idea where could be the problem?
How do you store the array in a hash and access it later?
You need to store your array by reference like this:
my %attribs = ( vsrv => \#vsrvAttribs );
Note the backslash before the # sigil. This tells perl that you want a reference to the array.
Then when access the array stored in $attribs{vsrv} you need to treat it as a reference instead of as an array. You'll do something like this:
foreach my $classTypeKey (keys %classTypes) {
# make a copy of the array by dereferencing
my #attribs = #{ $attribs{$classTypeKey} };
# OR just use the array reference if profiling shows performance issues:
my $attribs = $attribs{$classTypeKey}
# these will show the same thing if you haven't done anything to #attribs
# in the interim
print Dumper(\#attribs);
print Dumper($attribs);
}
Why did you only get one value and where did the rest of the array go?
Your missing values from #vsrvAttribs weren't lost they were assigned as keys and values to %attribs itself. Try adding the following just after you made your assignment and you'll see it for yourself:
my %attribs = ( vsrv => #vsrvAttribs );
print Dumper(\%attribs);
You'll see output like this:
$VAR1 = {
'vsrv_contact' => 'vsrv_state',
'vsrv_virt_platf' => 'vsrv_owner',
'vsrv' => 'Code',
'vsrv_name' => 'vsrv_vcpu_no',
'vsrv_mgmt_ip' => 'vsrv_os',
'Description' => 'vsrv_id',
'vsrv_vmem_size' => 'vsrv_vdspace_alloc'
};
This is because perl interpreted your assignment by expanding the contents #vsrvAttribs as multiple arguments to the list literal ():
my %attribs = (
# your key => first value from array
vsrv => 'Code',
# subsequent values of the array
Description => 'vsrv_id',
vsrv_name => 'vsrv_vcpu_no',
vsrv_vmem_size => 'vsrv_vdspace_alloc',
vsrv_mgmt_ip => 'vsrv_os',
vsrv_virt_platf => 'vsrv_owner',
vsrv_contact => 'vsrv_state',
);
This is legal in perl and there are reasons where you might want to do this but in your case it wasn't what you wanted.
Incidentally, you would have been warned that perl was doing something that you might not want if you had an even number of elements in your array. Your 13 elements plush the hash key "vsrv" makes 14 which is even. Perl will take any list with an even number of elements and happily make it into a hash. If your array had another element for 15 elements total with the hash key you would get a warning: Odd number of elements in hash assignment at foo.pl line 28.
See "Making References" and "Using References" in perldoc perlreftut for more information.
If you use a bare array in a hash definition like
my %attribs = ( vsrv => #vsrv_attribs )
the array is expanded and used as key/value pairs, so you will get
my %attribs = (
vsrv => 'Code',
Description => 'vsrv_id',
vsrv_name => 'vsrv_vcpu_no',
vsrv_vmem_size => 'vsrv_vdspace_alloc',
...
)
The value of a Perl hash element can only be a scalar value, so if you want an array of values there you have to take a reference, as shown below
It is also a bad idea to use capitals in Perl identifiers for anything except globals, such as package names. Local names are conventional lower-case alphanumeric plus underscore, so $class_type_key instead of $classTypeKey
use strict;
use warnings;
use Data::Dumper;
my #vsrv_attribs = qw(
Code
Description
vsrv_id
vsrv_name
vsrv_vcpu_no
vsrv_vmem_size
vsrv_vdspace_alloc
vsrv_mgmt_ip
vsrv_os
vsrv_virt_platf
vsrv_owner
vsrv_contact
vsrv_state
);
my %attribs = (
vsrc => \#vsrv_attribs,
);
for my $class_type_key (keys %attribs) {
my $attribs = $attribs{$class_type_key};
print Dumper $attribs;
}
output
$VAR1 = [
'Code',
'Description',
'vsrv_id',
'vsrv_name',
'vsrv_vcpu_no',
'vsrv_vmem_size',
'vsrv_vdspace_alloc',
'vsrv_mgmt_ip',
'vsrv_os',
'vsrv_virt_platf',
'vsrv_owner',
'vsrv_contact',
'vsrv_state'
];

Resources