Searching in a JSON array with Perl - arrays

I need to search for a specific term in a JSON array and get the index of the array element which contains the term as its value of a key, below is a sample JSON file that I'm working on, I need to get the array element whose "artifact_id" is "jar", so in this case I need to get the index of the array which is 1, the "artifact_id" is guaranteed to be unique:
{
"maven": [
{
"version": "1.2",
"artifact_id": "zip"
},
{
"version": "1.2",
"artifact_id": "jar"
}
]
}
I'm using the JSON lib and can decode the JSON to a Perl object, anyone can suggest the next step for me? Thanks a lot.
my $json_text = do {
open( my $json_fh, "<:encoding(UTF-8)", $filename[0] )
or die("Can't open \$filename\": $!\n");
local $/;
<$json_fh>;
};
my $json = JSON->new;
my $json_data = $json->decode($json_text);

Simply iterate through your array, and store away the index once the matching node is found. The example below should speak for itself, but given the contents of your post I guess you should read up on loops, comparitive operators, as well as basic programming.
After you have decoded your JSON-data, there is nothing special about iterating over the array, after all; it is just an array (reference given the tree like structure of JSON).
use JSON qw(decode_json);
my $needle = "jar";
my $needle_location;
my $data = decode_json( join '', <DATA> );
while( my( $idx, $elem ) = each( #{$data->{'maven'}} ) )
{
if( $elem->{'artifact_id'} eq $needle )
{
$needle_location = $idx;
last;
}
}
die "unable to find entry" unless defined $needle_location;
print "needle_location: $needle_location\n";
__DATA__
{
"maven": [
{
"version": "1.2",
"artifact_id": "zip"
},
{
"version": "1.2",
"artifact_id": "jar"
}
]
}
needle_location: 1

My answer is based on that of Filip Roséen - refp, but hopefully a more advanced and abbreviated approach:
use Modern::Perl;
use List::Util qw(first);
use JSON qw(decode_json);
my $needle = "jar";
my $data = decode_json do { local $/; <DATA> };
my $needle_location = first( sub {
my $artifact_id = $data->{ maven }->[ $_ ]->{ artifact_id };
defined($artifact_id) && $artifact_id eq $needle
}, 0 .. $#{ $data->{ maven } } ) //
die "unable to find entry";
say "needle_location: $needle_location";
__DATA__
{
"maven": [
{
"version": "1.2",
"artifact_id": "zip"
},
{
"version": "1.2",
"artifact_id": "jar"
}
]
}

Related

Remove duplicates from different JSON Arrays

I have a JSON structure that has a lot of arrays.I want to check if there are duplicates.Not inside the same array but in different arrays.Also i want the other fields to stay as is.
Here is an example of my structure:
{
"Collection":[
{
"field0":"string",
"field1":"string",
"field2":"string",
"field3":"string",
"field4":"string",
"field5":"string",
"field6":"string",
"field7":"string",
"field8":"string",
"field9":[
"test1"
"test2"
"test3"
]
},
{
"field0":"string",
"field1":"string",
"field2":"string",
"field3":"string",
"field4":"string",
"field5":"string",
"field6":"string",
"field7":"string",
"field8":"string",
"field9":[
"test8"
"test2"
"test9"
]
}
]
}
And here is what I expect:
{
"Collection":[
{
"field0":"string",
"field1":"string",
"field2":"string",
"field3":"string",
"field4":"string",
"field5":"string",
"field6":"string",
"field7":"string",
"field8":"string",
"field9":[
"test1"
"test2"
"test3"
]
},
{
"field0":"string",
"field1":"string",
"field2":"string",
"field3":"string",
"field4":"string",
"field5":"string",
"field6":"string",
"field7":"string",
"field8":"string",
"field9":[
"test8"
"test9"
]
}
]
}
I don't know if this is relevant but this is a firestore collection.
O'k, I don't know Swift, but I can show you how to do it:
<?php
$json = <<<JSON
{
"Collection":[
{
"field1":[
"test1",
"test2",
"test3"
]
},
{
"field2":[
"test8",
"test2",
"test9"
]
}
]
}
JSON;
$entities = json_decode($json, true); // Decode JSON
$collection = $entities['Collection']; // Grab array of fields inside collection
$elements = []; // Initialize an empty array of unique elements
$result = [];
foreach ($collection as $index => $fieldObject) {
$fieldName = array_keys($fieldObject)[0]; // Get field name
// Get value from array of values of this field
foreach ($fieldObject[$fieldName] as $valueKey => $value) {
// Check if your value is not in array of unique elements
if (!in_array($value, $elements)) {
$elements[] = $value; // Add value if is not
// Add value to your new array
$result['Collection'][$index][$fieldName][] = $fieldObject[$fieldName][$valueKey];
}
}
}
$result = json_encode($result); // Encode it back to JSON
Here is the working example in sandbox: http://sandbox.onlinephpfunctions.com/code/a78cff49cc31c15e7e1373f7e1f66b7951f129e9

Delete objects and arrays with jq which match a key

I have a JSON with the following content:
{
"data": [
{
"name": "Test",
"program": {
"publicAccess": "--------",
"externalAccess": false,
"userGroupAccesses": [
{
"access": "r-------"
},
{
"access": "rw------"
}
],
"id": "MmBqeMLIC2r"
},
"publicAccess": "rw------"
}
]
}
And I want to delete all keys (recursively) which match publicAccess or userGroupAccesses, so that my JSON looks like this:
{
"data": [
{
"name": "Test",
"program": {
"externalAccess": false,
"id": "MmBqeMLIC2r"
}
}
]
}
I've copied jq's builtin walk function from source.
# Apply f to composite entities recursively, and to atoms
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
# My Code
walk(if (type == "object" and .publicAccess)
then del(.publicAccess)
elif (type == "array" and .userGroupAccesses)
then del(.userGroupAccesses)
else
.
end )
Gives me jq: error (at <stdin>:2622): Cannot index array with string "userGroupAccesses". Also if I use .userGroupAccesses[] - How do I get the result?
Snippet on jqplay: https://jqplay.org/s/1m7wAeHMTu
Your problem is when type == "array" is true . will be an array so .userGroupAccesses won't work. What you want to do is focus on the case when . is an object. In your call to walk you only need to check for type == "object" and then remove the members you don't want. e.g.
walk(if type == "object" then del(.publicAccess, .userGroupAccesses) else . end)
Try it online at jqplay.org
You can also solve this without walk by using Recursive Descent .. e.g.
del(.. | .publicAccess?, .userGroupAccesses?)
Try it online at jqplay.org

Clean way to access a nested data structure

I have a segment of code that, although it works, does not look like a clean way to do things.
I build the structure using:
foreach my $n (#node_list)
{
chomp ($n);
foreach my $c (#cpes)
{
my #returned; #Interfaces to CPEs with MED settings
my #creturned; #General Customer Interfaces
my ($cust) = $c =~ /([a-zA-Z]+)[_-][a-zA-Z0-9]+/s;
print "\n\t\tCustomer is $cust\n";
chomp($c);
$c = uc $c;
my ($search) = $c;
(#returned) = `cat /curr/$n | grep "$search"`;
if (#returned)
{
my $cust_match = 'interface \"' . $cust;
(#creturned) = `cat /curr/$n | egrep -i "$cust_match" | grep -v "$search"`;
}
if (#creturned) #Have we found other CPEs on the same router
{
my ($nf) = $n =~ /([a-zA-Z0-9-]+).cfg/s;
my (#interfaces) = map { /([A-Z0-9_]+)/s } #creturned;
#interfaces = uniq(#interfaces);
unshift (#interfaces, $c);
push (#new_out, {$nf => {$cust => [#interfaces]}});
}
}
This will return:
$VAR1 = [
{
'router-xx-xx' => {
'50000' => [
[
'THXXXXVF_NLXXXX40_1121_2',
'10x.xx.x.50'
],
[
'THXXXPVF_NLXXXX66_1121_1',
'10x.xx.x.70'
],
[
'THXXXXVF_NLXXXX67_1121_2',
'10x.xx.x.78'
],
}
},
Each router can have a number of VPRNs and each VPRN can contain multiple interfaces. In the example above I've shown one router with one VPRN.
However, when it comes to accessing elements in the above, I've written the following convoluted (but working) code:
foreach my $candidate (#nodes)
{
my %node = %{ $candidate };
foreach my $n (keys %node)
{
print "\nRouter is $n\n";
foreach my $cust (keys %{ $node{$n} })
{
print "Customer on $n is \n" . Dumper $cust;
my #intlist = #{$node{$n}{$cust}};
my $med_cpe = $intlist[0]; #the CPE that was used to find node
{truncated}
}
}
}
}
You don't explain exactly what you find "convoluted" about the traversal code, but you have made it unnecessarily complex by duplicating data into #intlist and %node. The excessive and inconsistent indentation also makes it ungainly
I would write something closer to this
for my $node ( #nodes ) {
for my $n ( keys %$node ) {
print "\nRouter is $n\n";
for my $cust ( keys %{ $node->{$n} } ) {
print "Customer on $n is \n" . Dumper \$cust;
my $med_cpe = $node->{$n}{$cust}[0];
}
}
}
If you don't need the values of $node and $n except to access $med_cpe then you don't need a nested data structure at all: a simple array is fine. On the face of it, an array like this will do what you need
[
[
'router-xx-xx',
'50000',
'THXXXXVF_NLXXXX40_1121_2',
'10x.xx.x.50',
],
[
'router-xx-xx',
'50000',
'THXXXPVF_NLXXXX66_1121_1',
'10x.xx.x.70',
],
...
]

Array of Hashes to JSON File Ruby

I'm trying to create JSON files dynamically, where the content of each file belongs only to a certain entity.
I have an array of Ruby objects, where each object represents an entity.
entities = [#<Entity:0x0055f364d9cd78 #name="entity-B", #internal_asn_number=64514, #classification_id="B">,#<Entity:0x0055f364d89070 #name="entity-A", #internal_asn_number=64513, #classification_id="A">]
And I have an array of hashes, which contains several rules for each entity and I would like to write these hashes to JSON files, one file per entity.
I have the following code:
array_hashes = [{"rulename"=>"entity-B_source_fqdn_1", "if"=>{"source.fqdn"=>"mail.tec.dsr.entityB.com"}, "then"=>{"event_description.text"=>"B"}}, {"rulename"=>"entity-B_destination_fqdn_1", "if"=>{"destination.fqdn"=>"mail.tec.dsr.entity_B.com"}, "then"=>{"event_description.text"=>"B"}}, {"rulename"=>"entity-A_source_fqdn_1", "if"=>{"source.fqdn"=>"194-65-57-128.entityA.com"}, "then"=>{"event_description.text"=>"A"}}, {"rulename"=>"entity-A_destination_fqdn_1", "if"=>{"destination.fqdn"=>"194-65-57-128.entityA.com"}, "then"=>{"event_description.text"=>"A"}}]
path = "/home/mf370/Desktop/RubyProject/"
file_name = "_url_fqdn.conf"
entities.each do |entity|
File.open(path + entity.name + file_name, "w") do |f|
array_hashes.each do |rule|
if rule['rulename'].match(entity.name)
f.write(JSON.pretty_generate([rule]))
end
end
end
This code works and creates the files dynamically, however the content of the files is not what I was expecting.
This is the output from the code above:
[
{
"rulename": "entity-A_source_fqdn_1",
"if": {
"source.fqdn": "194-65-57-128.entityA.com"
},
"then": {
"event_description.text": "A"
}
}
][
{
"rulename": "entity-A_destination_fqdn_1",
"if": {
"destination.fqdn": "194-65-57-128.entityA.com"
},
"then": {
"event_description.text": "A"
}
}
]
And this is the output that I was looking for:
[
{
"rulename": "entity-A_source_fqdn_1",
"if": {
"source.fqdn": "194-65-57-128.entityA.com"
},
"then": {
"event_description.text": "A"
}
},
{
"rulename": "entity-A_destination_fqdn_1",
"if": {
"destination.fqdn": "194-65-57-128.entityA.com"
},
"then": {
"event_description.text": "A"
}
}
]
Probably this is very simple to solve, but I ran out of ideas on how to solve this, I'm new to Ruby Programming. Thank you for your help!
The problem is that you pack each rule inside an array :
JSON.pretty_generate([rule])
You should create an array of matching rules, and dump it as JSON :
entities.each do |entity|
File.open(File.join(path, entity.name + file_name), "w") do |f|
matching_rules = array_hashes.select{ |rule| rule['rulename'].match(entity.name) }
f.write(JSON.pretty_generate(matching_rules))
end
end
The Problem is that you wrap each rule in an array.
THis should solve the issue:
json_entities = []
entities.each do |entity|
File.open(path + entity.name + file_name, "w") do |f|
array_hashes.each do |rule|
if rule['rulename'].match(entity.name)
json_entities << rule
end
end
end
end
f.write(JSON.pretty_generate(json_entities))

Echo "part" of array (decoded from JSON)

I'am quite new to JSON and more "advanced" arrays. Therefore I don't know what I should search for...
I have this "JSON array" (what do you call it?):
{
"id": "321123321",
"statuses": {
"data": [
{
"message": "testmessage",
"updated_time": "2012-12-25T16:33:29+0000",
"id": "123321123"
}
],
"paging": {
"previous": "1",
"next": "1"
}
}
}​
I want to create a variable from "message" that is called $message and a variable from "up_datedtime" that is called $updated.
To get id I simple:
$json_a=json_decode($string,true);
$id $json_a['id'];
And for statuses:
$json_a=json_decode($string,true);
$status = $json_a['id']['statuses'];
But when I try to get "message" I get " Cannot use string offset as an array in":
$message = $json_a['id']['statuses']['data']['message'];
How do I get $message from the array the proper way?
You can get like this
$message = $json_a['id']['statuses']['data'][0]['message'];
or you can get from loop
$dataArr = $json_a['id']['statuses']['data'];
foreach ($dataArr as $val) {
echo "message".$val['message'];
}

Resources